Deliverable D3.2Deliverable D3.12

Detailed planning for allExecutive Summary

the tools and services including use cases for Task 3.WP3

Executive Summary for Task 3.3

WP3 New Grid Services and Tools

Document Filename: / CG-3.3-D3.2SDD-0100v1.10-TCD-ExecutiveSummary.docCG-WP3-0010 Abstract
Work package: / WP3 New Grid Services and Tools
Partner(s): / TCD, PSNC, CYFRONET, ICM, UCY, DATAMAT, TCD, CSIC, UAB, ALGO
Lead Partner: / TCDPSNC
Config ID: / CG-3.3--D3.2-v1.10SDD-TCD--0100 Executivee SummaryCG-WP3-SRS-0011 Abstract
Document classification: / PUBLIC
CG3.3-D3.2-v1.1-TCD-ExecutiveSummaryCG-3.3-SDD-0100 Executive SummaryCG-3.3-SDD-0100 Executive SummaryCG-3.3-SDD-0100.doc Executive SummaryCG-WP3-SRS-0011 Abstract / PUBLIC / 1 / 6
/ Deliverable D3.1 2 Detailed planning for all the tools and services including use cases for WP3
Executive Summary for Task 3.3
CG3.3-D3.2-v1.1-TCD-ExecutiveSummaryCG-3.3-SDD-0100 Executive SummaryCG-3.3-SDD-0100 Executive SummaryCG-3.3-SDD-0100.doc Executive SummaryCG-WP3-SRS-0011 Abstract / PUBLIC / 1 / 6
/ Deliverable D3.2 Executive Summary
Executive Summary for Task 3. 3Deliverable D3.1 Detailed planning for all the tools and services including use cases for WP3
Delivery Slip
Name / Partner / Date / Signature
From / WP3, Task 3.3 / TCD, CYFRONET, ICM / Aug 8th, 2002 / Brian Coghlan
Verified by
Approved by
Document Log
Version / Date / Summary of changes / Author(s)
1-0 / Aug 8th, 2002 / Summary of Design Documents coming from WP3 Task 3.3 / Brian Coghlan
1-1 / Aug 30th, 2002 / Added Appendix, fixed filename and config ID, modified text after internal review. / Brian Coghlan

AbstractExecutive Summary for WP3 Task 3.3

CrossGrid workpackage WP3 includes a set of tools and services, which will define (with the results of WP2) the middleware layer of the CrossGrid project. WP3 incorporates five Tasks, of which Task 3.3 (Grid Monitoring) is one.

The main objective of the workpackageWP3Task 3.3 (New Grid Services and Tools) is to develop Grid services and a software infrastructure required to support the monitoring needs of Grid users, applications and tools as defined in the workpackages WP1 and WP2. This workpackage includes a set of tools and services, which will define (also including the results of WP2) the middleware layer of the CrossGrid project. Additionally the workpackage includes extra tasks concerning tests and integration as well as the co-ordination. The results of Task 3.3 will improve the existing grid middleware by delivering both new kinds of monitoring services and enhancements to the existing DataGrid R-GMA monitoring services. The formal list of tasks in WP3 (Fig. 1):

Task 3.0 Co-ordination and management

Task 3.1 Portals and roaming access

Task 3.2 Grid resource management

Task 3.3 Grid monitoring

Task 3.4 Optimisation of data access

Task 3.5 Tests and integration

Fig. 1 Positioning of the WP3 results – TOOLS and services

Hence Tasks 3.31 ... 3.4 will develop a middleware software , in the form of (Fig. 2) delivered as tools (e.g. portals, roaming access) andgrid monitoring services (e.g. schedulers, remote data access, grid monitoring). In general the results of WP3 will improve the existing grid middleware by delivering new kind of tools and services or by enhancing the existing. Tasks 3.5 will integrate the tools in a common setup version, releasing it and making publicly available after tests performed in the testbed environment.

Fig. 2 The support of Grid services done by CrossGrid WP3. The workpackage WP3 will enhance the idea of developing Grid services

Thee first deliverable (D3.1) in Month 3 is released in a report manner (according to the CrossGrid Annex 1: contained theofSoftware Requirements Specification (SRS), with detailed planning for all the tools and services, including including use cases for:

  • requirements of the end user;

definition of the monitoring environment functionality and of interfaces to Task 2.4

specification of GRM agents

  • review of state-of-the-art and current related techniques;
  • use cases..

The definition of the end user is different depending on the task, e.g. for portals and roaming access (3.1) the end users requirements are described by applications (workpackage WP1). We organised a set of meetings and teleconferences with each application team, discussing the real requirements, work- and dataflow, trying to extract all common rules. The general rules and behaviour will define the bottom layer. Advanced and specific requirements will be delivered on the basis of the bottom layer. On the other hand , the end user of WP3 tasks is also defined by other workpackages (e.g. task 2.4 will directly use the results from Grid Monitoring 3.3), which means the requirements have also been taken into account while defining this report.

The deliverable of Month3 consists of the Software Requirements Specification (SRS) done for each task: 3.1..3.5, following this introduction. This current deliverable (D3.2) e SRS documents will also be enhanced later (deliverable no. D3.2) before the Month6 by definesing the architecture and,interfaces, and security issues. Therefore some parts of these documents may not be filled out and described as TBD (to be done) based on the SRS.

The goals of Task 3.3 are to develop:

  • Subtask 3.3.1 OCM-G[CYFRONET]
  • Subtask 3.3.2 SANTA-G[TCD]
  • Subtask 3.3.3 Jiro-based-monitoring[CYFRONET]
  • Subtask 3.3.4 Analysis[ICM]

OCM-G is a new service designed to enable monitoring of application tasks. SANTA-G extends the DataGrid R-GMA for ad-hoc non-invasive monitoring of grid components using external instruments. Jiro-based-monitoring is a new service for routinely monitoring grid components using built-in instrumentation. Analysis refers to in-depth analysis of information acquired from the enriched monitoring environment.

Since each of these represents a complex subsystem that extends the common monitoring environment in different ways, their design is discussed in four separate documents:

  • Subtask 3.3.1 OCM-G:CG3.3.1-D3.2-v1.0-CYF021-OCMGDesign.doc
  • Subtask 3.3.2 SANTA-G:CG3.3.2-D3.2-v1.2-TCD020-SantaGDesign.doc--SDD-SDD-01000100
  • Subtask 3.3.3 Jiro-based-monitoring: CG3.3.3-D3.2-v1.0-CYF022-JiroDesign.doc
  • Subtask 3.3.4 Analysis:CG3.3.4-D3.2-v1.0-ICM007-ICMSystemDesign.doc The purpose of the SRS 3.1 document is to define the requirements for the Roaming Access, Migrating Desktop and user portals. In detail, this document will provide a description of requirements for the Migrating Desktop and Application portal, which will allow the user to access his environment from remote computers. It will provide some requirements and problems for other CG WPs.

The SRS 3.2 describes the requirements of Scheduling Agents for parallel applications submitted to a grid environment. Scheduling Agents are schedulers, which make decisions about where, when, and how to run parallel jobs on Grid resources as specified by a set of policies, priorities, requirements and limits. They are responsible for allocating application tasks to computing resources in a way that tries to guarantee that applications are conveniently, efficiently and effectively executed. Agents will interact with other services developed in the CrossGrid project and this specification is aimed at being compatible with other on-going grid projects (namely, the Datagrid).

The SRS 3.3 document specifies the software requirements including those for the OCM-G monitoring system for Grid applications, additional services for non-invasive monitoring, and the Jiro-based services for Grid-infrastructure monitoring. The intended audience is both the Task itself and dependent tasks.

The SRS 3.4 document is aimed at describing requirements which inspire and influence the form of the Data Access Package. The document is intended to be a base for the final software design for task 3.4 and its co-operation with other tasks. It should be helpful to formulate a coherent vision of the CrossGrid environment. It presents the proposed architecture for data-access and optimisation problems for the CrossGrid Project. Dependencies between the system components and connections between them are shown with some more detailed specifications of interfaces. The architecture relies mainly on SOAP and GridFTP protocols with possible extensions to OGSA when it becomes available. The programming languages and libraries are mentioned as well.

The SRS 3.5 provides an overall description of the tests and the integration process, details the composition of the integration team and provides user and administrator support from the Integration Team.

To resumesummarize, the associated current version of SRS Design Ddocuments specifydescribes the software requirement design for each subtask of WP3 Ttask 3.3, including those for the OCM-G monitoring system for Grid applications, additional R-GMA services for non-invasive monitoring, the Jiro-based services for Grid-infrastructure monitoring, and an analysis system that exploits these. The intended audience is both the Task itself and dependent tasks.gives an input set of requirements to other workpackages, especially WP2 (Grid Application Programming Environment) and WP4 (International Testbed Organisation). Additionally the deliverable D3.1 can be treated as the first step towards the definition of the WP3 architecture and interfaces. The deliverable D3.1 includes a set of documents for each tasks 3.1...3.5, including in separate documents Software Specification Requirements (SRS) and remarks of last improvements done after the internal review process. There is also an overview of the current state of the art for each tasks (except the Grid monitoring, which is based on the overview done by the DataGrid project).

Appendix

For reference here is an overview of the Task 3.3 components:

This task will extend the Grid information system content to include three of the major sources of performance data: applications, instruments and infrastructure.

The products of Task 3.3 are:

(a)an OMIS-based application monitoring system, OCM-G,

(b)additional services, SANTA-G, for ad-hoc non-invasive monitoring, and

(c)Jiro-based services for Grid-infrastructure monitoring.

Figure 1 shows these components:


Figure 1 The Grid Monitoring System

OCM-G is a distributed monitoring system for obtaining information on and manipulating parallel distributed applications. The purpose of this system is to provide a basis for building tools supporting parallel application development.

SANTA-G services are a specialized non-invasive complement to other more intrusive monitoring services. The application of these services will be in validation and calibration of both intrusive monitoring systems and systemic models, and also for performance analysis. The objectives are to allow information captured by external monitoring instruments to be introduced into the Grid information system, and to support analysis of performance using this information.

The Jiro-based services for Grid-infrastructure monitoring are intelligent components for obtaining information from and manipulating Grid hardware devices. The application of the software is to gather information from hardware devices, make autonomous decisions based on this information, and take necessary actions. The objectives are to allow the user to specify desirable logic for managing hardware. The addition of these three components will greatly expand the quality and quantity of the Grid information system content.

Please refer to Section 2.1 of the Task 3.3 SRS for a more detailed description of the use, and interaction of these three components

CG3.3-D3.2-v1.1-TCD-ExecutiveSummaryCG-3.3-SDD-0100 Executive SummaryCG-3.3-SDD-0100 Executive SummaryCG-3.3-SDD-0100.doc Executive SummaryCG-WP3-SRS-0011 Abstract / PUBLIC / 1 / 6
/ Deliverable D3.1 2 Detailed planning for all the tools and services including use cases for WP3
Executive Summary for Task 3.3
CG3.3-D3.2-v1.1-TCD-ExecutiveSummaryCG-3.3-SDD-0100 Executive SummaryCG-3.3-SDD-0100 Executive SummaryCG-3.3-SDD-0100.doc Executive SummaryCG-WP3-SRS-0011 Abstract / PUBLIC / 1 / 6
/ Deliverable D3.1 2 Detailed planning for all the tools and services including use cases for WP3
Executive Summary for Task 3.3

.

CG3.3-D3.2-v1.1-TCD-ExecutiveSummaryCG-3.3-SDD-0100 Executive SummaryCG-3.3-SDD-0100 Executive SummaryCG-3.3-SDD-0100.doc Executive SummaryCG-WP3-SRS-0011 Abstract / PUBLIC / 1 / 6