250 likes | 394 Views
Standards, Industry, and the Roadmap to Grid Adoption. Dr. David Snelling Vice Chair of Standards Global Grid Forum / Fujitsu Labs Europe. Motivation. Need for Standards Stability, Choice, Flexibility, Competition, Collaboration, ... To Develop Standards we Need Clarity
E N D
Standards, Industry, and the Roadmap to Grid Adoption Dr. David Snelling Vice Chair of Standards Global Grid Forum / Fujitsu Labs Europe
Motivation • Need for Standards • Stability, Choice, Flexibility, Competition, Collaboration, ... • To Develop Standards we Need Clarity • Definitions of concepts • Organization of work through Architectural Frameworks • We also Need a Roadmap • Accelerate the development of the “right” specifications • Track gaps and requirements • Demonstrate progress • Support planning in industry and research
Notions of Grid • Collaboration Grids • Multiple institutions, secure, widely distributed, VOs • Service level agreements & commercial partnerships • Business model: Increase overall revenue • Enterprise Grids • Virtualization of enterprise resources and applications • Aggregation and centralization of management • Business model: Reduce total cost of ownership • Clusters • Networks of Workstations, Blades, etc. • Cycle scavenging, Homogeneous workload • Business model: Lower marginal costs • Parallel Processing Systems • Parallel processing for single applications Increasing Complexity and Revenue
Parallel Processing and Cluster Grids • Parallel Processing • Tightly coupled distributed systems • Standards: • MPI and OpenMP • Aimed at HPC • Code portability and performance! • Cluster Grids • Loosely coupled distributed systems • Efficient scheduling of nodes for throughput • No standards, lots of players • Queuing systems: LSF, PBS, LoadLeveler, ... • Specialist systems: CyberGRIP, gridMatrix, ...
Enterprise Grids Today • Enterprise Grids are about • Virtualization: Uniform encapsulation of resources: • Compute, data, applications, support, ... • Integration: Creation of a structured whole from the parts. • Automation: Most management tasks, mostly automatic. • Examples • Fujitsu’s Triole Strategy • Oracle’s 10g Platform • Sun’s N1 Suite • HP’s Adaptive Enterprise • IBM’s “On Demand” Business • Run your required services asefficiently as possible.
Collaboration Grids Today • Production First Generation Collaboration Grids • UK National Grid Service and TeraGrid • Running Globus GT2 • Team Shosholoza and others • Running Unicore • Web Service Collaboration Grids • Experimental Deployment • Globus GT4, Unicore/GS • Barriers • Confusion wrt Plain Web Services • Politics of the Standards Process • Create new business opportunities throughcollaboration • Enterprise Grid technology as a basis. • Requirements beyond Enterprise Grids: • Discovery, Security, Virtual Organizations (VOs),Decoupling, Composition ...
Convergence: Enterprise & Collaboration Grids • Technical Convergence • From Enterprise Grids • Sophisticated virtualization • Management infrastructure • Automation • From Collaboration Grids • Multi-domain security • Cyber partnerships (VOs) • Outsourcing • The Need for Standards • Within the Enterprise • Flexibility! • Between Enterprises • Interoperability! • Forrester’s • “Digital Business Networks”† † http://www.forrester.com/go?docid=38314
GGF and the Nature of Interoperability • GGF is about • Enabling the pervasive adoption of grid computing for research and industry by: • Defining grid specifications that lead to broadly adopted standards and interoperable software • Fostering and broadening an international community for the exchange of ideas, experiences, requirements, and best practices • Implicit process: • Requirements Specifications Standards Interoperability • Note: Implementations are required do do the last three steps well. • Definitions: • Specifications: Normative document sufficient for implementation • Standards: Specifications plus an open process.
Interoperability • In a SOA context, this is very precise • Implementations interact “on the wire” between different implementations, languages, and environments • WS-SOA Offers Unprecedented QoS in this respect • Better than http, not quite as good as hardware • Only possible by agreeing on a single specification • For GGF this specification is an Open Standard
Interoperation • Adaptor Based Interaction Possible • A simple service wrapper for each client type • e.g. JSDL to Unicore AJO to Globus JDL converters • Service composer frameworks possible • e.g. NAREGI Grid composes Unicore, GT2, GT4, and WSs • There is a Notion of “Abstract Service Equivalence” • OGSA V1.0 and V1.5 are instances of this • Greatly facilitates adaptor development and deployment • Language specific standards help build better adaptors • e.g. a Java API for the OGSA Base Profile or SAGA API. • If all clients (or services) implement adaptors for all services (or clients) it creates a pleasant illusion of interoperability
End User & Technology Community Use Cases and Requirements Solutions and Building Blocks Create Value Deliver Value Standards Groups/Orgs Vendor and Open Source Communities Architectures and Specifications The GGF Roadmap Process Communicate status and progress Manage and steer standards development Input to implementation & deployment planning
Roadmap Organization • Organized by Area, Group, and then Document • Content for each Document • Document name and short description • GGF Document Type • Progress against key millstones • Planned and completed dates for First Draft, Public Comment, and publication • Key Words • Informs Grid Design, Defines Grid Architecture, OGSA, Applications, Generic grid Component, Other, ... • Adoption Levels • Unimplemented, Implemented, Interoperable, Community, Adopted, and Ubiquitous.
Adoption Level Definitions • Unimplemented • Although the specification exists and may be viewed as stable, no implementation exists. There may be prototypes under development within various organizations, which are not available outside that organization. • Implemented • There exists at least one implementation that is generally available for testing and/or deployment that according to the authors (or third parties) implement the specification. • Interoperable • There exists at least two implementations, as defined above, that interoperate. There must be a report detailing at least one interoperability workshop.
Adoption Level Definitions Continued • Community • At least one of the interoperable implementations, as defined above, is deployed and used on a regular basis by a specific community. This may be due to either a lack of acceptance of the specification by the community at large or due to the specialist nature of a specific specification. • Adopted • There exists more than one interoperable implementation, as defined above, and each implementation is used across several communities. Commercially supported implementations are available. This may be either as a product or support for an open source implementation. There may be some restriction on which platforms support the implementations or other aspects that restrict the availability of the implementations. • Ubiquitous • Interoperable implementations exist for virtually all platforms. Commercial support is available, but provided transparently as part of the supporting infrastructure.
Some Roadmap Statistics • Roadmap Documents by Type • Recommendation Documents 26 • Informational Documents 30 • Experimental Documents 3 • Roadmap Documents by Area • Applications 9 • Architecture 6 • Compute 9 • Data 13 • Infrastructure 6 • Management 9 • Security 7
Some More Statistics • Published Documents • Compute/SRM 6 • Data 10 • Architecture 7 • Applications/APME 7 • Infrastructure/ISP/P2P 8 • Security 10 • Management 2 • GFSG 5 • Published Draft-Recommendations Documents 9
The Current Pipeline 18 Documents in 12 Months • Statistics: • Published since GGF 15 9 • In or after Public Comment 22 • Others in the pipeline 5 • Publication Highlights • GFD.53: OGSA Roadmap • GFD.56: JSDL 1.0 • GFD.58: Namespaces for XML Infosets • GFD.59: OGSA Profile Definition • Progress Highlights • GWD.xx: WSRF OGSA Base Profile through Public Comment • GWD.xx: WS-Agreement through Public Comment • Highlights from Public Comment • GWD.xx: ByteIO Suite - 2 specs • GWD.xx: DAI Suite - 3 specs
OGSA: Status November 2004 Warning: Data may be inaccurate SYSTEMS MANAGEMENT GRID COMPUTING UTILITY COMPUTING Use Cases & Applications Distributed query processing Data Centre Collaboration Persistent Archive ASP Multi Media VO Management OGSA Self Mgmt OGSA-EMS WS-DAI Information WSDM Discovery GGF-UR WS-BaseNotification Naming Core Services Privacy Trust GFD-C.16 WSRF-RP WSRF-RL Data Model WSRF-RAP WS-Security SAML/XACML X.509 Base Profile WS-Addressing HTTP(S)/SOAP WSDL CIM/JSIM Data Transport Hole Gap Evolving Standard
OGSA: Status February 2006 (or soon) Warning: Data may be inaccurate SYSTEMS MANAGEMENT GRID COMPUTING UTILITY COMPUTING Use Cases & Applications Distributed query processing Data Centre Collaboration Persistent Archive ASP Multi Media VO Management OGSA Self Mgmt OGSA-EMS WS-DAI Information WSDM Discovery GGF-UR WS-BaseNotification Naming Core Services Privacy Trust GFD-C.16 WSRF-RP WSRF-RL Data Model WSRF-RAP WS-Security SAML/XACML X.509 Base Profile WS-Addressing HTTP(S)/SOAP WSDL CIM/JSIM Data Transport Hole Gap Evolving Standard
Implementations of GGF Specifications • GFD.56: JSDL 6 • GFD.62: PMA Charter 3 • GFD.24: GSSAPI extensions 6 • GFD.15: OGSI 5 • GFD.20: GridFTP 5 • GFD.52: GridRPC API 4 • GFD.22: DRMAA 4
Implementations of GGF Drafts • GWD.xx: SAML authorization callout 3 • GWD.xx: VOMS attribute certificate format 4 • GWD.xx: Daonity 1 • GWD.xx: OGSA BES 2 • GWD.xx: GGF Usage Record 4 • GWD.xx: Usage Record Service 4 • GWD.xx: WS-Agreement 6 • GWD.xx: OGSA Byte IO 2 • GWD.xx: WS-Naming 1 • GWD.xx: SAGA 4
Implementations of GGF Drafts • GWD.xx: CDDLM Smart Frog Language 1 • GWD.xx: CDDLM Component Model 4 • GWD.xx: CDDLM Deployment API 4 • GWD.xx: CDDLM XML-CDL 4 • GWD.xx: ACS 2 • GWD.xx: WSRF OGSA Base Profile 3 • GWD.xx: OGSA BSP Core 3 • GWD.xx: OGSA BSP Secure Channel 3
Other Implementations • GGF Derived Specifications • RFC3820 5 • WSRF 5 • WSN 5 • GFD.16 Certificate Policy Model 40+
Summary • 103 Implementations of GGF Specifications • The pipeline is still flowing • Thanks Greg! • More help is (always) needed • Give yourselves a hand. • Thank you