BOF: Globus Trends & Future Directions

1. BOF: Globus Trends & Future Directions Dan Fraser - fraser@mcs.anl.gov Director: Community Driven Improvement of Globus Software (CDIGS) Stuart Martin – smartin@mcs.anl.gov GRAM Project Chair

2. Outline • What is Globus? (Very brief) • Where is Globus going? • Making our efforts & roadmap even more accessible • http://wiki.mcs.anl.gov/CDIGS • Community building via Incubator Projects • Four planned pushes for next year • Technology directions • How can we work together? • Help direct the roadmap(s) • Use the technology • Contribute to the projects (docs & code)

3. A B 1 1 9 9 Shared Distributed Infrastructure Underlying Problem:The Application-Infrastructure Gap Dynamicand/orDistributedApplications

4. More Specifically, I May Want To … • Create a service for use by my colleagues • Manage who is allowed to access my service (or my experimental data or …) • Ensure reliable & secure distribution of data from my lab to my partners • Run 10,000 jobs on whatever computers I can get hold of • Monitor the status of the different resources to which I have access

5. Tool Tool Workflow Uniform interfaces, security mechanisms, Web service transport, monitoring Registry Credent. DAIS GRAM User Svc User Svc GridFTP Host Env Host Env Bridging the Application-Resource Gap User Application Database Specialized resource Computers Storage

6. Mobius Globus BPEL GRAM Globus myProxy OGSA-DAI Globus Toolkit GSI CAS caCORE Globus Cancer Biomedical Informatics Grid Functions Management Schema Management Metadata Management ID Resolution Workflow Security Resource Management Service Registry Service Service Description Grid Communication Protocol Transport Spans 60 NIH cancer centers across the U.S. Slide credit: Peter Covitz, National Institutes of Health

7. Globus Software: dev.globus.org Globus Projects OGSA-DAI GT4 MPICH G2 Java Runtime MyProxy Data Rep Replica Location Delegation GridWay CAS C Runtime GSI- OpenSSH GridFTP MDS4 Incubation Mgmt Python Runtime C Sec GRAM Reliable File Transfer GT4 Docs Incubator Projects Common Runtime Security Execution Mgmt Data Mgmt Info Services Other

8. Globus Software: dev.globus.org Globus Projects OGSA-DAI GT4 MPICH G2 Java Runtime MyProxy Data Rep Replica Location Delegation GridWay CAS C Runtime GSI- OpenSSH GridFTP MDS4 Incubation Mgmt Python Runtime C Sec GRAM Reliable File Transfer GT4 Docs Incubator Projects Swift GEMLCA RAVI MonMan GAARDS MEDICUS Cog WF Virt WkSp NetLogger GDTE GridShib OGRO UGP Dyn Acct Gavia JSC DDM Metrics Introduce PURSE HOC-SA LRMA WEEP Gavia MS SGGC ServMark Common Runtime Security Execution Mgmt Data Mgmt Info Services Other

9. Dev.Globus • Infrastructure for Globus development projects • Email lists, CVS repository, conflict resolution • Based on Apache Jakarta • All Globus projects go through this process • Control is in the hands of the project committers • You can make a difference: • Create a new Incubator Project • Visibility to all the Globus community • Leverage the capabilities of the Globus team • Become part of the Globus Distribution • Contribute to an existing project • Join the discussion via email • Request features, participate in roadmap reviews • Contribute to documentation

10. Where is Globus today? • http://incubator.globus.org/metrics • > 75, 000 GT4 downloads • > 95% are production downloads • Maintaining production quality code • Supporting the most important OGF standards • Innovating with new features • Incorporating Community Involvement

11. The Globus Community: A Sample OAK RIDGE NATIONAL LAB. OGSA-DAI TEAM OMII-EUROPE OMII-UK OPEN GRID COMPUTING ENVIRONMENTS PROJECT OPEN GRID FORUM GIN WG OPEN GRID FORUM JSDL WG OPEN GRID FORUM OGSA WGOPEN GRID FORUM BES WG OPEN GRID FORUM HPCP WG OPEN SCIENCE GRIDINTEGRATION TESTBED OPEN SCIENCE GRID VALIDATION TESTBED OPEN SCIENCE GRID VDT OASIS PEGASUS PROJECT PHILIPPS-UNIV. MARBURG PRAGMA PYGLOBUS PROJECT RWTH AACHEN UNIV. SANDIA NATIONAL LAB. SANTA CLARA UNIV. SOUTHERN CALIFORNIA EARTHQUAKE CENTER SEMANTIC BITS SHANGHAI JIAO TONG UNIV. SRM PROJECT SWEDISH INSTITUTE OF  TECHNOLOGY TECHNISCHE U. DRESDEN TECHNISCHE U. DELFT TERAGRID DATA WG TERAGRID NCSA RP APACHE APOLLO PROJECT APACHE HERMES PROJECT AUSTRALIAN PARTNERSHIP FOR ADV. COMPUTING BARCELONA SUPERCOMP. CENTER BILD UMEAA UNIV. CABIG CANADIAN FOREST SERVCIE CANARIE PROJECT CEDPS PROJECT CHINAGRID CONDOR TEAM COREGRID LABORATORY D-GRID DHIRUBHAI AMBANI INST. OF INFO. AND COMM. TECH. EARTH SYSTEM GRID ENABLING GRIDS FOR E-SCIENCE IN EUROPE EPCC FERMI NATIONAL LAB. FLORIDA STATE UNIV. GARUDA GRID GEMLCA TEAM GENIVIA CORP. GEON PROJECT GLOBUS CONSORTIUM GRIDWAY TEAM HANOI U. OF TECH. HARVARD UNIV. HEWLETT PACKARD HPSS TEAM IBM CORP. INDIANA UNIV. INRIA FRANCE INSTITUTE OF INFORMATICS, TECHNISCHE U. MUENCHEN INTEL CORP. JOINT INST. FOR NUCLEAR RESEARCH, MOSCOW KITWARE INC. LARGE HADRON COLLIDER LARGE SYNOPTIC SURVEY TELESCOPE LAWRENCE BERKELEY NAT’L LAB. LEAD PORTAL TEAM LEIBNIZ SUPERCOMPUTING CENTER LASER INTERFEROMETER GRAVITATIONAL WAVE OBSERVATORY LOS ALAMOS NATIONAL LAB. L.A. CHILDREN'S HOSPITALLOUISIANA STATE UNIV. MDS TEAM MOAB TEAM MPICH-G2 TEAM NATIONAL CENTER FOR ATMOSPHERIC RESEARCH INST. ADV. INDUSTRIAL SCIENCE AND TECHNOLOGY NATIONAL RESEARCH COUNCIL OF CANADA NATIONAL RESEARCH GRID INITIATIVE, JAPAN NEON PROJECT NORTHERN ILLINOIS UNIV. TERAGRID OPERATIONS TERAGRID PSC RP TERAGRID SCIENCE GATEWAYS TERAGRID SOFTWARE WG TIGRE PROJECT UKQCD PROJECT UNIV. COMPLUTENSE DE MADRID UNIV. OF APPLIED SCIENCES, FACHBEREICH INFORMATIK /  TELEKOMMUNIKATION UNIV. OF BRITISH COLUMBIA UNIV. OF CALGARY UNIV. OF CALIFORNIA AT LOS ANGELES UNIV. OF MALAYSIA UNIV. OF SOUTHAMPTON UNIV. OF TWENTE UNIV. OF ULSTER AT COLERAINE UNIV. OF VICTORIA UNIV. OF VIRGINIA UNIV. OF WESTMINSTER UNIV. OF WISCONSIN SECURITY  COMMITTEE UNIV. PASSAU UNIV. POLITECNICA DE  CATALUNYA UNIV. WIEN UNIVA CORP. USC IMAGE PROCESSING AND  INFORMATICS LAB VANDERBILT UNIV. MEDICAL  CENTER WESTFALISCHE WILHELMS-UNIV.  MUNSTER

12. The Globus Community: A Sample OAK RIDGE NATIONAL LAB. OGSA-DAI TEAM OMII-EUROPE OMII-UK OPEN GRID COMPUTING ENVIRONMENTS PROJECT OPEN GRID FORUM GIN WG OPEN GRID FORUM JSDL WG OPEN GRID FORUM OGSA WGOPEN GRID FORUM BES WG OPEN GRID FORUM HPCP WG OPEN SCIENCE GRIDINTEGRATION TESTBED OPEN SCIENCE GRID VALIDATION TESTBED OPEN SCIENCE GRID VDT OASIS PEGASUS PROJECT PHILIPPS-UNIV. MARBURG PRAGMA PYGLOBUS PROJECT RWTH AACHEN UNIV. SANDIA NATIONAL LAB. SANTA CLARA UNIV. SOUTHERN CALIFORNIA EARTHQUAKE CENTER SEMANTIC BITS SHANGHAI JIAO TONG UNIV. SRM PROJECT SWEDISH INSTITUTE OF  TECHNOLOGY TECHNISCHE U. DRESDEN TECHNISCHE U. DELFT TERAGRID DATA WG TERAGRID NCSA RP APACHE APOLLO PROJECT APACHE HERMES PROJECT AUSTRALIAN PARTNERSHIP FOR ADV. COMPUTING BARCELONA SUPERCOMP. CENTER BILD UMEAA UNIV. CABIG CANADIAN FOREST SERVCIE CANARIE PROJECT CEDPS PROJECT CHINAGRID CONDOR TEAM COREGRID LABORATORY D-GRID DHIRUBHAI AMBANI INST. OF INFO. AND COMM. TECH. EARTH SYSTEM GRID ENABLING GRIDS FOR E-SCIENCE IN EUROPE EPCC FERMI NATIONAL LAB. FLORIDA STATE UNIV. GARUDA GRID GEMLCA TEAM GENIVIA CORP. GEON PROJECT GLOBUS CONSORTIUM GRIDWAY TEAM HANOI U. OF TECH. HARVARD UNIV. HEWLETT PACKARD HPSS TEAM IBM CORP. INDIANA UNIV. INRIA FRANCE INSTITUTE OF INFORMATICS, TECHNISCHE U. MUENCHEN INTEL CORP. JOINT INST. FOR NUCLEAR RESEARCH, MOSCOW KITWARE INC. LARGE HADRON COLLIDER LARGE SYNOPTIC SURVEY TELESCOPE LAWRENCE BERKELEY NAT’L LAB. LEAD PORTAL TEAM LEIBNIZ SUPERCOMPUTING CENTER LASER INTERFEROMETER GRAVITATIONAL WAVE OBSERVATORY LOS ALAMOS NATIONAL LAB. L.A. CHILDREN'S HOSPITALLOUISIANA STATE UNIV. MDS TEAM MOAB TEAM MPICH-G2 TEAM NATIONAL CENTER FOR ATMOSPHERIC RESEARCH INST. ADV. INDUSTRIAL SCIENCE AND TECHNOLOGY NATIONAL RESEARCH COUNCIL OF CANADA NATIONAL RESEARCH GRID INITIATIVE, JAPAN NEON PROJECT NORTHERN ILLINOIS UNIV. TERAGRID OPERATIONS TERAGRID PSC RP TERAGRID SCIENCE GATEWAYS TERAGRID SOFTWARE WG TIGRE PROJECT UKQCD PROJECT UNIV. COMPLUTENSE DE MADRID UNIV. OF APPLIED SCIENCES, FACHBEREICH INFORMATIK /  TELEKOMMUNIKATION UNIV. OF BRITISH COLUMBIA UNIV. OF CALGARY UNIV. OF CALIFORNIA AT LOS ANGELES UNIV. OF MALAYSIA UNIV. OF SOUTHAMPTON UNIV. OF TWENTE UNIV. OF ULSTER AT COLERAINE UNIV. OF VICTORIA UNIV. OF VIRGINIA UNIV. OF WESTMINSTER UNIV. OF WISCONSIN SECURITY  COMMITTEE UNIV. PASSAU UNIV. POLITECNICA DE  CATALUNYA UNIV. WIEN UNIVA CORP. USC IMAGE PROCESSING AND  INFORMATICS LAB VANDERBILT UNIV. MEDICAL  CENTER WESTFALISCHE WILHELMS-UNIV.  MUNSTER • Dev.Globus Software Developers • Scientists • Application Developers • Educators • Science Collaborations • Corporations • Medical Centers • Cyberinfrastructure Operators • National Research Councils

13. Where is Globus Going? • More toward Solutions… • Middleware not so beautiful to users • Yet extremely important • Hide the middleware whenever possible (plumbing) • Increasing focus on production quality code • Increasing Community Involvement even more… • Making our plans and roadmap visible • We need your help & directions • Incubators • Add your Innovative contributions

14. How are we getting there? • Our community is helping us! • Also through our ongoing internal development, of course.

15. Globus Incubator Projects(Partial List) • CoG Workflow — Fine-grained workflow system • GEMLCA — Deploy Legacy Apps as Grid Svcs • GridShib — Integration with Shibboleth • GridWay — Meta-scheduler • gt-hs — Integration of Handle System • MEDICUS — Medical image management • Metrics — Infrastructure for usage reporting • OGCE — Portal toolkit • PURSe — Portal-based user registration service • ServMark — Grid service performance tester • Virtual Workspaces — Virtual machine mgmt

16. http://dev.globus.org/wiki/Incubator/Introduce Shannon Hastings hastings@bmi.osu.edu Multiscale Computing Laboratory Department of Biomedical Informatics The Ohio State University

17. Introduce -- Grid ServiceAuthoring Tooolkit • GUI for creating and manipulating a grid service • A framework that enables fast and easy creation of Globus-based grid services • Utilizes a best practice layered grid service architecture • Handles core service architecture requirements for strongly typed and highly interoperable grid services • Using Introduce, our “Create a Globus Service” tutorial examples reduced from 3 hours to about 15 minutes • Committers: Ohio State and UC/ANL

18. Service-Oriented Science People create services (data or functions) … which I discover (& decide whether to use) … & compose to create a new function ... & then publish as a new service.  I find “someone else” to host services, so I don’t have to become an expert in operatingservices & computers!  I hope that this “someone else” can manage security, reliability, scalability, … ! ! “Service-Oriented Science”, Science, 2005

19. Service-Oriented Science& Cancer Biology caBIG: sharing of infrastructure, applications, and data. Data Integration!

20. Appln Service Create Store Advertize Discover Transfer GAR Invoke; get results Deploy RAVI • Remote Application Virtualization Infrastructure • Builds on Introduce • Define service • Create skeleton • Discover types • Add operations • Configure security • Wrap arbitrary executables Introduce Repository Service Index service Container Ravi Madduri et al., Argonne/U.Chicago & Ohio State University

21. RAVI Collaboration • We are interested in collaborating… • We are currently gathering requirements for the Service Oriented Science program. • Also, if you have an application you want to expose as a Grid service, let us know. • Contact madduri@mcs.anl.gov

22. Better Error Propagation • Will be forming a Tiger Team • Plan is to establish best practices for error reporting. • Needs community leadership & know-how • Goal is to make it easier for users to understand what is going on • Fix problems on the spot • Send helpful error codes to the support teams • Would you like to be involved? • Contact fraser@mcs.anl.gov

23. Updated Security Mechanisms • Transition toward “Attribute Based Authorization” • E.g. the fact that you are a resident of a particular state gives you certain privileges. • So, being a member of a VO gives similar privileges • May be able to eliminate Gridmap files … • Will work with the community to determine an architecture • Plan to submit the architecture to the community for review • Timing Q2 2008

24. Technology Improvements • Core • Reliability & Performance Improvements • GridFTP/XIO • Package UDT protocol (fast over distances) • Make sure filespace is available • Transfer files immediately as they are created • New Data Placement Service • Designed to offload data management from workflow managers • GRAM • Improved Staging Performance & Reliability • Advance reservation capability (works with MOAB, Catalina) • Monitoring (MDS) • Scalability improvements • Metrics • Enable Communities to collect their own information

25. Conclusions • Production Quality Software is a Must • Trend is toward Solutions • Four Pushes for 2008 • Reliability & Performance • Service Oriented Science • Improved Error Reporting & diagnostics • Security – attribute based authorization • Community Driven Efforts • Incubator projects advancing the technology • Invitation to Participate • Add your Incubator • Join in the discussions • Help Steer

26. MDS Plans/Goals • Improve scalability with respect to data size • Documentation/support for REST interfaces to MDS queries • Improved error reporting (toolkit-wide goal) • Improved support for community-contributed components • info providers, xslt transforms, etc. • Documentation, best practices guides, web page links to external projects • Roll out trigger service improvements • More support for modifying existing triggers • Technology preview of archive service

27. RLS is a scalable, efficient distributed registry Records the locations of data copies Allows replica discovery New Features for 2007 Embedded SQLite database for easier RLS deployment SQLite offers good performance and scalability on queries Does not support multiple simultaneous writers Pure Java client implementation Overcomes problems with JNI-based client on 64-bit platforms Improves reliability of portals that use RLS Java client WS-RLS interface: WS-RF compatible Web service interface to RLS easier integration of RLS services into GT4 environments New Developments in the Replica Location Service (RLS)

28. Data Replication Service Client • Data-intensive applications need data management services that integrate lower level Grid functionality • Efficient data transfer (GridFTP, RFT) • Replica registration and discovery (RLS) • DRS Functionality: • Discovers replicas (possible source files) in the Grid • Uses Replica Location Service (RLS) • Select among source files • Invoke Globus Reliable File Transfer Service to copy data • Uses GridFTP Data Transfer • Register new replicas in RLS Data Replication Service Reliable File Transfer Service Replica Location Service Catalogs GridFTP Server GridFTP Server

29. GRAM2 and GRAM4Performance Comparisons

30. Concurrent Jobs • Average seconds per 1000 jobs • Condor-g to GRAM to Condor LRM • VDT 1.6.0 (GT 4.0.3 + Patches)

31. Concurrent Jobs • Average seconds per 1000 jobs • Condor-g to GRAM to Condor LRM • GT 4.0.5

32. Improving performance forstaging jobs in GT 4.0.5 • Local in-process method calls between WSRF services • In GT 4.0.5, GRAM was modified to optionally invoke RFT via local method calls instead of Web service calls. • Local method calling will be available for general use in Java WS Core in the 4.2.x series • Any WSRF service which calls another in-process service will be able to make similar modifications for local calls and likely benefit from improved performance

33. Sequential Jobs • Average seconds per job (Fork) • VDT 1.6.x (GT 4.0.3 + Patches)

34. Sequential Jobs • Average seconds per job (Fork) • VDT 1.8.x / GT 4.0.5

35. Future Plans

36. GRAM4 in 4.0 series • RFT caching of GridFTP server connections (4.0.6) • Finalizing RFT enhancements to manage a cache of GridFTP server connections amongst all RFT requests (not just amongst file transfers within a request). This has shown to improve performance and reliability for GRAM4 file staging jobs. • 30% improvement in some tests • RFT clients submitting many separate RFT transfer requests will benefit from this enhancement too.

37. GRAM4 in 4.2 Series • 4.1.3 coming Dec 07 (very soon) • Core spec upgrade • Service auditing to DB • Backed out some targeted 4.2 GRAM features in favor of a more stable reliable GRAM implementation from 4.0.x series • JSDL support • Persistence data stored in DB • resource manager adapter API • All service interactions queued and processed

38. GRAM4 in 4.2 Series • 4.1.4 (?) • New job termination interface • Asynchronous model needed to prevent core threads from blocking • Improve notification interface for GramJob API • Support a single client notification consumer for many job submissions • RFT • GridFTP connection caching

39. Improve GRAM4 Client Documentation • Working with condor-g and LEAD we realize the GRAM java API doc needs to be improved and include more examples • Better integration with Cog’s job submission API

40. New Globus Resource Manager Adapter Incubator Project • Focussed project toward providing a variety of GRAM LRM adapter implementations • Coordinate efforts / avoid duplicate work by integrators • Project Chair: JP Navarro, TeraGrid

41. Standards Compliance • JSDL • Hopefully can resume JSDL work in 2008 • OGSA-BES • No specific plans yet for support

42. Advance Reservation • Started new Globus incubator project • GARS - Globus Advance Reservation Service • SC07 demo’ing early alpha version of GARS • SDSC Booth Wednesday 2 pm • Enable GT4 users to create and manage advance reservations of compute nodes. • Leverage advance reservation support provided by the LRM via adapter interface (ala GRAM) • Leverage GT4 Authn and Authz security model / callouts • Service middleware for co-schedulers like GUR

43. GARS Approach ARFS create reservations LRM Adapter LRM ARS manage Client J1 J1 J3 J3 create jobs J1 J1 J3 MJFS J2 J2 manage MJS Compute Cluster • Client creates: • a reservation for 4 nodes • a 1st job (J1) with 4 processes • a 2nd job (J2) with 2 processes • a 3rd job (J3) with 3 processes GT WSRF Container

44. Open Source Grid & Cluster Conference • GlobusWorld, Grid Engine Workshop, Rocks-A-Palooza • Hear the experiences of others who are usingGlobus, Grid Engine, Rocks, Ganglia, Cluster Express, and many other innovative software tools and techniques in their work • May 2008 in the USA • Share your ideas about the benefits, incentives, and risks of using grid techniques • Learn how you can use campus grids, community grids, and national grid systems to enhance your work and strengthen your collaborations • Meet the developers responsible for this open source grid and cluster software. • Email info@OpenSourceGridCluster.org to be notified of details www.OpenSourceGridCluster.org

45. GRAM2 and GRAM4Performance Comparisons

46. Concurrent Jobs • Average seconds per 1000 jobs • Condor-g to GRAM to Condor LRM • VDT 1.6.0 (GT 4.0.3 + Patches)

47. Concurrent Jobs • Average seconds per 1000 jobs • Condor-g to GRAM to Condor LRM • GT 4.0.5

48. Improving performance forstaging jobs in GT 4.0.5 • Local in-process method calls between WSRF services • In GT 4.0.5, GRAM was modified to optionally invoke RFT via local method calls instead of Web service calls. • Local method calling will be available for general use in Java WS Core in the 4.2.x series • Any WSRF service which calls another in-process service will be able to make similar modifications for local calls and likely benefit from improved performance

49. Sequential Jobs • Average seconds per job (Fork) • VDT 1.6.x (GT 4.0.3 + Patches)

50. Sequential Jobs • Average seconds per job (Fork) • VDT 1.8.x / GT 4.0.5