Science Gateways on the TeraGrid

1. Science Gateways on the TeraGrid Charlie Catlett, Sebastien Goasguen, Jim Marsteller, Stuart Martin, Don Middleton, Kevin J. Price, Anurag Shankar, Von Welch, Nancy Wilkins-Diehr

2. Welcome to MadisonSome motivation from Vince Lombardi • “It is time for us all to stand and cheer for the doer, the achiever - the one who recognizes the challenges and does something about it.” • And a lesser known quote • “A school without football is in danger of deteriorating into a medieval study hall.”

3. Today’s Topics • Work first presented as an invited paper at Grid Computing Environments (GCE06) at SC06 • GCE06 featured many portal developers • Our paper was invited to provide a look at what it meant to integrate a local portal with large, shared resources • Gateway overview – 5 min. • Gateway integration issues – 15 min. • Accounting • GRAM • Stu Martin, 11am yesterday • Security • Commsh • Attribute-based authentication • Von Welch, 3:30pm today • Metrics and successful peer-review • Future work – 10 min. • Gateway primer • Best practices

4. Gateways are part of TeraGrid’s 3-pronged strategy to further science • DEEP Science: Enabling Terascale Science • Make science more productive through an integrated set of very-high capability resources • Advanced Support for TeraGrid Applications (ASTA) projects • WIDE Impact: Empowering Communities • Bring TeraGrid capabilities to the broad science community • Science Gateways • OPEN Infrastructure, OPEN Partnership • Provide a coordinated, general purpose, reliable set of services and resources • Grid interoperability working group

5. Science Gateways Can Take a Variety of Forms Workflow Composer • Increasing investment by communities in their own cyberinfrastructure, but heterogeneous: • Resources • Users – from expert to K-12 • Software stacks, policies • Science Gateways • Provide “TeraGrid Inside” capabilities • Leverage community investment • Three common forms: • Web-based Portals • Application programs running on users' machines but accessing services in TeraGrid • Coordinated access points enabling users to move seamlessly between TeraGrid and other grids.

6. TeraGrid Science Gateways Overview • October, 2004 “Science Gateway” term originates • “Our TeraGrid WIDE strategy aims at this broader community and is embodied in the concept of a “science gateway,” providing community-tailored access to TeraGrid services and capabilities.” • Gateway teams identified to help TeraGrid define what it will need to support these entirely new usage models • Address needs of large ITR projects • Spring, 2005 Science Gateway Requirements Analysis Team (RAT) • Origin of the RAT (and RAT slides) • Identification of common needs across the gateways • Goal is production use of TG resources in the gateway as well as development of process and policy within TG for scalable gateway program and services • Tremendous sharing of experiences amongst talented developers

7. 10 initial projects as part of TG proposal >20 Gateway projects today No limit on how many gateways can use TG resources Prepare services and documentation so developers can work independently Open Science Grid (OSG) Special PRiority and Urgent Computing Environment (SPRUCE) National Virtual Observatory (NVO) Linked Environments for Atmospheric Discovery (LEAD) Computational Chemistry Grid (GridChem) Computational Science and Engineering Online (CSE-Online) GEON(GEOsciences Network) Network for Earthquake Engineering Simulation (NEES) SCEC Earthworks Project Network for Computational Nanotechnology and nanoHUB GIScience Gateway (GISolve) Biology and Biomedicine Science Gateway Open Life Sciences Gateway The Telescience Project Grid Analysis Environment (GAE) Neutron Science Instrument Gateway TeraGrid Visualization Gateway, ANL BIRN Gridblast Bioinformatics Gateway Earth Systems Grid Astrophysical Data Repository (Cornell) Many others interested Gateway Growth Continues at a Rapid Pace

8. Gateway overview– 5 min. • Gateway integration issues – 15 min. • Accounting • GRAM • Security • Commsh • Attribute-based authentication • Metrics • Future work – 10 min. • Gateway primer • Best practices

9. Tremendous Opportunities Using the Largest Shared Resources - Challenges too! • What’s different when the resource doesn’t belong just to me? • Resource discovery • Accounting • Security • Proposal-based requests for resources (peer-reviewed access) • Code scaling and performance numbers • Justification of resources • Gateway citations • Tremendous benefits at the high end, but even more work for the developers • Potential impact on science is huge • Small number of developers can impact thousands of scientists • But need a way to train and fund those developers and provide them with appropriate tools

10. What Did We Learn About Common Gateway Requirements? • Web Services • GT4 deployment, identification of remaining capabilities • Information services, WebMDS • Auditing • Need to retrieve job usage info on production resources • GRAM audit deployed in test mode in September, inclusion in CTSSv4 • Community Accounts • Policy finalized, security approaches being tested by RPs • Attribute-based authentication testing • Allocations • Changes in allocation procedures, the mechanisms used to evaluate science impact, and models for identity management, authentication and authorization that are more tuned to virtual organizations. • Scheduling • Metascheduling RAT • On-demand via SPRUCE framework • Outreach • Talks, Schools/workshops (NVO, GISolve), major project demonstrations (LEAD) • SURA, HASTAC, GEON, CI-Channel, SC, Grace Hopper, MSI-CI2, Lariat, Science Workflows and On Demand Computing for Geosciences Workshop • Primer • Living document in wiki, provides up-to-date overview and instructions for new gateway developers (“how to make your portal a TeraGrid science gateway”)

11. “Per Job” Accounting is Key Functionality for GatewaysStuart Martin, ANL • Common gateway structure • Web front end, users log on to gateway • Jobs run as single user on TeraGrid • Need to tie usage to individual users • Globus used by many gateway developers to access TeraGrid resources • GRAM operated in a fire and forget mode • When a job finishes there is no straightforward way to determine how many CPU hours the job consumed • That information is critical to attributing usage to individual users using a Science Gateway account on the TeraGrid

12. GRAM Audit Extension Provides Need Accountability • GRAM2 and GRAM4 services were enhanced to create audit records that are written to a database local to the GRAM services • Enhancements provide a persistent link between the grid service’s job id and the local resource manager’s (LRM) job id • Open Grid Services Architecture-Data Access and Integration (OGSA-DAI) provide a service interface for TeraGrid’s audit and accounting information

13. Individual Usage Tracking Now Possible • Gateways can remotely submit jobs to TeraGrid and • Account for usage on a per job basis without needing to understand the details of the various local resource managers chosen by TeraGrid resource providers. • Capability will be very useful for other projects using Globus where per-job usage information is needed. • Enhancements reduce the complexity for gateways to interface with TeraGrid’s computational resources • Allows TeraGrid to simultaneously support an increasing number of gateways.

14. Linked Environments for Atmospheric Discovery • Providing tools that are needed to make accurate • predictions of tornados and hurricanes • Meteorological data • Forecast models • Analysis and visualization tools • Data exploration and Grid workflow

15. Highlights: LEAD Inspires Students • A student gets excited about what he was able to do with LEAD • “Dr. Sikora:Attached is a display of 2-m T and wind depicting the WRF's interpretation of the coastal front on 14 February 2007. It's interesting that I found an example using IDV that parallels our discussion of mesoscale boundaries in class. It illustrates very nicely the transition to a coastal low and the strong baroclinic zone with a location very similar to Markowski's depiction. I created this image in IDV after running a 5-km WRF run (initialized with NAM output) via the LEAD Portal. This simple 1-level plot is just a precursor of the many capabilities IDV will eventually offer to visualize high-res WRF output. Enjoy! • Eric” (email, March 2007)

16. Securing Community AccountsKevin Price, NCSA, Aaron Shelmire, PSC • Additional risks arise when providing community account and web interfaces to high performance resources. • TeraGrid security working group analyzing risks developing mitigation approaches. • Sites may take independent approaches to risk mitigation • One approach being developed at NCSA is the Community Shell, or Commsh • Commsh allows for two methods of account restriction: • a configuration file is created that defines which commands (or sets of commands) a given account can execute. • commands can be specified using wildcards and regular expressions for flexibility • change-root (or chroot) jailing. Change-root jailing effectively creates a filesystem-based "sandbox" for the account, only allowing commands to be executed from within this sandbox

17. Federated Identity ManagementVon Welch, NCSA • Traditionally each resource or resource-providing site was responsible for the management of their users identities • The science gateway model brought an out-sourcing of identity management from the resource to the gateway • For maximum scalability, the goal is to shift identity management all the way back to the user’s home institution and leverage the existing identity management infrastructure • Mechanisms to achieve this based on Shibboleth, GridShib, myVocs are other technologies are currently being evaluated by TeraGrid

18. Individual User Environment (G)Id uid (G)Id uid (G)Id uid project O(10) O(1) O(1) Resource TGCDB O(1000) O(1000) Grant Process O(10) Use cases: Traditional users, Development

19. Authenticated User Environment O(10) O(1) O(1) (G)Id O(1) (G)Id O(10) ? uid project (G)Id O(10) Resource TGCDB Grant Process Use cases: Grid-savvy user communities, Production runs, user managed services

20. Gateway Environment O(10) O(1000) O(100-106) ? O(1) ? O(100) O(10) O(1000) O(1) GId uid O(100) ? O(1) ? O(1) O(1) uid project O(10) O(100) Gateway ComId Resource TGCDB Grant Process Use cases: Large communities of users, novice users, public

21. Community Gateway Accounts • Shift authentication and authorization from RP to the Science Gateway • Whole community then appears as “one” user to the RP in terms of authorization • One grid-mapfile and /etc/password entry • or perhaps (a mapped set of) virtual machine images • Except accounting and troubleshooting. We still need an individual identifier

22. The Proposal • Plan for a world where users can be authenticated via their home campus identity management system • Enable attribute-based authorization of users by RP site • Allow for user authentication with authorization by community • Prototype system in testbed, with involvement of interested parties to work out issues • All usage still billed to an allocation • Community or individual

23. MetricsDon Middleton, NCAR • Metrics of success are commonly requested for government funded programs • Successful gateway design will allow principal investigators to highlight gateway usage as well as science accomplishments due to the gateway • Some gateways may set up a mechanism for researchers to cite the use of the gateway in publications • Success both in funding the gateway and in requesting TeraGrid resources can be traced to scientific accomplishments and a history of publications

24. Sample Metrics Collected by ESG • The DOE-sponsored Earth System Grid (ESG) project includes a Metrics Service that tracks • logins • file and aggregation downloads • browse and search requests • total volume of activity conducted via its portal. • This information is very useful to principal investigators and sponsors in terms of determining the overall impact of the project • As ESG begins to utilize TeraGrid resources, it will need to track computational and data services that are delivered to it as a Science Gateway

25. NCAR Earth System Grid • Science Gateway for climate research • Enabling analysis and understanding gained from global Earth System computational models • ESG originally a distributed data management/access system but it has evolved into more. • User registration, authorization controls, and metrics tracking • Community Climate System Model (CCSM) model source, initialization datasets, post-processing codes, and analysis and visualization tools. • Prototypes of model- submission environments • Eventually real-time tracking of model status along with references to available output datasets. • Expect to see more model runs at higher- resolution and with greater component scope.

27. Gateway Overview – 5 min. • Gateway integration issues – 15 min. • Accounting • GRAM • Security • Commsh • Attribute-based authentication • Metrics • Future work – 10 min. • Gateway primer • Best practices

28. Science Gateway Primer • Framework for more developed documentation coming this summer/fall • Primer components • TeraGrid resources and services available to Science Gateways • Requirements for using TeraGrid resources • Best practices when designing a gateway • Software contribution area • Wiki-based • Very dynamic development community • Counting on them for contributions • http://www.teragridforum.org/mediawiki/index.php?title=TeraGrid_Science_Gateways_Primer

29. Resources and Services for Gateways • Compute, data and visualization resources • Software • Common TeraGrid Software Stack (CTSS) • Third party applications on a variety of platforms • Community Software Area for user-maintained software • Software packaging and distribution mechanisms • Accounting services • Developer accounts • Community accounts • In the future, dynamic accounts • Web services • External relations staff available to help publicize successes

30. Requirements and Recommendations for Gateways • Additional information to be provided when requesting community accounts • IP address of portal • Data and compute expectations • Directories from which commands will be executed • Logs including requesting IP address, date stamp (Universal Time Code), and username for all users of the gateway • Mechanisms to restrict problem jobs

31. Planning Assess If Gateway-ing Adds Any Value Create a Precise List of Requirements the GW Must Meet Plan for the Long Term (= GW Lifetime) Design Use Formal Design Principles Involve Users in the Design Use Mockups to Perform Usability Testing Design a Focused and Uncluttered UI Implementation Choose Technologies Based on Resources/Time Hire/Use Developers with UI Experience Develop in Stages Use Reusable Components Operation Monitor Gateway Components 24x7 Institute Help Desk ProcessesMonitor & Implement New Technologies Keep Content Current/Relevant Desirable Gateway Characteristics Universal, Secure Access Ability to Personalize Based on Open Standards (JSR 168/286, OGSA, etc.) Use of Modular, Reusable Design (Use Portlets) Use of Technologies With a Rich API/Abstraction Layer Platform Independence (Web, Java, XML, etc.) Rapid Development Capability Ease of Integration into Existing Infrastructure Availability of Workflows Use of Commodity Software Airtight Security Extensibility Maintainability Scalability Extensive Help and documentation Lots of Best Practices!Thanks Anurag

32. Future activities will include • Targeted support for new gateways • ASTA-like model • One or more TeraGrid staff members at a minimum of 25% FTE for a 4-12 month effort • Generalized help desk support • Gateway developers are a growing community with unique needs • Getting started support and documentation • Portal frameworks • Data management approaches • Workflow tools • Collaboration tools • Training • TG07 tutorial modules • Development of tools production gateways can use • Web Service interfaces • Application Hosting Environments • Tracking number of users, use of TG resources • Accounting/authorization tools • Citation capabilities • Proposal tips • “Where can I run now” capability • RP capabilities • CTSS SGW kit

33. Would development of a gateway help your research? • Think about your current bottlenecks • What would you like to explore if only you had • Lots of disk • Lots of compute resources • Powerful analysis capabilities • A nice interface to information • Contact us it you believe TeraGrid resources could help your work • wilkinsn@sdsc.edu