EGEE-III and EGI Design Study Future European Grid Landscape CSC Finnish IT Center for Science

1. EGEE-III and EGI Design Study Future European Grid Landscape CSC Finnish IT Center for Science

2. Contents • CSC as a part of the international e-Infrastructures • Overview of EGEE II/III • EGI design study

3. CSC and the European e-Infrastructures • Operational grids: • EGEE • EGEE, EGEE-II, EGEE-III • DEISA • DEISA, eDEISA, DEISA2 • NDGF • Finnish M-grid • Preparotory phase • PRACE • EGI_DS • e-IRGSP2

4. The EGEE Infrastructure Test-beds & Services Operations Coordination Centre Production Service Pre-production service Regional Operations Centres Certification test-beds (SA3) Global Grid User Support EGEE Network Operations Centre (SA2) Operational Security Coordination Team Security & Policy Groups Joint Security Policy Group EuGridPMA (& IGTF) Grid Security Vulnerability Group Operations Advisory Group (+NA4) Support Structures & Processes Training activities (NA3) Training infrastructure (NA4) EGEE'07; 2nd October 2007 EGEE'07; 2nd October 2007 4

5. Resources EGEE'07; 2nd October 2007 EGEE'07; 2nd October 2007 5

6. Increasing workloads Still expect factor 5 increase for LHC experiments over next year 32% EGEE'07; 2nd October 2007 EGEE'07; 2nd October 2007 6

7. Use of the infrastructure EGEE: ~250 sites, >45000 CPU 24% of the resources are contributed by groups external to the project ~>20k simultaneous jobs EGEE'07; 2nd October 2007 EGEE'07; 2nd October 2007 7

8. Infrastructures geographical or thematic coverage Support Actions key complementary functions Applications improved services for academia, industry and the public Registered Collaborating Projects 25 projects have registered as of Sept 2007:web page

9. EGEE-II to EGEE-III • EGEE-III • To be funded under European Commission call INFRA-2007-1.2.3 • 32M€ compared to ~37M € for EGEE-II • Key objectives • Expand/optimise existing EGEE infrastructure, include more resources and user communities • Prepare migration from a project-based model to a sustainable federated infrastructure based on National Grid Initiatives • 2 year period – May 2008 to April 2010 • No gap between EGEE-II and EGEE-III • Consortium • Now structured on a national basis (National Grid Initiatives/Joint Research Units) • 43 partners (compared to 90+ in EGEE-II)

10. Northern Europe Effort in EGEE-III

11. What’s special about “the Grid” • Grids have proven to be an excellent way of federating resources across computer centres of varying sizes into much larger quasi-homogeneous infrastructures. • This matches well with the needs of manydisciplines, allowing resources at participating institutes to meet the needs of the entire collaboration(s). • This in turn adds value to the individual sites, leading to a positive feedback situation. 11

12. But there’s (much) more… • Grid Computing (potentially) offers value to a wide range of applications, broadly classified as follows: • Provisioned • Large scale, long term “Grand Challenge” • e.g. LHC (“space microscopes”), space telescopes, …. 12

13. But there’s (much) more… • Grid Computing (potentially) offers value to a wide range of applications, broadly classified as follows: • Provisioned • Scheduled • Require large resources for short periods • Far too expensive to provision for a single ‘application’ • Not (always) time critical – disaster response? 13

14. But there’s (much) more… • Grid Computing (potentially) offers value to a wide range of applications, broadly classified as follows: • Provisioned • Scheduled • Opportunistic • Which includes the above but also other areas which are less “real time” 14

15. But there’s (much) more… • Grid Computing (potentially) offers value to a wide range of applications, broadly classified as follows: • Provisioned • Scheduled • Opportunistic • These can co-exist, sharing the same Grid infrastructure(s), to each others’ mutual benefit! 15

16. Routine Usage Testbeds Utility Service Evolution National Global

17. Why Sustainability TODAY? • Dependency: some application domains depend on grids already today • Protection of Investment: Investment in grids, both from funding organizations and from users, need to be protected • Perspective: Today’s grid users are grid enthusiasts, tomorrows grid users ask for a longer term perspective

18. Routine Usage Testbeds Utility Service Evolution National European e-Infrastructure Global

19. Terminology “European Grid Initiative (EGI)” = the future EGI Organisation + the NGIs “EGI Organisation” = the future team with central responsibility, including the management structure “EGI Design Study” = the present EU FP7 project producing a blueprint for a future EGI

20. EGI Objectives • Ensure the long-term sustainability of the European e-infrastructure • Coordinate the integration and interaction between National Grid Infrastructures • Operate the European level of the production Grid infrastructure for a wide range of scientific disciplines to link National Grid Infrastructures

21. 38 European NGIs + Asia, US, Latin America + PRACE + OGF-Europe + …

22. EGI Design Study Project proposal: • submitted to FP7-INFRASTRUCTURES-2007-1, 1.2.1 Design Studies Goal: • Conceptual setup and operation of a new organizational model of a sustainable pan-European grid infrastructure • Consortium: 9 Partners  EGI Preparation Team

23. EGI Preparation Team Members: • Johannes Kepler Universität Linz (GUP) • Greek Research and Technology Network S.A. (GRNET) • Istituto Nazionale di Fisica Nucleare (INFN) • CSC - Scientific Computing Ltd. (CSC) • CESNET, z.s.p.o. (CESNET) • European Organization for Nuclear Research (CERN) • Verein zur Förderung eines Deutschen Forschungsnetzes - DFN-Verein (DFN) • Science & Technology Facilities Council (STFC) • Centre National de la Recherche Scientifique(CNRS)

24. EGI Policy Board

25. EGI Policy Board

26. EGI Design Study Project proposal: • submitted to FP7-INFRASTRUCTURES-2007-1, 1.2.1 Design Studies Goal: • Conceptual setup and operation of a new organizational model of a sustainable pan-European grid infrastructure • Consortium: 9 Partners  EGI Preparation Team • NGI Representatives  EGI Policy Board • Person months: ~300 • Duration: 1 Sept 2007 – 30 Nov 2009 (27 Months)

27. EGI_DS Schedule Duration27 months: Develop EGI Proposal NGIs signing Proposal Start of EGEE-III Final Draft of EGI Blueprint Proposal Submission of EGEE-III EGI Blueprint Proposal Start of EGI Design Study EGEE-III transition to EGI-like structure EU Call Deadline for EGI Proposal EGI Entity in place EGEE-III (2YEARS)‏ EGI operational EGEE-II (2YEARS)‏ 2008 2010 2009

28. Work Distribution • WP2: EGI Requirements Consolidation(Fotis Karayannis, GRNET) • WP3: EGI functionality definition(Laura Perini, INFN) • WP4: Study of EGI legal and organisational options(Beatrice Merlin, CNRS) • WP5: Establishment of EGI(Jürgen Knobloch, CERN) • WP6: EGI Promotion and Links with Other Initiatives(Per Öster, CSC)

29. EGI Webpage www.eu-egi.org

30. Characteristics of NGIs Each NGI • … should be a recognized national body with a single point-of-contact • … should mobilise national funding and resources • … should operate the national e-Infrastructure • … should supports user communities (application independent, and open to new user communities and resource providers) • … should contribute and adhere to international standards and policies Responsibilities between NGIs and EGI are split to be federated and complementary

31. NGI Responsibilities • Regional operations functions • Resource provisioning • Accounting and monitoring • Application support • Help desk facilities • Contribution to middleware development

32. EGI Operations Principles • Reliability of Grid services and SLAs; • Multi-level operation model; • EGI, NGI and ROC; • Multiple middleware stacks; • Planning, coordination and gathering of new requirements; • Cooperation; • Federation, interoperability and data aggregation. www.eu-egi.org 32

33. EGI Transition Scenario • Many applications rely now on production quality grid infrastructures. • Sciences using today’s grid operations (supported by EGEE, DEISA, …) and other projects should be able to transit without disruption to the envisaged sustainable EGI/NGI-based model. • Example: EGEE-III using JRU’s as NGI equivalents

34. EGI Transition Scenario Important: Human Expertise • Many developments and operational tasks are performed by highly skilled staff, which has built up their expertise through the lifetime of the current grid projects. • Care must be taken that this expertise can be retained during and after the transition period.

35. Convention (Statutes) of the EGI Organisation • Goal: Defining a set of agreed formal criteria and rules for the EGI organisation to be ratified by the EGI Advisory Board • Contents: • Purpose of the EGI Organisation • Membership • Organisational structure (Council, Directors, …) • Budget contributions, audits, …

36. EGI – European Grid Initiative • Future EGI Organisation = “Glue” between various grid communities in Europe and beyond • EGI_DS defines required mechanisms and functionalities of the EGI Organisation  Towards a sustainable environment for the application communities utilizing grid infrastructures for their everyday work

37. Acknowledgements • Slides from different projects and contributors • EGEE II-III • Bob Jones • Ian Bird • EGI-DS • Dieter Kranzlmüller