1 / 43

GridPP1 and AHM , Collaboration Meeting, Liverpool, 14 Sept 2004

GridPP1 and AHM , Collaboration Meeting, Liverpool, 14 Sept 2004. Contents. Here lies GridPP1.. What we desired, planned, or attempted A potted history of GridPP1 from meeting 1 to 10.. All Hands Meeting 2004 interlude.. Towards an e - Infrastructure for Research and Innovation

rauls
Download Presentation

GridPP1 and AHM , Collaboration Meeting, Liverpool, 14 Sept 2004

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. GridPP1 and AHM, Collaboration Meeting, Liverpool, 14 Sept 2004

  2. Contents • Here lies GridPP1.. • What we desired, planned, or attempted • A potted history of GridPP1 from meeting 1 to 10.. • All Hands Meeting 2004 interlude.. • Towards an e-Infrastructure for Research and Innovation • Building a National e-Infrastructure • Packed agenda (not a summary) • LHC Grid Deployment status: a turning point • Are we a Grid? • What was GridPP1? • GridPP2: first steps from base camp • History teaches us… • Summary GridPP1 and AHM

  3. Here lies GridPPR.I.P. Born: 1/9/01 Re-Born: 1/9/04

  4. What wedesired, planned, or attempted Premise The next IT revolution will be the Grid. The Grid is a practical solution to the data-intensive problems that must be overcome if the computing needs of many scientific communities and industry are to be fulfilled over the next decade. Aim The GridPP Collaboration aims to develop and deploy a large-scale science Grid in the UK for use by the worldwide particle physics community. Objectives SCALE: GridPP will deploy open source Grid software (middleware) and hardware infrastructure to enable the testing of a prototype of the Grid for the LHC of significant scale. INTEGRATION: The GridPP project is designed to integrate with the existing Particle Physics programme within the UK, thus enabling early deployment and full testing of Grid technology and efficient use of limited resources. DISSEMINATION: The project will disseminate the GridPP deliverables in the multi-disciplinary e-science environment and will seek to build collaborations with emerging non-PPARC Grid activities both nationally and internationally. UK PHYSICS ANALYSES (LHC): The main aim is to provide a computing environment for the UK Particle Physics Community capable of meeting the challenges posed by the unprecedented data requirements of the LHC experiments. UK PHYSICS ANALYSES (OTHER): The process of creating and testing the computing environment for the LHC will naturally provide for the needs of the current generation of highly data intensive Particle Physics experiments: these will provide a live test environment for GridPP research and development. DATAGRID: Open source Grid technology is the framework used to develop this capability. Key components will be developed as part of the EU DataGrid project and elsewhere. LHC COMPUTING GRID: The collaboration builds on the strong computing traditions of the UK at CERN. The CERN working groups will make a major contribution to the LCG research and development programme. INTEROPERABILITY: The proposal is also integrated with developments from elsewhere in order to ensure the development of a common set of principles, protocols and standards that can support a wide range of applications. INFRASTRUCTURE: Provision is made for facilities at CERN (Tier-0), RAL (Tier-1) and use of up to four Regional Centres (Tier-2). OTHER FUNDING: These centres will provide a focus for dissemination to the academic and commercial sector and are expected to attract funds from elsewhere such that the full programme can be realised. GridPP1 and AHM

  5. 1 Teamwork title.open ( ); revolution {execute}; LHC Computing Challenge Methodology? Hierarchical Information in a Global Grid Supernet Aspiration? HIGGS DataGRID-UK Aspiration? ALL Data Intensive Computation Tony Doyle - University of Glasgow

  6. 1 GridPP Year Book Minimisation problem... ...with (your) constraints Tony Doyle - University of Glasgow

  7. 2 I: £2.49m  £1.2m Experiment J £7.1m  £6.7m  £6.0m Objectives 2.6% CERN H: 3.2% H*: 5.4% Software Support 90.0% G: Prototype Grid 9.7% £3.2  £2.9  £2.45m UK Managers 1.9% 1.5% 1.9% Work Groups A - F 1.7% F* UK Capital 1.5% F 2.7% E 0.6% 1.1% 1.9% D* 1.5% 0.4% 1.4% D C* C B* B A* A £17m 3-Year Project Tony Doyle - University of Glasgow

  8. 3 Summary • Most effort so far directed towards DataGrid=middleware • Significant UK contributions • Startup difficulties – largely overcome • Recruitment, late filling of posts – funded & unfunded • Lost ground being recovered.. • All WP with UK involvement active & contributing • Testbed-1 : so far modest, predictable teething problems (many !) • Next major challenges focus on INTEGRATION • US experiments programme - planning for today/tomorrow.. • LCG programme - planning for 2007... • Testbed (focus for tomorrow’s discussions) - requires input from system managers as well as software developers. • Positioned reasonably well ~6 months into GridPP.. • with many challenges ahead... Tony Doyle - University of Glasgow

  9. 4 OGSA Grid - What’s been happening? GRID A unified approach • A lot… • GGF4, OGSA and support of IBM (and others) • [as opposed to .NET development framework and passports to access services] • Timescale? September 2002 • W3C architecture for web services • Chose (gzipped) XML as opposed to other solutions for metadata descriptions… and web-based interfaces • linux • [as opposed to other platforms… lindows??] • C++ (experiments) and C, Java (middleware) APIs • [mono - Open Source implementation of the .NET Development Framework??] Tony Doyle - University of Glasgow

  10. Coordinates resources that are not subject to centralized control … using standard, open, general-purpose protocols and interfaces … to deliver nontrivial qualities of service YES. This is why development and maintenance of a UK-EU-US testbed is important. YES... Globus/Condor-G/EDG ~meet this requirement. Common experiment application layers are also important here e.g. SAM, GANGA. NO(T YET)… Experiments should define whether this is true via this year’s data analyses and challenges. 5 Are we a Grid? http://www-fp.mcs.anl.gov/~foster/Articles/WhatIsTheGrid.pdf Tony Doyle - University of Glasgow

  11. 6 Timeline DataGrid GridPP-Procure, Install, Compute, Data Develop, Test, Refine LHC Computing Grid Prototypes Production Initial Grid Tests Worldwide Grid Demonstrations Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 2001 2002 2003 2004 2005 GridPP II Grid Service EGEE (DataGrid II?) Middleware and Hardware upgrades Transition and Planning Phase… Tony Doyle - University of Glasgow

  12. 7 GridPP2 Proposal http://www.gridpp.ac.uk/docs/gridpp2/ ~30 page proposal + figures/tables + 11 planning documents: • Tier-0 • Tier-1 • Tier-2 • The Network Sector • Middleware • Applications • Hardware Requirements • Management • Travel • Dissemination • From Testbed to Production A lot of work by everyone in a relatively short time… Tony Doyle - University of Glasgow

  13. 8 Visible progress this year in GridPP1 Management via the Project Map and Project Plan High level tasks and metrics: under control Major component is LCG We contribute significantly to LCG and our success depends critically on LCG Middleware components on critical path w.r.t. LCG adoption Deployment – high and low level perspectives merge via monitoring/accounting Resource centre and experiment accounting are both important Today’s operations in the UK are built around a small team Future operations planning expands this team: Production Manager being appointed Middleware deployment focus on Information Service performance Security (deployment and policy) is emphasised “Production Grid” will be difficult to realise: need to start GridPP2 planning now (already underway) GridPP2 proposal: formal feedback in November Transition period for: Middleware/Security/Networking Groups Experiments Phase II Production Grid Planning Conclusions Tony Doyle - University of Glasgow

  14. 9 4. Background… GridPP2 steps towards project spec. 0. PPARC Call (February 2003) 1.GridPP2 Proposal (30/5/03) 2. Feedback from Reviewers 3. Responses from the PMB See http://www.gridpp.ac.uk/docs/gridpp2/ 4. Projects Peer Review Panel (14-15/7/03) 5. Grid Steering Committee (28-29/7/03) See http://www.gridpp.ac.uk/talks/GridPP_GSC_030728.ppt 6. Tender Document (11/11/03) Seehttp://www.gridpp.ac.uk/cb/doc/GridPP2_Tender.doc 7. Science Committee (17-18/11/03) 8. PPARC (25/11/03) 9. CB (4/12/03) 10.PRSC (16/1/04). PRSC summary 11.PMB (26/1/04).Work area effort tables 12. GridPP9 Collaboration Meeting (4-5/2/04) 13.OC (18/2/04)… GridPP2 presentation. (unlucky for some..) Tony Doyle - University of Glasgow

  15. 10 GridPP Summary: From Prototype to Production BaBarGrid BaBar EGEE SAMGrid CDF D0 ATLAS EDG LHCb ARDA GANGA LCG ALICE CMS LCG CERN Tier-0 Centre CERN Prototype Tier-0 Centre CERN Computer Centre UK Tier-1/A Centre UK Prototype Tier-1/A Centre RAL Computer Centre 4 UK Tier-2 Centres 19 UK Institutes 4 UK Prototype Tier-2 Centres Separate Experiments, Resources, Multiple Accounts Prototype Grids 'One' Production Grid 2004 2007 2001 Tony Doyle - University of Glasgow

  16. Towards an e-Infrastructure for Research and Innovation:A Progress Report on e-Science Tony Hey Director, UK e-Science Core Programme http://www.nesc.ac.uk/events/ahm2004/

  17. Web Services and Grids • Computing models developed for sequential machines led to the distributed object model of distributed computing represented by Java and CORBA • Experience has shown that the distributed object model ties distributed entities together too tightly • Resulted in ‘fragile’ distributed software systems when going from LANs to WANs • Replace distributed objects by ‘services’ connected by “one-way” messages and not by request-response messages • IT industry has ‘united’ around Web Services

  18. 4. Building a National e-Infrastructure Three major new activities for Phase 2 of the Core Programme: • Deployment of National Grid Service (NGS) and establishment of a Grid Operation Support Centre • Establish Open Middleware Infrastructure Institute (OMII) for testing, software engineering and UK repository • Set up Digital Curation Centre (DCC) to lead on long-term data preservation issues

  19. 6. Science & Innovation Investment Framework 2004 - 2014 Major Components of the UK Vision • Multidisciplinary Working • Creation of a multidisciplinary research environment • Links between Funding Councils and RCUK • Uses e-Science exemplars from Earth Systems Science and Systems Biology

  20. Science & Innovation Investment Framework 2004 - 2014 • Information Infrastructure • Access to experimental data sets and publications • Collection and preservation of digital information • Importance of National e-Infrastructure • Tie into international efforts • OST to take the lead • Capital Infrastructure – Large Facilities • Diamond Synchroton to open 2007 • Second Target Station for ISIS Neutron Source from 2008 • Large Hadron Collider operative from 2007 • Plus • Hector HPC Facility • ITER Fusion Machine

  21. e-Infrastructure for Research and Innovation • Ten-year investment framework is collaboration between the Treasury, the DfES and the DTI • The RCUK e-Science Programme with the Core Programme have made a good start at building the UK e-Infrastructure • Need continuing collaboration between RCUK, JISC and the DTI • Essential to continue ring-fenced funding for e-Infrastructure and e-Research in SR2004 settlement

  22. scale GridPP prototype Grid > 1,000 CPUs • 500 CPUs at the Tier-1 at RAL > 500 CPUs at 11 sites across UK organised in 4 Regional Tier-2s > 500 TB of storage > 800 simultaneous jobs • Integrated with international LHC Computing Grid (LCG) > 5,000 CPUs > 4,000 TB of storage > 70 sites around the world > 4,000 simultaneous jobs • monitored via Grid Operations Centre (RAL) Tony Hey

  23. A Packed Agenda • Proceedings of the UK e-Science All Hands Meeting 2004 • Wednesday 1 September 20040900-1230hrsPlenary Session0900 - 0915hrsWelcome from Ron Perrott0915 - 1000hrsKeynote talk from Tony Hey1000 - 1100hrsKeynote talk by Sangtae Kim (NSF) 1100 - 1130hrsCoffee Break1130 - 1230hrsKeynote Talk by Mike Brady (Oxford)1230 - 1400hrsLunch1400 - 1515hrsParallel Session 1 Mini-Workshop: Experiences of OGSA-DAIRoom: AuditoriumOrganiser: Mario AntoniolettiProtecting Application Developers – A Client Toolkit for OGSA-DAITom SugdenBioinformatics Data and the Grid: The GeneGrid Data ManagerNoel KellyFirstDIG: Data investigations using OGSA-DAIT SloanPerformance Analysis of the OGSA-DAI SoftwareMike JacksonSession 1.1: Sensor Arrays and Remote SensingRoom: Conference Room 1Sensor Grids for Air Pollution MonitoringMark RichardsDistributed BioSensor systems for GM Crop MonitoringS HassardImageodesy on MPI & grid for co-seismic shift study using satellite imageryJ. G. LiuSession 1.2: JISC ProjectsRoom: Conference Room 2Production Quality e-Science Grid Andrew RichardsFrom Data Deluge to Data CurationPhilip LordDeploying a distributed data storage system for grid applications on the UK National Grid Service using federated SRBA.S. ManandharSession 1.3: XML TechnologiesRoom: Conference Room 3Developments in BinX, the Binary XML description languageRobert CarrollDiverse data to diverse visualization systems end to endJulian GallopA Framework for Text Mining ServicesEwan KleinSession 1.4: Resource MonitoringRoom: Conference Room 4Ubiquitous Grid Resource MonitoringMark BakerProduction services for information and monitoring in the GridJohn WalkGrid network performance monitoringMark Leese1515 - 1600hrsCoffee Break1600 - 1740hrsParallel Session 2 Mini-Workshop: Experience of Running Production Grid ServicesRoom: AuditoriumOrganiser: Robin MiddletonResource Brokering: The EUROGRID/GRIP ApproachDonal FellowsLeveraging UK eScience with very large Condor pools: the demand for transforming significant wasted power into results.Paul B, WilsonThe GRIDPP Tier1 CentreAndrew SansumGrid tool integration within the eMinerals projectMark CallejaExperience in Running LCG as a Production Grid ServiceIan BirdSession 2.1: Provenance and MetadataRoom: Conference Room 1Formalising a protocol for recording provenance in GridsPaul GrothUse of Data Provenance and the Grid in Medical Image Analysis and Drug Discovery - An IXI ExemplarKelvin LeungThe myGrid Information ModelNick SharmanSupport for Provenance in a Service-based Computing GridShrija RajbhandariSession 2.2: Collaborative Tools and EnvironmentsRoom: Conference Room 2Collaborative tools in support of the eMinerals Virtual OrganisationMartin DoveChain ReAKTing: Collaborative Advanced Knowledge Technologies in the Comb-e-Chem GridDavid De RoureIntroducing the Access Grid Support CentreMichael, Daw   Paper withdrawn Session 2.3: Workflow CompositionRoom: Conference Room 3Composing workflows in the environmental sciences using InfernoJon BlowerAutomated Composition of Semantic Grid ServicesShalil MajithiaWorkflow Support for Advanced Grid-Enabled ComputingFenglian, XuWorkflow Advisor on The GridMax OngSession 2.4: Visualisation and SteeringRoom: Conference Room 4Grid enabled application visualisation services for Oceanographic diagnostics studiesLakshmi SastryApplying the GRID to cartoon animationPaul CockshottDeveloping a Roaming PDA-Based Interface for a Steering Client for OGSI::Lite using .Net: Practical Lessons LearnedSimon P,NeeImportant issues concerning interactive user interfaces in grid based computational steering systemsRoy Kalawsky1800 - 2200hrsDemonstrations, Poster Session & BuffetThursday 2 September 20040900 - 1000hrsKeynote talk by Ian Foster, Argonne National Laboratory and University of Chicago1000 - 1025hrsCoffee Break1025 - 1230hrsParallel Session 3 Mini-Workshop: Computation Steering and Visualisation on the Grid: Pratice & ExperienceRoom: AuditoriumOrganiser: Ian Grimstead, Steven KennyThe importance of locality in the visualization of large datasetsJohn BrookeVisualisation on the Grid: A Web Service ApproachStuart M. ChartersPutting Visualization First in Computational SteeringHelen WrightPractical Tools for Computational SteeringA R PortergViz: Visualization and Computational Steering on the GridKen BrodlieThe TeraGyroid project -- collaborative steering and visualization in an HPC grid for modelling complex fluidsJonathan ChinSession 3.1: Ontologies and the Semantic GridRoom: Conference Room 1Designing User Interfaces to Minimise Common Errors in Ontology Development: The CO-ODE and HyOntUse ProjectsMatthew Horridge, Nick Drummond, Hai WangLess is More: Lightweight Ontologies and User Interfaces for Smart LabsHugo MillsApplying the Semantic Web to Manage Knowledge on the GridFeng TaoeBank UK – linking research data, scholarly communication and learning. Simon Coles & Liz LyonMIAKT: Combining Grid and Web Services for Collaborative Medical Decision MakingNigel ShadboltSession 3.2: Support for Virtual OrganisationsRoom: Conference Room 2Access Control for Dynamic Virtual OrganisationsDuncan RussellSupporting Formation and Operation of Virtual Organisations in a Grid EnvironmentJ ShaoUsing the VOM portal to manage policy within Globus Toolkit, Community Authorisation Service & ICENI resourcesAsif SaleemSupporting Collaborative Virtual Organisations in the Construction Industry via the GridJaspreet, Singh Pahwa; Pete, BurnapPrivacy-Enhanced Transactions for Virtual OrganisationsErica Y. YangSession 3.3: Data Services and MiddlewareRoom: Conference Room 3Eldas (Enterprise Level Data Access Services)Stephen RutherfordThree Possible Mappings for the DAIS Concepts: WS-I, WS-I plus WS-Context, and WS-RF SpecificationsSimon LawsOGSA-DAI Status Report and Future DirectionsNeil P. Chue HongOGSA-DAI Usage Scenarios and Behaviour: Determining good practiceMario AntoniolettiCondor services for the Global Grid: Interoperability between Condor and OGSAClovis ChapmanSession 3.4: Job Submission and SchedulingRoom: Conference Room 4AliBaBa: Running BaBar jobs on the grid using gsubMike JonesA Standards Based Approach To Job Submission Through Web ServicesWilliam LeeWorkflow Enactment in ICENIStephen McGoughTOG and JOSH: Grid scheduling with Grid Engine and GlobusT SloanPerformance guided scheduling in GENIE through ICENIMurtaza Gulamali1230 - 1410hrsLunch1410 - 1550hrsParallel Session 4 Mini-Workshop: Requirements Capture and Analysis in e-Science ProjectsRoom: AuditoriumOrganiser: Marina JirotkaUser requirements for UK e-Science grid environmentsBruce BecklesEnterprise specification of the NERC DataGridAndrew WoolfGathering Requirements for an Integrative Biology ProjectLloyd,SharonTowards Understanding Requirements for eScience: the eDiaMoND case studyMarina JirotkaSession 4.1: Data Management and Storage IRoom: Conference Room 1Applying the OAIS standard to CCLRC’s British Atmospheric Data Centre and the Atlas Petabyte Storage ServiceDr David GiarettaDeployment and Exploitation of Grid-enabled Data Management for EngineersJasmin WasonEfficient Data Storage and Analysis for Generic Biomolecular Simulation DataMuan Hong NgMySpace: distributed data storage for the VOKeith NoddleSession 4.2: Middleware and InfrastructureRoom: Conference Room 2Extending Globus to support Multicast TransmissionKarl JeacleReliable Multicast for the Grid: a comparison of protocol implementationsMarinho P. BarcellosChained Negotiation for Distributed Notification ServicesRichard LawleyRemoving digital certificates from the end-user’s experience of grid environmentsBruce BecklesSession 4.3: Portals and Problem-Solving Environments IRoom: Conference Room 3Virtual Research in the UK: Advanced Portal ServicesMark BakerPortal Framework for Computation within the eMinerals ProjectDr R TyerBuilding a Biodiversity Problem-Solving EnvironmentRichard WhiteGRENADEStephen PicklesSession 4.4: Service-Oriented Architectures and GridsRoom: Conference Room 4RealityGrid: An Integrated Approach to Middleware through ICENIAnthony MayerGeneGrid: A practical Workflow Implementation for a Grid Based Virtual Bioinformatics LaboratoryDavid SimpsonDeveloping LHCb Grid Software: Experiences and AdvancesIan Stokes-ReesService-Oriented Architecture on the Grid for FDI IntegrationX Ren1550 - 1620hrsCoffee Break1620 - 1820hrsBoFsBoF: Usability/Interaction in eScienceFriday 3 September 20040900 - 1040hrsParallel Session 5 Mini-Workshop: e-Science and Data MiningRoom: AuditoriumOrganiser: Bob MannThe e-Science and Data Mining Special Interest Group: Launch, Aims and Preliminary Requirements AnalysisBob MannPattern Matching Against Distributed DatasetsMark JessopWhy Grid-based Data Mining Matters? Fighting Natural Disasters on the Grid: From SARS to Land SlidesYike Guoe-Science Tools for Analysing Complex SystemsOlusola C, IdowuMapping of Scientific Workflow within the e-Protein project to Distributed ResourcesAngela OBrienAssociation of variations in I kappa B-epsilon with Graves’ disease using classical and myGrid methodologiesPeter LiSession 5.1: Algorithms and ApplicationsRoom: Conference Room 1Discovery Processes in Discovery NetJameel SyedScience outcomes from the use of Grid tools in the eMinerals projectStephen WellsA Grid Enabled Visual Tool for Time Series Pattern MatchB. LiangThe Application of Distributed Computing to the Investigation of Protein Conformational ChangeChristopher WoodsSession 5.2: Information Management and DiscoveryRoom: Conference Room 2The NERC DataGrid: Googling Secure DataBryan LawrenceGrid Services Supporting the Usage of Secure Federated, Distributed Biomedical DataRichard SinnottPrinciples of Personalisation of Service DiscoverySimon MilesTowards a grid enabled Engineering Body ScannerKevin T W TanSession 5.3: Bio-Informatics and the GridRoom: Conference Room 3Developing Grid-based Systems for Microbial Genome Comparisons: The Microbase ProjectAnil WipatExploring Williams-Beuren Syndrome Using myGridRobert StevensIntegrating Biomedical Text Mining Services into a Distributed Workflow EnvironmentRob GaizauskasSARS Analysis on the GridVasa CurcinSession 5.4: Services and Toolkits for the GridRoom: Conference Room 4CamGrid: Experiences in constructing a university-wide, Condor-based, grid at the University of CambridgeBruce BecklesUser Deployment of Grid Toolkits to EngineersSimon J, CoxIntegration of chargeable Web Services into Engineering Applications M MolinariRecycling Services and Workflows through Discovery and ReuseChris Wroe1040 - 1110hrsCoffee Break1110 - 1315hrsParallel Session 6 Mini-Workshop: Grid Performability Modelling and MeasurementRoom: AuditoriumOrganiser: Nigel ThomasPerformance Architecture within ICENIStephen McGoughOptimal Tree Structures for Large-Scale GridsJ. PalmerjGMA: A lightweight implementation of the Grid Monitoring ArchitectureMatthew, GrovePerformance of a semi blind service schedulerNigel ThomasPerformance Modelling of a Self-adaptive and Self-optimising Resource Monitoring System for Dynamic Grid EnvironmentsDr Stephen A, JarvisSession 6.1: Healthcare and the GridRoom: Conference Room 1Artificial Neural Networks in Cancer ManagementRobin MarshallDesigning Grid-enabled Image Registration Services For MIAKTYalin ZhengIntegrative Biology - exploiting e-Science to combat fatal diseasesDamian F. Mac Randale-DiaMoND: challenges for the e-Scientist in the e-Health domainSharon LloydJoining upHealth and BioInformatics: E-Science meets E-HealthAlan RectorSession 6.2: Portals and Problem-Solving Environments IIRoom: Conference Room 2e-HTPX – HPC, Grid and Web-Portal Technologies in High Throughput Protein CrystallographyDr David MeredithThe CCLRC Data PortalGlen DrinkwaterA Portlet Service Model for GECEMMaria LinReDReSS Portal Services for Awareness and TrainingRob CrouchleyThe RealityGrid Web PortalKevin StratfordSession 6.3: Data Management and Storage IIRoom: Conference Room 3Replica Management Services in the European DataGrid ProjectDavid CameronTuning GENIE Earth System Model Components using a Grid Enabled Data Management SystemAndrew PriceAxiope – the SASIAR approach to bioscience data management Fred HowellEnvironmental Molecular Processes: Management of Simulation Data and AnnotationLisa BlanshardSRB in a Production ContextPeter BerrisfordSession 6.4: Application GridsRoom: Conference Room 4A Grid for Particle Physics - From Testbed to ProductionJ. ColesGridCast: A Service Architecture for the Broadcasting MediaTJ HarmerFinGrid: Financial Information Grid –an ESRC e-Social Science Pilot Project for the Financial MarketsK. AhmadBioinformatics Application Integration and Management in GeneGrid: Experiments and ExperiencesP.V. JitheshExperiences in Setting up a Pan-European Datagrid using QCDgrid technology as part of the ENACTS Demonstrator ProjectChris Johnson1315 - 1445hrsLunch1445 - 1545hrsKeynote talk by Bob Jones (CERN)1545 - 1615hrsPrize Draw GridPP1 and AHM

  24. AHM News • Grid-lock?ZDNet.co.uk, UK - Sep 6, 2004Last week, UK particle physicists demonstrated the world’s largest working grid at the e-Science All-Hands meeting in Nottingham. ... [more] • World's biggest grid unveiled by UK scientistsSilicon.com, UK - Sep 6, 2004Particle physicists in the UK will demonstrate the world's largest working computer grid this week, at the e-Science All Hands meeting in Nottingham. ...[more] • World's largest computing grid goes livePC Pro, UK - Sep 6, 2004The world's largest permanent computing grid has gone online, giving scientists access to the processing power of 6,000 computers at 78 locations, including ...[more] • World's Largest Working Computing GridPhysOrg.com, United States - Sep 5, 2004Particle physics experiments at the Large Hadron Collider (LHC), currently under construction at CERN in Geneva will produce around 15 Petabytes of data each ...[more] • Brits to demo world's largest computing gridThe Register, UK - Sep 3, 2004Particle physicists in the UK will demonstrate the world's largest working computer grid this week, at the e-Science All Hands meeting in Nottingham. ...[more] • The Grid becomes a realityInnovations-Report, Germany - Sep 3, 2004This week, UK particle physicists have demonstrated the world’s largest, working computing Grid. With over 6,000 computers at ...[more] • GRID Computing a reality: UK trial this weekPublicTechnology.net, UK - Sep 3, 2004... Grid computing has been a target for IT developers and scientists for more than five years. It allows scientists to access computer ...[more] • Grid computer moves to second stageEE Times Online (subscription) - Sep 2, 2004At the 2004 UK e-Science All Hands Meeting in Nottingham, particle physicists representing a collaboration of 20 UK institutions will explain to biologists ... [more] • The Grid becomes a realitySpace Ref - Sep 1, 2004... At the 2004 UK e-Science All Hands Meeting in Nottingham, particle physicists representing a collaboration of 20 UK institutions will explain to biologists ... [more] • All the AHM news was from GridPP and LCG… GridPP1 and AHM

  25. Current LCG-2 sites: 7/9/04 • 73 Sites • 7700 CPU • 26 sites at 2_2_0 • 33 sites at 2_1_1 • others at ?? • 29 pass all tests GDB Meeting – 8 September 2004 - 25

  26. Data Challenges • Ongoing.. • Grid and non-Grid Production • Grid now significant • ALICE - 35 CPU Years • Phase 1 done • Phase 2 ongoing LCG • CMS - 75 M events and 150 TB: first of this year’s Grid data challenges GridPP1 and AHM

  27. Data Challenge • 7.7 M GEANT4 events and 22 TB • UK ~20% of LCG • Ongoing.. • (3) Grid Production > 1000 CPU years • Largest single Challenge over the summer • Small fraction of what ATLAS need.. GridPP1 and AHM

  28. LHCb Data Challenge 186 M Produced Events Phase 1 Completed 3-5 106/day LCG restarted LCG paused LCG in action 1.8 106/day DIRAC alone 424 CPU years, 186M events • UK’s input significant (>1/4 total) • LCG(UK) resource: • Tier-1 7.7% • Tier-2 sites: • London 3.9% • South 2.3% • North 1.4% • DIRAC: • Imperial 2.0% • L'pool 3.1% • Oxford 0.1% • ScotGrid 5.1% GridPP1 and AHM

  29. Transition to Grid… DIRAC-LCG Share 424 CPU · Years May: 89%:11% 11% of DC’04 Jun: 80%:20% 25% of DC’04 Jul: 77%:23% 22% of DC’04 Aug: 27%:73% 42% of DC’04 GridPP1 and AHM

  30. Issues First large-scale Grid production problems being addressed… at all levels GridPP1 and AHM

  31. Issues GridPP1 and AHM

  32. Issues GridPP1 and AHM

  33. Issues GridPP1 and AHM

  34. Issues GridPP1 and AHM

  35. Issues GridPP1 and AHM

  36. Issues GridPP1 and AHM

  37. Issues GridPP1 and AHM

  38. Coordinates resources that are not subject to centralized control … using standard, open, general-purpose protocols and interfaces … to deliver nontrivial qualities of service YES. This is why development and maintenance of LCG is important. YES. VDT (Globus/Condor-G) + EDG/EGEE(Glite) ~meet this requirement. YES. LHC experiments data challenges over the summer of 2004. 5 Are we a Grid? http://www-fp.mcs.anl.gov/~foster/Articles/WhatIsTheGrid.pdf http://agenda.cern.ch/fullAgenda.php?ida=a042133 GridPP1 and AHM

  39. What was GridPP1? • A team that built a working prototype grid of significant scale > 1,000 (7,000) CPUs > 500 (4,000) TB of storage > 800 (6,000) simultaneous jobs • A complex project where 82% of the 190 tasks for the first three years were completed A Success “The achievement of something desired, planned, or attempted” GridPP1 and AHM

  40. What lies ahead? Some mountain climbing.. Annual data storage: 12-14 PetaBytes per year CD stack with 1 year LHC data (~ 20 km) Non-Grid Approach?: Importance of step-by-step planning… Pre-plan your trip, carry an ice axe and crampons and arrange for a guide… 100 Million SPECint2000 Concorde (15 km) In production terms, we’ve made base camp  100,000 PCs (3 GHz Pentium 4) We are here (1 km) Quantitatively(?), we’re ~7% of the way there in terms of CPU (7,000 ex 100,000) and disk (4 ex 12-14*3-4 years)… GridPP1 and AHM

  41. GridPP2: first steps from base camp Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 2001 2006 2007 2002 2003 2004 2005 I. Experiment Layer II. Application Middleware III. Grid Middleware IV. Facilities and Fabrics User Board Importance of step-by-step planning… But Mountain climbing is also about individual feats of endurance as well as teams working together Deployment Board Finally we need to get to concorde heights to get to where we need to be.. GridPP1 and AHM

  42. History teaches us.. All interactions 9 orders of magnitude The HIGGS • “When you are face to face with a difficulty you are up against a discovery” Lord Kelvin • “Radio has no future” • “X-rays will prove to be a hoax” GridPP1 and AHM

  43. Summary • Midway through the GridPP six-year project the glass is.. • half-full GridPP1 and AHM

More Related