1 / 17

Jean-Yves Nief, CC-IN2P3 Wilko Kroeger, SCCS/SLAC Adil Hasan, CCLRC/RAL

BaBar data distribution using the Storage Resource Broker (SRB). Jean-Yves Nief, CC-IN2P3 Wilko Kroeger, SCCS/SLAC Adil Hasan, CCLRC/RAL. HEPiX, SLAC October 11th – 13th, 2005. BaBar overview. Large amount of data produced by BaBar: > 800 TB. Using the collaboration Computing Centres:

uta
Download Presentation

Jean-Yves Nief, CC-IN2P3 Wilko Kroeger, SCCS/SLAC Adil Hasan, CCLRC/RAL

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. BaBar data distribution using the Storage Resource Broker (SRB) Jean-Yves Nief, CC-IN2P3 Wilko Kroeger, SCCS/SLAC Adil Hasan, CCLRC/RAL HEPiX, SLAC October 11th – 13th, 2005

  2. BaBar overview. • Large amount of data produced by BaBar: > 800 TB. • Using the collaboration Computing Centres: • Data (re)processing (prompt reco, skimming). • Simulation production. • Data analysis  allow foreign collaborators to access data. • 6 Tier A: • SLAC stores all the data + simulation: central repository. • Computing tasks divided among all the Tier A sites. HEPiX Conference, SLAC, October 11th-13th 2005

  3. BaBar data organization. + simulation production (~20 sites involved)  Lots of data movement HEPiX Conference, SLAC, October 11th-13th 2005

  4. Requirements. • Short turn-around between data production and analysis. • Data must be transfered efficiently and quickly to Computing Centres. • Data transfer tools must be: • Reliable. • Robust. • Automated. • Scalable. • Easy to use. HEPiX Conference, SLAC, October 11th-13th 2005

  5. SLAC & CC-IN2P3. • First analysis centres opened to the BaBar collaboration. • Both held complete copies of data (Objectivity). • Now only SLAC old a complete copy of the data. • Natural candidates for testing and deployment of grid middleware. • SRB: good candidate for data distribution. HEPiX Conference, SLAC, October 11th-13th 2005

  6. What’s SRB ? • Storage Resource Broker: developed by SDSC (San Diego). • Provides an uniform interface to heterogeneous storage system (disk, tape, databases) for data distributed in multiple sites. • Who is using SRB ? • Biology, biomedical applications (e.g: BIRN). • Astrophysics, Earth Sciences (e.g: NASA). • Digital libraries (e.g: NARA). • Used world wide: USA, Europe, Asia, Australia. HEPiX Conference, SLAC, October 11th-13th 2005

  7. Some SRB features. • Files organized in a logical space (directories, subdirectories …): logical view totally decoupled from the physical location of the files. • Replica management. • Search for files based on their attributes (metadata). • Fine granularity for SRB users’ACLs. • Interfaced with various Mass Storage System: • Access to HPSS transparent to users. • Small files management. • And many more… HEPiX Conference, SLAC, October 11th-13th 2005

  8. MCAT SRB SRB SRB Site 2 Site 3 (4) (1) (2) SRB architecture. • 1 zone: • 1 SRB/MetaCatalog server: contains list of files, physical resources, users registered. • several SRB servers to access the data at their physical location. (3) Site 1 test1.txt Application (asking for test1.txt) HEPiX Conference, SLAC, October 11th-13th 2005

  9. (2) (1) (3) HPSS/Lyon HPSS/SLAC MCAT SRB SRB SRB client SRB SRB BaBar architecture (2001-2004). 1 Zone (SLAC) client CC-IN2P3 (Lyon) SLAC (Stanford, CA) HEPiX Conference, SLAC, October 11th-13th 2005

  10. Extra details. • Hardware: • SUN servers (Solaris 5.8, 5.9): NetraT 1405, V440. • Software: • Oracle 8, 9 for the SLAC MCAT (530,000 files registered). • Multithreaded client application (Python). • Advantages: • All the 3 steps controlled by the client application in //  no latency due to prestaging. • Small amount of cache needed on both sides (a few GBs). • Drawbacks: • No registration into the MCAT of the files stored in Lyon. • Performance: • Up to 3 TB/day from tape to tape. HEPiX Conference, SLAC, October 11th-13th 2005

  11. ESNET Traffic with one server on both sides (April 2004). SLAC (US)  IN2P3 (FR) 1 Terabyte/day Fermilab (US) CERN SLAC (US) INFN Padva (IT) Fermilab (US)  U. Chicago (US) U. Toronto (CA)  Fermilab (US) Helmholtz-Karlsruhe (DE) SLAC (US) CEBAF (US)  IN2P3 (FR) INFN Padva (IT)  SLAC (US) Fermilab (US)  JANET (UK) SLAC (US)  JANET (UK) DOE Lab  DOE Lab Argonne (US)  Level3 (US) DOE Lab  DOE Lab Fermilab (US)  INFN Padva (IT) Argonne  SURFnet (NL) IN2P3 (FR)  SLAC (US) HEPiX Conference, SLAC, October 11th-13th 2005

  12. (2) (1) (3) HPSS/Lyon HPSS/SLAC SRB MCAT SRB MCAT SRB SRB SRB SRB BaBar architecture (since 2005). 2 Zones (SLAC + Lyon) SRB CC-IN2P3 (Lyon) SLAC (Stanford, CA) HEPiX Conference, SLAC, October 11th-13th 2005

  13. Extra details. • Hardware: • Still the same. • Software: • Oracle 10g for the SLAC MCAT. • Oracle 9i for the Lyon MCAT (migration to 10g foreseen). • MCATs synchronization: only users and physical resources. • Comparison of the MCATs contents to transfer the data. • Advantage: • External client can pick up data from SLAC or Lyon without interacting with the other site. HEPiX Conference, SLAC, October 11th-13th 2005

  14. Overall assessment. • A lot of time saved for developping applications thanks to the SRB. • Transparent access to data: • Very useful in an hybrid environment (disk, tape). • Easy to scale the service (adding new servers). • Not dependent of physical locations changes in the application. • Fully automated procedure. • Easy for SLAC to recover corrupted data. • 132 TB (232,000 files) shipped to Lyon. • Up to 3 TB /day from tape to tape (minimum latency). HEPiX Conference, SLAC, October 11th-13th 2005

  15. RAL SRB HPSS/Lyon HPSS/SLAC CC-IN2P3 SLAC SRB SRB Future plans. • Possibly include RAL as part of the system: • Would reduce the network activity on the SLAC & ESNET. • Some tests: • Assessment of web services technologies. HEPiX Conference, SLAC, October 11th-13th 2005

  16. Summary and outlook. • SRB used by BaBar for more than 3 years. • Data shipment fully automated. • Lightweight development to make it work. • Easy administration. • More servers on both side: • Increase peak transfer rate to 5 TB/day. • Hundreds of TB foreseen. • Scalability: the sky is the limit… well and the money too… • Aiming at including RAL. HEPiX Conference, SLAC, October 11th-13th 2005

  17. Never forget… Cannot be achieved without the work of: • The network and system admins. • The database admins. Many thanks to them! HEPiX Conference, SLAC, October 11th-13th 2005

More Related