1 / 10

The ALICE Transfers

This presentation discusses the file transfer service tests conducted using the FTS (File Transfer Service) and FTD (File Transfer Daemon) for scheduled replication of data between computing centers in the ALICE experiment. The stability and integration of FTS and FTD in transferring different types of data are evaluated, along with the challenges faced and improvements made. The findings and future plans for deployment are also discussed.

idam
Download Presentation

The ALICE Transfers

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The ALICE Transfers • Patricia Méndez Lorenzo (CERN, IT/PSS) • On behalf of the ALICE Collaboration • LCG-France Workshop • Clermont, 15th March 2007

  2. File Transfer Service Tests • FileTransfer Service deployed at all sites • Used for scheduled replication of data between computing centers • Used as plug-in in the AliEn File Transfer Daemon (FTD) • FTS has been implemented through the FTS Perl APIs • FTD running in the VO-box as one of the ALICE services • Access to the SRM SE at all sites also required

  3. File Replication Strategy Submits job User ALICE central services Optimizer Update FC Origin GUID Space reservation to SRM Destination SURL GUID Update VO-Box LCG ALICE File Catalogue ALICETrnasfer queue User Job ALICE catalogues Transfer broker FTD SA FTD SA SRM FTS SRM LFC LFC

  4. Automatization of FTS • Raw file inserted in the catalogue • Trigger issues a transfer to a random T1 • FTD requests the transfer • FTD uses FTS to transfer the file • FTD notifies that the transfer has finished • Transfer manager updates the catalogue AliEn File Catalogue 1 2 Transfer queue 6 Transfer Manager Transfer broker 5 3 FTD FTD 4 MSS MSS

  5. FTS Tests: Strategy • The main goal is to test the stability of FTS as service and integration with FTD • T0-T1 (disk to tape): 7 days required of sustained transfer rates to all T1s • Exercise still continuing • T1-T2 (disk to disk) and T1-T1 (disk to disk): 2 days required of sustained transfers to T2 • STILL TO BE FULLY PERFORMED • Data types • T0-T1: Migration of raw and 1st pass reconstructed data • T1-T2 and T2-T1: Transfers of ESDs, AODs (T1-T2) and T2 MC production for custodial storage (T2-T1) • T1-T1: Replication of ESDs and AODs

  6. Data Movement: MonaLisa Most of the exercise counting on 3 sites of 5 in total Most of the problems reported via GGUS Still to be improved Good support from thr Service Challenge team

  7. CERN-CCIN2P3 Transfers CCIN2P3 – 220 TB of data stored (40% of total) Average transfer rate - 40MB/s

  8. Data Movement: DashBoard • Created by the ARDA team to report FTS problems to the experts

  9. Data Distribution Experiences • T0-T1: disk-tape transfers at an aggregate rate of 300MB/s from CERN was required • Main Problems observed • Instabilities in FTS, storage, VO-boxes, ALICE services to sustain the goal rate (300MB/s) in November-December • We have now a good understanding how to use FTS and its current limitations • FTS is fully integrated in the ALICE high-level services • Expert support was excellent • The whole exercise could only use 3 out of 5 sites for most of the time • Instabilities at SARA (VOBOX down, SE down..) • At RAL the speed has improved only after we had access to CASTOR2 • Distributed according the MSS resources pledged by the sites in the LCG MoU: • CNAF: 20%, CCIN2P3: 20%, GridKA: 20%, SARA: 10%, RAL: 10%

  10. New Bunch of FTS ALICE Transfers • At the end of 2006 the exercise was over • ALICE declared to be part of the multi-VO exercise • We will be back in transfer production with FTS2.0 not before • We consider it makes no sense to repeat the same bunch of exercises performed with FTS1.5 • Deployment on April (at CERN), same time than ALICE dress rehearsal

More Related