1 / 36

HEP Data Grid in Japan

HEP Data Grid in Japan. Takashi Sasaki Computing Research Center KEK. Contents. Japanese HENP projects Belle GRID ATLAS-Japan Regional Center Collaboration in Asia Pacific. Major HENP projects in Japan. KEK Proton Synchrotron K2K etc. KEKB BELLE experiment

dawson
Download Presentation

HEP Data Grid in Japan

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. HEP Data Grid in Japan Takashi Sasaki Computing Research Center KEK

  2. Contents • Japanese HENP projects • Belle GRID • ATLAS-Japan Regional Center • Collaboration in Asia Pacific Takashi.Sasaki@kek.jp

  3. Major HENP projects in Japan • KEK Proton Synchrotron • K2K etc. • KEKB • BELLE experiment • 300 members from 54 institutes in10 countries • Super-B is planning • J-PARC • New facility under construction at Tokai (~2008)  • particle physics, material science and life science • T2K • International collaboration • LHC ATLAS, CERN • HERA ZEUS, DESY • RHIC, BNL • etc. Takashi.Sasaki@kek.jp

  4. KEK-B Belle Experiment Takashi.Sasaki@kek.jp

  5. 1km Takashi.Sasaki@kek.jp

  6. Belle Data • 15MB/sec data rate in maximum • 200TB/year raw data recorded • Equivalent number of Monte Carlo events Takashi.Sasaki@kek.jp

  7. Belle PC farms Takashi.Sasaki@kek.jp

  8. Super KEK-B • L = 10^35cm^-2s^-1 in 2007? • Data rate ~ 250MB/s Takashi.Sasaki@kek.jp

  9. K2K now upgrading to T2K Takashi.Sasaki@kek.jp

  10. T2K (Tokai to Kamioka) • J-PARC • for particles physics, material and life science • joint project of JAERI and KEK • 100 times intense neutrino beam • High trigger rate at the near detector • Operational in 2007 Takashi.Sasaki@kek.jp

  11. LHC-ATLAS • Japanese contribution • Semiconductor Tracker • Endcap Muon Trigger System • Muon Readout Electronics • Superconducting Solenoid • DAQ system • Regional Analysis Facility • ICEPP will the site of the regional center Takashi.Sasaki@kek.jp

  12. HEP GRID related activities • Networking • domestic/international • BELLE • data distribution and remote MC production • ALTAS Japan • collaboration between ICEPPand KEK • Hadron therapy simulation • application of HEP tools into medical field Takashi.Sasaki@kek.jp

  13. SuperSInet Major HEP sites have a DWDM connection to KEK with 1Gbps Takashi.Sasaki@kek.jp

  14. To Novosibirsk Russia 0.5 M USA SuperSINET Chicago Tsukuba WAN 5 G -> 10G TransPAC Korea NY APII 1G SuperSINET China 10 G CERNET 155 M Hawaii 155M APAN Current Network Connections around Japan ASNET/TANET2 622 M SINET2M Taiwan Thailand Takashi.Sasaki@kek.jp As of July 2004

  15. BELLE GRID • Distributed analysis and data sharing among institutes • Experimental data distribution to remote sites • Monte Carlo production at remote sites and send back events to KEK • remote job submission • etc • Testing SRB with GSI now • among Australia, Taiwan and KEK • hopefully Korea, soon Takashi.Sasaki@kek.jp

  16. Why SRB? • Easy to install, use and manage • Most of Belle’s collaborating institutes are middle or small size universities • poor in man power • Useful features • file replication • parallel data transfer • Grid aware: GSI authentication • Command line interface, API, GUI • Fancy Web interface • etc • Excellent user support • quick response • seminars • Available today! • Solution is needed for Belle Takashi.Sasaki@kek.jp

  17. SRB(Storage Resource Broker) SRB client SRB client Internet Zone SRB server SRB server SRB server SRB server tape RAID disk nfs DB MCAT Takashi.Sasaki@kek.jp

  18. SRB zone federation disk DB MCAT ZoneB SRB server SRB server SRB server Internet SRB server SRB server SRB server SRB server ZoneA tape RAID disk nfs DB MCAT Takashi.Sasaki@kek.jp

  19. SRB Command line interface Takashi.Sasaki@kek.jp

  20. SRB test bed system Australia KEK DMZ ZONE:gtsrb13 Internet store Fire Wall gtdmz01 ZONE:kekgt15 BELLE Secure net MCAT NFS (Belle data) bcs20 MCAT PostgreSQL client BELLE Computer system ZONE:anusf RAID 800GB MCAT DB2 gt15 gt13 federation MCAT PostgreSQL gl03 gl01 HPSS 120TB ZONE:glsrb01 Takashi.Sasaki@kek.jp

  21. Status • Zone federation between ANU and KEK has been established • ANU, University of Melbourne and University of Sydney are collaborating for BELLE/ATLAS Grid issue • One MCAT running at ANU • Data can store/retrieve on Belle data system and also HPSS in KEK side • Zone federation between Academia Sinica, Taiwan and KEK has been established • still need to solve some problems Takashi.Sasaki@kek.jp

  22. BELLE GRID Future plan • Participation of Korean sites • Mutual job submission using Globus • LCG2+SRB (if they wish) • LCG2 is under testing at KEK also with help of ICEPP. • Because many foreign institutes are working both for BELLE and one of LHC experiments, they want to use LCG rather than vanilla Globus • baby “tier-0” at KEK and “tier-1” at ICEPP • SRB and LCG-RLS synchronization will be tried based on Simon Matson’s (Bristol, CMS) work Takashi.Sasaki@kek.jp

  23. Grid in ATLAS Japan • Regional analysis center for ATLAS • ICEPP, the University of Tokyo • Joint collaboration between ICEPP and KEK for Grid deployment Takashi.Sasaki@kek.jp

  24. LCG MW Deployment in Japan • RC Pilot Model System@ICEPP • Since 2002 • LCG testbed. Now LCG2_1_1 • LCG2 test bed@KEK • baby tier-0 • LCG2_1_1 • Regional Center Facility • Will be introduced in JFY2006 • Aiming “Tier1” size resource Takashi.Sasaki@kek.jp

  25. Takashi.Sasaki@kek.jp

  26. SuperSInet Performance Measurement “A” setting TCP 479Mbps -P 1 -t 1200 -w 128KBTCP 925Mbps -P 2 -t 1200 -w 128KBTCP 931Mbps -P 4 -t 1200 -w 128KBUDP 953Mbps -b 1000MB -t 1200 -w 128KB “B” setting (longer window size) TCP 922Mbps -P 1 -t 1200 -w 4096KBUDP 954Mbps -b 1000MB -t 1200 -w 4096KB DWDM "A" setting: 104.9MB/s "B" setting: 110.2MB/s Takashi.Sasaki@kek.jp

  27. NorduGrid - grid-manager - gridftp-server Globus-mds Globus-replica PBS server NorduGrid - grid-manager - gridftp-server Globus-mds PBS server 1Gbps 100Mbps GRID testbed environmentwith HPSS through GbE-WAN HPSS servers ICEPP KEK SE HPSS 120TB CE SE ~ 60km 0.2TB CE 6CPUs CE 100 CPUs PBS clients PBS clients User PCs Takashi.Sasaki@kek.jp

  28. Pftp→pftp HPSS mover disk Client disk 80 Even 3ms latency affects on results 60 to/dev/null Aggregate Transfer speed (MB/s) KEK client (LAN) 40 ICEPP client(WAN) 20 Ftp buffer=64MB to clientdisk Client disk speed @ KEK = 48MB/s Client disk speed @ ICEPP=33MB/s 0 0 2 4 6 8 10 # of file transfer in parallel Takashi.Sasaki@kek.jp

  29. microscopic monitoring on network performance w/ packet loss w/o packet loss • speed=increment of sum of TCP data size in every 10ms • window size grows slowly after packet losss • longer windows is not perfect when you have packet loss Takashi.Sasaki@kek.jp

  30. FAST • http://netlab.caltech.edu/FAST/ • Looks nice and we want to try, but we haven’t because of the patent and IP issue. • Some people at KEK afraid that they might have a difficulty to work in the similar topic once they see the source code of it. • Is this true? Takashi.Sasaki@kek.jp

  31. CA • A key issue of GRID • ICEPP and KEK will run one CA jointly for BELLE and ATLAS • only for active KEK users to simplify the procedure • current situation • ICEPP depends on a foreign CA to join LCG • KEK is running a test CA locally • CA management costs are not cheap. Any good idea? Takashi.Sasaki@kek.jp

  32. Storage evaluation • SAN solutions • IBM SANfile (aka. StorageTank) • Linux+AIX in server side • tested AIX, Solaris and Linux clients • Linux clients were fastest in our tests • HP Lustre • waiting for beta product delivery Takashi.Sasaki@kek.jp

  33. Distributed simulation in advanced radio therapy • A model on hospitals and computer centers • Hospitals send CT, MRI or PET images (DICOM) with treatment planning data to a computing center as input • higher security is required to protect personal data • Full simulation using Geant4 at computing centers • parallel simulation • Analysis results and feedbacks in DICOM to hospitals • Validation of treatment planning Takashi.Sasaki@kek.jp

  34. Toward Asia-Pacific collaboration • Taka an advantage on working together with people in neighboring time zones • HEP population in Asia-Pacific • ATLAS • Australia(7), China(15), Japan(45) and Taiwan(5) • 72/1446 = 5.0% • CMS • China(31), Korea(17) and Taiwan(14) • 62/1676 = 3.7% • Alice • China(20), Japan(3) and Korea(12) • 35/747 = 4.7% • LHC-b • China(19) • 19/737 = 2.6% • Belle • Australia(8),China(11),Japan(122),Korea(15) and Taiwan(13) • 169/246 = 67% (excluding Japan 20%) Takashi.Sasaki@kek.jp

  35. Takashi.Sasaki@kek.jp

  36. Summary • Japan is an unique country in Asia-Pacific region which have large accelerators for HENP • should have a “tier-0” center • SRB is under testing for BELLE • ICEPP, U-Tokyo and KEK are collaborating together to build the ATLAS regional center in Japan • We seek a collaboration among Asia-Pacific countries • BELLE, LHC • More bandwidth is necessary among sites Takashi.Sasaki@kek.jp

More Related