1 / 48

Networks and Grids for HENP as Global e-Science

Networks and Grids for HENP as Global e-Science. Harvey B. Newman California Institute of Technology CHEP2004, Interlaken September 30, 2004. ICFA and Global Networks for Collaborative Science. Given the wordwide spread and data-intensive challenges in our field

gala
Download Presentation

Networks and Grids for HENP as Global e-Science

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Networks and Grids for HENP as Global e-Science Harvey B. Newman California Institute of TechnologyCHEP2004, InterlakenSeptember 30, 2004

  2. ICFA and Global Networks for Collaborative Science • Given the wordwide spread and data-intensive challenges in our field • National and International Networks, with sufficient (rapidly increasing) capacity and seamless end-to-end capability, are essential for • The daily conduct of collaborative work in both experiment and theory • Experiment development & construction on a global scale • Grid systems supporting analysis involving physicists in all world regions • The conception, design and implementation of next generation facilities as “global networks” • “Collaborations on this scale would never have been attempted, if they could not rely on excellent networks”

  3. History of Bandwidth Usage – One Large Network; One Large Research Site ESnet Accepted Traffic 1990 – 2004Exponential Growth Since ’92;Annual Rate Increased from 1.7 to 2.0X Per Year In the Last 5 Years L. Cottrell W. Johnston Progressin Steps SLAC Traffic ~400 Mbps; Growth in Steps (ESNet Limit): ~ 10X/4 Years.Projected: ~2 Terabits/s by ~2014

  4. Int’l Networks BW on Major Links for HENP: US-CERN Example • Rate of Progress >> Moore’s Law (US-CERN Example) • 9.6 kbps Analog (1985) • 64-256 kbps Digital (1989 - 1994) [X 7 – 27] • 1.5 Mbps Shared (1990-3; IBM) [X 160] • 2 -4 Mbps (1996-1998) [X 200-400] • 12-20 Mbps (1999-2000) [X 1.2k-2k] • 155-310 Mbps (2001-2) [X 16k – 32k] • 622 Mbps (2002-3) [X 65k] • 2.5 Gbps  (2003-4) [X 250k] • 10 Gbps  (2005) [X 1M] • 4x10 Gbps or 40 Gbps (2007-8) [X 4M] • A factor of ~1M Bandwidth Increase since 1985; ~4M by 2007-8 A factor of ~5k Since 1995; • HENP has become a leading applications driver, and also a co-developer of global networks

  5. Tier2 Center Tier2 Center Tier2 Center Tier2 Center Tier2 Center LHC Data Grid Hierarchy:Developed at Caltech CERN/Outside Resource Ratio ~1:2Tier0/( Tier1)/( Tier2) ~1:1:1 ~PByte/sec ~100-1500 MBytes/sec Online System Experiment CERN Center PBs of Disk; Tape Robot Tier 0 +1 Tier 1 10 - 40 Gbps FNAL Center IN2P3 Center INFN Center RAL Center ~10 Gbps Tier 2 ~1-10 Gbps Tier 3 Institute Institute Institute Institute Tens of Petabytes by 2007-8.An Exabyte ~5-7 Years later. Physics data cache 1 to 10 Gbps Tier 4 Workstations Emerging Vision: A Richly Structured, Global Dynamic System

  6. Challenges of Next Generation Science in the Information Age • Flagship Applications • High Energy & Nuclear Physics, AstroPhysics Sky Surveys: TByte to PByte “block” transfers at 1-10+ Gbps • Fusion Energy: Time Critical Burst-Data Distribution; Distributed Plasma Simulations, Visualization, Analysis • eVLBI: Many real time data streams at 1-10 Gbps • BioInformatics, Clinical Imaging: GByte images on demand • Advanced integrated Grid applications rely on reliable, high performance operation of our LANs and WANs • Analysis Challenge: Provide results with rapid turnaround, over networks of varying capability in different world regions Petabytes of complex data explored and analyzed by 1000s of globally dispersed scientists, in hundreds of teams

  7. Internet 2 Land Speed Records (LSR):Redefining the Role and Limits of TCP • Judged on product of transfer speed and distance end-to-end, using standard (TCP/IP) protocols, Across Production Net: e.g. Abilene • IPv6: 4.0 Gbps Geneva-Phoenix (SC2003) • IPv4 with Windows & Linux: 6.6 Gbps Caltech-CERN (15.7 kkm; “Grand Tour of Abilene”) June 2004 • Exceeded 100 Petabit-m/sec • 7.48 Gbps X 16 kkm (Linux, 1 Stream) Achieved in July • 11 Gbps (802.3ad) Over LAN in Sept. • Concentrate now on reliable Terabyte-scale file transfers • Note System Issues: CPU, PCI-XBus, NIC, I/O Controllers, Drivers LSR History – IPv4 single stream Petabitmeter (10^15 bit*meter) • Monitoring of the Abilene traffic in LA: June 2004 Record Network SC04: 100 Gbps Challenge See Talk by S. Ravot

  8. HENP Bandwidth Roadmap for Major Links (in Gbps) Continuing Trend: ~1000 Times Bandwidth Growth Per Decade;Compatible with Other Major Plans (ESNet, NLR; GN2, GLIF)

  9. Evolving Quantitative Science Requirements for Networks (DOE High Perf. Network Workshop) See http://www.doecollaboratory.org/meetings/hpnpw/

  10. National Lambda Rail (NLR): www.nlr.net Transition beginning now to optical, multi-wavelength Community owned or leased “dark fiber” (10 GbE) networks for R&E • NLR • Coming Up Now • Initially 4 10G Wavelengths • Northern Route LA-JAX by 4Q04 • Internet2 HOPI Initiative (w/HEP) • To 40 10G Waves in Future • Initiatives in: nl, ca, pl, cz, uk, ko, jp • + 18 US States (CA, IL, FL, IN, …)

  11. ESnet Beyond FY07 (W. Johnston) AsiaPac SEA CERN Europe Europe Japan Japan CHI SNV NYC DEN DC Japan ALB ATL SDG MANs Qwest – ESnet hubs ELP NLR – ESnet hubs High-speed cross connects with Internet2/Abilene Major DOE Office of Science Sites Production IP ESnet core 10Gb/s 30Bg/s40Gb/s High-impact science core 2.5 Gbs10 Gbs Lab supplied Future phases Major international

  12. GLIF: Global Lambda Integrated Facility: www.glif.is “GLIF is a World Scale Lambda based Lab for Application and Middleware development, where Grid applications ride on dynamically configured networks based on optical wavelengths ... Coexisting with more traditional packet-switched network traffic 4th GLIF Workshop:Nottingham UK Sept. 2004 Also JGN2 (Japan) and KREONet (Korea) 10 Gbps Wavelengths For R&E Network Development Are Prolifering, Across Continents and Oceans

  13. CalTech/Newman FL/Avery SLAC Optiputer Ed Seidel HOPI UW/Rsrch Chnl NLR/SC04 Waves (v11) Starlight NLR-STAR-SEAT-10GE-6 SEA NLR-PITT-CHIC-10GE-20 NLR-PITT-LOSA-10GE-14 NLR-PITT-STAR-10GE-13 NLR-PITT-LOSA-10GE-15 NLR-PITT-STAR-10GE-16 NLR-PITT-SUNN-10GE-19 NLR-PITT-LOSA-10GE-14 NLR-PITT-SEAT-10GE-18 NLR-PITT-LOSA-10GE-15 NLR-PITT-SUNN-10GE-19 CHI NLR-PITT-SEAT-10GE-18 NLR-PITT-LOSA-10GE-14 NLR-PITT-LOSA-10GE-15 NLR-SEAT-SAND-10GE-7 SC04 PSC NLR-PITT-WASH-10GE-21 SVL NLR-PITT-JACK-10GE-19 WDC CalTech LA SD JAX All lines 10GE

  14. UltraLight Collaboration:http://ultralight.caltech.edu • Caltech, UF, UMich, SLAC,FNAL, CERN, MIT, FIU, NLR, CENIC, UCAID, Translight, UKLight, Netherlight, UvA, UCLondon, KEK, Taiwan, KNU (Korea),UERJ (Rio), Sao Paolo • Cisco, Level(3) • Integrated hybrid experimental network, leveraging Transatlantic R&D network partnerships; packet-switched + dynamic optical paths • 10 GbE across US and the Atlantic: NLR, DataTAG, TransLight, NetherLight, UKLight, etc.; Extensions to Japan, Taiwan, Korea, Brazil • End-to-end monitoring; Realtime tracking and optimization; Dynamic bandwidth provisioning • Agent-based services spanning all layers of the system, from the optical cross-connects to the applications.

  15. ICFA SCIC (Since 1998) http://cern.ch/icfa-scic Three 2004 Reports; Presented to ICFA in February • Main Report: “Networking for HENP” [H. Newman et al.] • Includes Updates on Monitoring, the Digital Divide and Advanced Technologies [*] • A World Network Overview (with 27 Appendices): Status and Plans for the Next Few Years of National & Regional Networks, and Optical Network Initiatives • Monitoring Working Group Report [L. Cottrell] • Digital Divide in Russia [V. Ilyin]August 2004 Update Reports at the SCIC Web Site: See http://icfa-scic.web.cern.ch/ICFA-SCIC/documents.htm • Asia Pacific, Latin America, GLORIAD (US-Ru-Ko-China);Brazil, Korea, ESNet, etc.

  16. ICFA Report Update (8/2004): Main Trends Continue, Some Accelerate • Current generation of 2.5-10 Gbps network backbones and major Int’l links arrived in 2-3 Years [US+Europe+Japan; Now Korea and China] • Capability Grew 4 to 100s of Times; Much Faster than Moore’s Law • Proliferation of 10G links across the Atlantic Now • Direct result of Falling Network Prices: $ 0.5 – 1M Per Year for 10G • Ability to fully use long 10G paths with TCP continues to advance: 7.5 Gbps X 16kkm (August 2004) • Technological progress driving equipment costs in end-systems lower • “Commoditization” of Gbit Ethernet (GbE) ~complete: ($20-50 per port); 10 GbE commoditization underway: < $ 2K acad. • Move to owned or leased optical nets (us, ca, nl, sk, po, ko, jp) well underway in several areas of the world • Emergence of “Hybrid” Network Model: GNEW2004; UltraLight, GLIF • While there is progress in some less-advantaged regions, thegap between the technologically “rich” and “poor” is widening

  17. SCIC Main Conclusion for 2003-4 • The disparity among regions in HENP could increase even more sharply, as we learn to use advanced networks effectively, and we develop dynamic Grid systems in the “most favored” regions • We must therefore take action, and workto Close the Digital Divide • To make Physicists from All World Regions Full Partners in Their Experiments; and in the Process of Discovery • This is essential for the health of our global experimental collaborations, our plans for future projects, and our field. • Critical Path Items (for All Regions) • A coherent approach to End-to-end monitoring that allows physicists throughout the world to extract clear information • Upgrading campus infrastructures.To support Gbps data transfers in most HEP centers. • Removing local, last mile, and nat’l and int’l bottlenecks end-to-end, whether technical or political in origin.Bandwidths across borders, the countryside or the city is much less than the major backbones [This is true in many countries: From China to Brazil to NE US]

  18. ICFA SCIC Monitoring WG (L. Cottrell)See www.slac.stanford.edu/grp/scs/net/talk03/icfa-aug04.ppt Central Asia, Russia, SE Europe, L. America, Middle East, China: 4-5 yrs behind India, Africa: 7-8 yrs behind, and falling farther PingERWorld View from SLAC View from CERNConfirms This View Important for policy makers

  19. PROGRESS in SE Europe (Sk, Pl, Cz, Hu, …) Slovak Academic Network ( 2004) 1660 km of Dark Fiber CWDM Links, up to 112 km. 1 to 4 Gbps (GbE) August 2002:First NREN in Europe to establish Int’l GbE Dark Fiber Link, to AustriaApril 2003 to Czech Republic. Planning 10 Gbps Backbone; dark fiber link to Poland Slovakia VRVS Team

  20. The Advantage of Dark Fiber CESNET Case Study (Czech Republic) Leased 1 x 2,5G Leased fibre with own equipment 1 x 2,5G (EURO/Month) (EURO/Month) about 150km (e.g. Ústí n.L. - Liberec) 7,000 * 5 000 2513 km Leased Fibers (Since 1999) about 300km (e.g. Praha - Brno) 8,000 ** 7 000 * 2 x booster 18dBm ** 2 x booster 27dBm + 2 x preamp + 6 x DCF Leased 4 x 2,5G Leased fibre with own equipment 4 x 2,5G (EURO/Month) (EURO/Month) Case Study ResultWavelength ServiceVs. Fiber Lease: Cost Savings of 50-70% Over 4 Yearsfor Few Hundred Km 2.5G -10G Links about 150km (e.g. Ústí n.L. - Liberec) 14,000 * 8 000 about 300km (e.g. Praha - Brno) 23,000 ** 11 000 * 2 x booster 24dBm, DWDM 2,5G ** 2 x (booster+In-line+preamp), 6 x DCF, DWDM 2,5G Leased 1 x 10G Leased fibre with own equipment 1 x 10G (EURO/Month) (EURO/Month) about 150km (e.g. Ústí n.L. - Liberec) 14,000 * 5 000 about 300km (e.g. Praha - Brno) 16,000 ** 8 000 * 2 x booster 21dBm, 2 x DCF ** 2 x (booster 21dBm + in-line + preamplifier) + 6 x DCF Leased 4 x 10G Leased fibre with own equipment 4 x 10G (EURO/Month) (EURO/Month) about 150km (e.g. Ústí n.L. - Liberec) 29,000 * 12 000 about 300km (e.g. Praha - Brno) 47,000 ** 14 000 * 2 x booster 24dBm, 2 x DCF, DWDM 10G ** 2 x (booster +In-line+preampr), 6 x DCF, DWDM 10G

  21. Asia Pacific Academic Network Connectivity APAN Status July 2004 RU 200M 20.9G JP Europe 34M 2G KR 1.2G US 155M 310M Connectivity to US from JP, KO, AU is Advancing Rapidly (> 30G)Progress in the Region, and to Europe is Much Slower 9.1G 622M CN • TW 777M 45M `722M 90M HK 2M 7.5M • IN 45M 155M 1.5M TH PH VN 155M 1.5M 932M (to 21 G) MY • LK 2M 12M SG ID 2.5M Access Point Exchange Point Current Status 2004 (plan) 16M AU Moves to Better North/South Linkages within Asia JP-SG link: 155Mbps in 2005 is proposed to NSF by CIREN JP- TH link: 2Mbps  45Mbps in 2004 is being studied. Concept of an AP Ring with GLORIAD + IEEAF

  22. AmPath Research Networking in Latin America: Just Taking Off in 2004 • AmPathProvided connectivity for some Latin American countries • Argentina, Brazil, Chile, Mexico, Venezuela • New CHEPREO Sao Paolo-Miami Link at 622 Mbps Starting This Month New: CLARA (Funded by EU) • Regional Network Connecting 19 Countries: Argentina Dominican Republic Panama Brasil Ecuador ParaguayBolivia El Salvador PeruChile Guatemala UruguayColombia Honduras VenezuelaCosta Rica Mexico NicaraguaCuba Also WHREN NSF Proposal: 2.5G to US 155 Mbps Backbone with 10-45 Mbps Spurs;4 Mbps Satellite to Cuba; 622 Mbps to Europe Brazilian HEPGrid:Rio, Sao Paolo etc.

  23. OC3 circuits Moscow-Chicago-Beijing since January 2004 • OC3 circuit Moscow-Beijing July 2004 (completes the ring) • Rapid traffic growth with heaviest US use from DOE (FermiLab), NASA, NOAA, NIH and 260+ Univ. (UMD, IU, UCB, UNC, UMN… Many Others) • Plans for Central Asian extension, with Kyrgyz Gov’t Global Ring Network for Advanced Applications Development www.gloriad.org: US-RUSSIA-CHINA + KOREA Global Optical Ring • Aug. 8 2004: P.K. Young, Korean IST Advisor to President Announces • Korea Joining GLORIAD as a full partner • TEIN gradually to 10G, connected to GLORIAD > 5TBytes now transferred monthly via GLORIAD to US, Russia, China GLORIAD 5-year Proposal (with US NSF) for expansion to 2.5G-10G Moscow-Amsterdam-Chicago-Pacific-Hong Kong-Pusan-Beijing early 2005; 10G ring around northern hemisphere 2007; Multi-wavelength hybrid service from ~2008-9

  24. Wireline Dial Up ISDN Broadband 23.4M 45.0M 4.9M 9.8M Internet in China (J.P.Wu APAN July 2004) • Internet users in China: from 6.8 Million to 78 Million within 6 months • Backbone：2.5-10G DWDM+Router • International links：20G • Exchange Points：> 30G（BJ，SH，GZ） • Last Miles • Ethernet，WLAN，ADSL，CTV，CDMA，ISDN，GPRS，Dial-up • IP Addresses: 32M（1A+233B+146C); Need IPv6

  25. AFRICA: NectarNet Initiativewww.nectarnet.org W. MatthewsGeorgia Tech Growing Need to connect academic researchers, medical researchers & practitioners to many sites in Africa Examples: • CDC & NIH: Global AIDS Project, Dept. of Parasitic Diseases, Nat’l Library of Medicine (Ghana, Nigeria) • Gates $ 50M HIV/AIDS Center in Botswana • Monsoon Project, Dakar [cf. East US Hurricanes tp Africa] • US Geological Survey: Global Spatial Data Infrastructure • Distance Learning: Emory Univ.-Ibadan (Nigeria); Research Channel But Africa is Hard: 11M Sq. Miles, 600 M People, 54 Countries • Little Telecommunications Infrastructure Approach:Use SAT-3/WASC Cable (to S. Africa to Portugal); GEANT Across Europe, AMS-NY Link Across Atlantic, Peer with Abilene in NYC • Cable Landings in 8 West African Countries and South Africa • Pragmatic approach to reach end points: VSAT,ADSL,microwave, etc. Note: World Conference on Physics and Sustainable Development,10/31 – 11/2/05 in Durban South Africa; Part of World Year of Physics 2005. Sponsors: UNESCO, ICTP, IUPAP, APS, SAIP

  26. HEP Active in the World Summit on the Information Society 2003-2005 • WSIS I (Geneva 12/03): SIS Forum, CERN/Caltech Online Stand • CERN RSIS Event • Visitors at WSIS I • Kofi Annan, UN Sec’y General • John H. Marburger, Science Adviser to US President • Ion Iliescu, President of Romania, … • Planning Now Underway for WSIS II: Tunis 2005 • GOAL: To Create an “Information Society”. Common Definition Adopted (Tokyo Declaration, January 2003):“… One in which highly developed ICT networks, equitable and ubiquitous access to information, appropriate content in accessible formats, and effective communication can help people achieve their potential” • Kofi Annan Challenged the Scientific Community to Help (3/03)

  27. HEPGRID and Digital Divide Workshop UERJ, Rio de Janeiro, Feb. 16-20 2004 • Theme: Global Collaborations, Grids and Their Relationship to the Digital Divide • For the past three years the SCIC has focused on understanding and seeking the means of reducing or eliminating the Digital Divide. It proposed to ICFA that these issues, as they affect our field of High Energy Physics, be brought to our community for discussion. This led to ICFA’s approval, in July 2003, of the First Digital Divide and HEP Grid Workshop • Review of R&E Networks; Major Grid Projects • Perspectives on Digital Divide Issues by Major HEP Experiments, Regional Representatives • Focus on Digital Divide Issues in Latin America; Relate to Problems in Other Regions • More Info: http://www.lishep.uerj.br NEWS:Bulletin: ONE TWOWELCOME BULLETIN General InformationRegistrationTravel InformationHotel Registration Participant List How to Get UERJ/HotelComputer Accounts Useful Phone Numbers Program Contact us: SecretariatChairmen • Tutorials • C++ • Grid Technologies • Grid-Enabled Analysis • Networks • Collaborative Systems Sessions & Tutorials Available(w/Video) on the Web SPONSORS CLAF  CNPQ FAPERJ        UERJ

  28. International ICFA Workshop on HEP Networking, Grids and Digital Divide Issues for Global e-Science Dates: May 23-27, 2005 Venue: Daegu, Korea Dongchul Son Center for High Energy Physics Kyungpook National University ICFA, Beijing, China Aug. 2004 Approved by ICFA August 20, 2004

  29. International ICFA Workshop on HEP Networking, Grids and Digital Divide Issues for Global e-Science • Workshop Goals • Review the current status, progress and barriers to effective use of major national, continental and transoceanic networks used by HEP • Review progress, strengthen opportunities for collaboration, and explore the means to deal with key issues in Grid computing and Grid-enabled data analysis, for high energy physics and other fields of data intensive science, now and in the future • Exchange information and ideas, and formulate plans to develop solutions to specific problems related to the Digital Divide in various regions, with a focus on Asia Pacific, as well as Latin America, Russia and Africa • Continue to advance a broad program of work on reducing or eliminating the Digital Divide, and ensuring global collaboration, as related to all of the above aspects.

  30. Networks and Grids for HENP and Global Science • Network backbones and major links used by HENP and other fields are advancing rapidly • To the 10 G range in < 3 years; much faster than Moore’s Law • New HENP and DOE Roadmaps: a factor ~1000 BW Growth per decade • We are learning to use long distance 10 Gbps networks effectively • 2004 Developments: to 7.5 Gbps flows with TCP over 16 kkm • Transition to community-operated optical R&E networks (us, ca, nl, pl, cz, sk, kr, jp …); Emergence of a new generation of “hybrid” optical networks • We Must Work to Close to Digital Divide • To Allow Scientists in All World Regions to Take Part in Discoveries • Removing Regional, Last Mile, Local Bottlenecks and Compromises in Network Quality are now On the Critical Path • Important Examples on the Road to Progress in Closing the Digital Divide • CLARA, CHEPREO, and the Brazil HEPGrid in Latin America • Optical Networking in Central and Southeast Europe • APAN Links in the Asia Pacific: GLORIAD and TEIN • Leadership and Outreach: HEP Groups in Europe, US, Japan, & Korea

  31. Extra Slides Follow

  32. Internet Growth in the World At Large Amsterdam Internet Exchange Point Example 5 MinuteMax 30 Gbps 20 Gbps Average 11.08.04 Some Annual Growth Spurts;Typically In Summer-Fall The Rate of HENP Network Usage Growth (~100% Per Year) is Similar to the World at Large

  33. HENP Lambda Grids:Fibers for Physics • Problem: Extract “Small” Data Subsets of 1 to 100 Terabytes from 1 to 1000 Petabyte Data Stores • Survivability of the HENP Global Grid System, with hundreds of such transactions per day (circa 2007)requires that each transaction be completed in a relatively short time. • Example: Take 800 secs to complete the transaction. Then Transaction Size (TB)Net Throughput (Gbps) 1 10 10 100 100 1000 (Capacity of Fiber Today) • Summary: Providing Switching of 10 Gbps wavelengthswithin ~2-4 years; and Terabit Switching within 5-8 years would enable “Petascale Grids with Terabyte transactions”,to fully realize the discovery potential of major HENP programs, as well as other data-intensive research.

  34. [Legends ] <１０G> 　・Ishikawa Hi-tech Exchange Center (Tatsunokuchi-machi, Ishikawa Prefecture) <１００M> 　・Toyama Institute of Information Systems (Toyama) 　・Fukui Prefecture Data Super Highway AP * (Fukui) 20Gbps 10Gbps 1Gbps Optical testbeds Access points <１００M> 　・Hokkaido Regional Network Association AP * (Sapporo) Core network nodes <１G> 　・Teleport Okayama (Okayama) 　・Hiroshima University (Higashi Hiroshima) <１００M> 　・Tottori University of Environmental Studies (Tottori) 　・Techno Ark Shimane (Matsue) 　・New Media Plaza Yamaguchi (Yamaguchi) <１G> 　・Tohoku University (Sendai) 　・NICT Iwate IT Open Laboratory (Takizawa-mura, Iwate Prefecture) <１００M> 　・Hachinohe Institute of Technology (Hachinohe, Aomori Prefecture) 　・Akita Regional IX *(Akita) 　・Keio University Tsuruoka Campus (Tsuruoka, Yamagata Prefecture) 　・Aizu University (Aizu Wakamatsu) <１０G> 　・Kyoto University (Kyoto) 　・Osaka University (Ibaraki) <１G> 　・NICT Kansai Advanced Research Center (Kobe) <１００M> 　・Lake Biwa Data Highway AP * (Ohtsu) 　・Nara Prefectural Institute of Industrial Technology (Nara) 　・Wakayama University (Wakayama) 　・Hyogo Prefecture Nishiharima Technopolis (Kamigori-cho, Hyogo Prefecture) Sapporo <１００M> 　・Niigata University (Niigata) 　・Matsumoto Information Creation Center (Matsumoto, Nagano Prefecture) <１０G> 　・Kyushu University (Fukuoka) <１００M> 　・NetCom Saga (Saga) 　・Nagasaki University (Nagasaki) 　・Kumamoto Prefectural Office (Kumamoto) 　・Toyonokuni Hyper Network AP *(Oita) 　・Miyazaki University (Miyazaki) 　・Kagoshima University (Kagoshima) <１０G> 　・Tokyo University (Bunkyo Ward, Tokyo) 　・NICT Kashima Space Research Center (Kashima, Ibaraki Prefecture) <1G> 　・Yokosuka Telecom Research Park (Yokosuka, Kanagawa Prefecture) <100M> 　・Utsunomiya University (Utsunomiya) 　・Gunma Industrial Technology Center (Maebashi) 　・Reitaku University (Kashiwa, Chiba Prefecture) 　・NICT Honjo Information and Communications Open Laboratory (Honjo, Saitama Prefecture) 　・Yamanashi Prefecture Open R&D Center (Nakakoma-gun, Yamanashi Prefecture) Fukuoka Sendai Kanazawa NICT Kita Kyushu IT Open Laboratory Nagano NICT Koganei Headquarters Osaka Okayama Kochi NICT Tsukuba Research Center Nagoya NICT Keihannna Human Info-Communications Research Center Otemachi USA Okinawa <１００M> 　・Kagawa Prefecture Industry Promotion Center (Takamatsu) 　・Tokushima University (Tokushima) 　・Ehime University (Matsuyama) 　・Kochi University of Technology (Tosayamada-cho, Kochi Prefecture) <１００M> 　・Nagoya University (Nagoya) 　・University of Shizuoka (Shizuoka) 　・Softopia Japan (Ogaki, Gifu Prefecture) 　・Mie Prefectural College of Nursing (Tsu) *IX:Internet eXchange AP:Access Point JGN2: Japan Gigabit Network (4/04 – 3/08)20 Gbps Backbone, 6 Optical Cross-Connects

  35. ICFA Standing Committee on Interregional Connectivity (SCIC) • Created by ICFA in July 1998 in Vancouver • CHARGE: Make recommendations to ICFA concerning the connectivity between the Americas, Asia and Europe • As part of the process of developing theserecommendations, the committee should • Monitor traffic • Keep track of technology developments • Periodically review forecasts of future bandwidth needs • Provide early warning of potential problems • Representatives: Major labs, ECFA, ACFA; North American and Latin American Physics Communities • Monitoring, Advanced Technologies, and Digital DivideWorking Groups Formed in 2002

  36.         

  37. SCIC Focus on the Digital Divide: Several Perspectives • Work on Policies and/or Pricing: pk, in, br, SE Europe, … • Find Ways to work with vendors, NRENs, and/or Gov’ts • Point to Model Cases: e.g. Poland, Slovakia, Czech Republic • Share Pricing and Technology-Cost Information • Inter-Regional Projects • GLORIAD, Russia-China-US Optical Ring • Latin America: CHEPREO (US-Brazil); EU CLARA Project • Workshops and Tutorials/Training Sessions • For Example: Digital Divide and HEPGrid Workshop,UERJ Rio, Feb. 2004; Next DD Workshop in Daegu May 2005 • Help with Modernizing the Infrastructure • Raise Technology Awareness; Help Commission, Develop • Provide Tools for Effective Use: Monitoring, Collaboration • Participate in Standards Development; Open Tools • Advanced TCP stacks; Grid systems

  38. Bandwidth prices in Africa vary dramatically; are in general many times what they could be if universities purchase in volume Sample Bandwidth Costs for African Universities Avg. Unit Cost is 40X US Avg.; Cost is Several Hundred Times, Compared to Leading Countries Sample size of 26 universitiesAverage Cost for VSAT service: Quality, CIR, Rx, Tx not distinguished Roy Steiner Internet2 Workshop

  39. Managing Global Systems: Dynamic Scalable Services Architecture MonALISA: http://monalisa.cacr.caltech.edu 24 X 7 OperationsMultiple Orgs. • Grid2003 • US CMS • CMS-DC04 • ALICE • STAR • VRVS • ABILENE • GEANT • + GLORIAD • “Station Server” Services-engines at sites host many“Dynamic Services” • Scales to thousands of service-Instances • Servers autodiscover and interconnect dynamically to form a robust fabric • Autonomous agents + CLARENS: Web Services Fabric and Portal Architecture

  40. GEANT and CERNlink • GEANT plays a role in Europe similar to Abilene and ESnet in the US – it interconnects the European National Research and Education networks, to which the European R&E sites connect • GEANT currently carries essentially all ESnet international traffic (LHC use of CERNlink to DOE labs is still ramping up) • GN2 is the second phase of the GEANT project • The architecture of GN2 is remarkably similar to the new ESnet Science Data Network + IP core network model • CERNlink will be the main CERN to US, LHC data path • Both US, LHC tier 1 centers are on ESnet (FNAL and BNL) • ESnet directly connects at 10 Gb/s to the CERNlink • The ESnet new architecture (Science Data Network) will accommodate the anticipated 40 Gb/s from LHC to US

  41. ESnet New Architecture Goal FY05Science Data Network Phase 1 and SF BA MAN AsiaPac SEA Europe • CERN (2X10Gb/s) Europe Japan Japan NewCore CHI SNV NYC DEN DC Japan ALB SDG Existing ESnet Core ATL MANs current ESnet hubs ELP new ESnet hubs High-speed cross connects with Internet2/Abilene Major DOE Office of Science Sites Qwest ESnet core UltraSciNet NLR ESnet core 2.5 Gbs10 Gbs Lab supplied Future phases Major international

  42. ESnet New Architecture Goal FY06Science Data Network Phase 2 and Chicago MAN AsiaPac SEA Europe • CERN (3X10Gb/s) Europe Japan Japan CHI SNV NYC DEN DC Japan ALB SDG ATL MANs current ESnet hubs ELP new ESnet hubs High-speed cross connects with Internet2/Abilene Major DOE Office of Science Sites ESnet IP core (Qwest) UltraSciNet ESnet SDN core 2.5 Gbs10 Gbs Lab supplied Future phases Major international

  43. Abilene Map During LSR Trial CERN Monalisa

  44. TCP variants performance Tests between CERN and Caltech Capacity = OC-192 9.5Gbps; 264 ms round trip latency; 1 flow • Sending station: Tyan S2882 motherboard, 2x Opteron 2.4 GHz , 2 GB DDR. • Receiving station (CERN OpenLab):HP rx4640, 4x 1.5GHz Itanium-2, zx1 chipset, 8GB memory • Network adapter: S2io 10 GE 5.0 Gbps 7.3 Gbps 3.0 Gbps 4.1 Gbps Linux TCP Linux Westwood+Linux BIC TCP FAST

  45. High Throughput Disk to Disk Transfers: From 0.1 to 1GByte/sec Server Hardware (Rather than Network) Bottlenecks: • Write/read and transmit tasks share the same limited resources: CPU, PCI-X bus, memory, IO chipset • PCI-X bus bandwidth: 8.5 Gbps [133MHz x 64 bit] • Link aggregation (802.3ad): Logical interface with two physical interfaces on two independent PCI-X buses. • LAN test: 11.1 Gbps (memory to memory) Performance in this range (from 100 MByte/sec up to 1 GByte/sec) is required to build a responsive Grid-based Processing and Analysis System for LHC

  46. UltraLight Optical Exchange Point • L1, L2 and L3 services • Interfaces • 1GE and 10GE • 10GE WAN-PHY (SONET friendly) • Hybrid packet- and circuit-switched PoP • Interface between packet- and circuit-switched networks • Control plane is L3 Photonic switch

  47. SC2004: HEP network layout • Joint Caltech, FNAL, CERN, SLAC, UF…. • 11 10 Gbps waves to HEP’s show floor • Bandwidth challenge: aggregate throughput of 100 Gbps • FAST TCP

More Related