Next Generation Abilene and Advanced National & Regional Infrastructure

1. Next Generation Abilene and Advanced National & Regional Infrastructure CENIC 2002 San Diego

2. Key topics • Abilene Network upgrade • Quilt Project • Optical networking progress

3. Abilene background & milestones • Abilene is a UCAID project in partnership with • Qwest Communications (SONET & DWDM service) • Cisco Systems (routers, switches & access) • Juniper Networks (routers) • Nortel Networks (SONET kit) • Indiana University (network operations) • ITECs in North Carolina and Ohio (test and evaluation) • Timeline • Apr 1998: Project announced at White House • Jan 1999: Production status for network • Oct 1999: IP version of HDTV (215 Mbps) over Abilene • Nov 2001: Raw HDTV/IP (1.5 Gbps) over Abilene

4. Abilene – May, 2002 • IP-over-SONET backbone (OC-48c, 2.5 Gbps) 53 direct connections • 4 OC-48c connections • 1 Gigabit Ethernet trial • 23 will connect via at least OC-12c (622 Mbps) by 1Q02 • Number of ATM connections decreasing • 215 participants – research universities & labs • All 50 states, District of Columbia, & Puerto Rico • 15 regional GigaPoPs support ~70% of participants • Expanded access • 50 sponsored participants • New: Smithsonian Institution, Arecibo Radio Telescope • 23 state education networks (SEGPs)

6. Abilene international connectivity • Transoceanic R&E bandwidths growing! • GÉANT – 5 Gbps between Europe and New York City now • Key international exchange points facilitated by Internet2 membership and the U.S. scientific community • STARTAP & STAR LIGHT – Chicago (GigE) • AMPATH – Miami (OC-3c  OC-12c) • Pacific Wave – Seattle (GigE) • MAN LAN - New York City (GigE/10GigE EP soon) • CA*NET3/4: Seattle, Chicago, and New York • CUDI: CENIC and Univ. of Texas at El Paso • International transit service • Collaboration with CA*NET3 and STARTAP

7. Packetized raw High Definition Television (HDTV) • Raw HDTV/IP – single UDP flow of 1.5 Gbps • Project of USC/ISIe, Tektronix, & U. of Wash (DARPA) • 6 Jan 2002: Seattle to Washington DC via Abilene • Single flow utilized 60% of backbone bandwidth • 18 hours: no packets lost, 15 resequencing episodes • End-to-end network performance (includes P/NW & MAX GigaPoPs) • Loss: <0.8 ppb (90% c.l.) • Reordering: 5 ppb • Transcontinental 1-Gbps TCP requires loss of • <30 ppb (1.5 KB frames) • <1 ppm (9KB jumbo)

8. End-to-End Performance:‘High bandwidth is not enough’ • Bulk TCP flows (transfers > 10 Mbytes) • Current median flow rate over Abilene: 1.9 Mbps • 95th percentile: 7.0 Mbps

9. Future of Abilene • Original UCAID/Qwest agreement amended on October 1, 2001 • Extension of for another 5 years – until October, 2006 • Originally expired March, 2003 • Upgrade of Abilene backbone to optical transport capability - ’s (unprotected) • x4 increase in the core backbone bandwidth • OC-48c SONET (2.5 Gbps) to 10-Gbps DWDM

10. Key aspects of next generation Abilene backbone - I • Native IPv6 • Motivations • Resolving IPv4 address exhaustion issues • Preservation of the original End-to-End Architecture model • p2p collaboration tools, reverse trend to CO-centrism • International collaboration • Router and host OS capabilities • Run natively - concurrent with IPv4 • Replicate multicast deployment strategy • Close collaboration with Internet2 IPv6 Working Group on regional and campus v6 rollout • Addressing architecture

11. Key aspects of next generation Abilene backbone - II • Addition of new measurement capabilities • Enhance active probing (Surveyor) • Latency & jitter, loss, TCP throughput • Add passive measurement • Support for computer science research – “Abilene Observatories” – archive/measurement/experiment • Support of Internet2 End-to-End Performance Initiative • Intermediate performance beacons • Network resiliency • Abilene ’s will not be protected like SONET • Increasing use of videoconferencing/VoIP impose tighter restoration requirements (<100 ms) • Options: • MPLS/TE fast reroute (initially) • IP-based IGP fast convergence (preferable)

13. Next generation router selection • Extensive router specification and test plan developed • Team effort: UCAID staff, NOC, NC and Ohio ITECs • Discussions with four router vendors • Tests focused on next gen advanced services • High performance TCP/IP throughput • High performance multicast • IPv6 functionality & throughput • Classification for QoS and measurement • 3 router platforms tested & commercial ISPs referenced •  New Juniper T640 platform selected

14. Two leading national initiatives in the U.S. • Next Generation Abilene • Advanced Internet backbone • connects entire campus networks of the research universities • 10 Gbps nationally • TeraGrid • Virtual machine room for distributed computing (Grid) • Connecting 4 HPC centers initiallly • Illinois: NCSA, Argonne • California: SDSC, Caltech • 4x10 Gbps: Chicago  Los Angeles • Ongoing collaboration between both projects

15. Quilt Project • Coalition of advanced regional networks • Now includes 18 GigaPoPs with SURA and EDUCAUSE • Including CENIC and Pacific/Northwest • Led by Wendy Huntoon (Pittsburgh) • Became a project of UCAID last summer • In parallel with Internet2 & Abilene projects • Core activities • Commercial Internet Services • Regional Optical Networking ( Next Gen Abilene) • Measurement ( End-to-end Performance) • New Internet2/Quilt K-20 Joint Initiative

16. Optical networking technology drivers • Aggressive period of fiber construction on the national & metro scales in U.S. • Many university campuses and regional GigaPoPs with dark fiber • Dense Wave Division Multiplexing (DWDM) • Allows the provisioning of multiple channels (’s) over distinct wavelengths on the same fiber pair • Fiber pair can carry 160 channels (1.6 Tbps!) • Optical transport is the current focus • Optical switching is still in the realm of experimental networks, but is nearing practical application

17. DWDM technology • DWDM fundamentally is an analog optical technology • Combines multiple channels (2-160+) over the same fiber pair • Employs discrete wavelengths (’s) of light • Generally supports 2.5 or 10 Gbps channels • Physical obstacles to long-distance transmission of light • Attenuation • Solved by amplification (OO) • Dispersion • Requires periodic signal regeneration (OEO)

18. Optical network project differentiation

19. Regional optical fanout • Next generation architecture: Regional & state based optical networking projects are critical • Three-level hierarchy: Backbones, GigaPoPs, campuses • Leading examples • CENIC ONI (California), I-WIRE (Illinois), I-LIGHT (Indiana) • State-based projects can be viewed as cost sharing • Collaboration with the Quilt • Regional Optical Networking project • Interface with next generation Abilene and future projects • U.S. carrier DWDM access is not widespread • 30-60 cities for DWDM now • ~120 cities for SONET (circa 1998) • Further expansion constrained by current telecom crunch

20. National optical networking options • 1 – Provision incremental wavelengths • Obtain 10-Gbps ’s as with SONET • Exploit smaller incremental cost of additional ’s • 1st  cost is ~10x than subsequent ’s • 2 – Build dim fiber facility • Partner with a facilities-based provider • Acquire 2 fiber pairs on a national scale • Outsource operation of transmission equipment • Needs lower-cost optical transmission equipment • Find ELH/ULH optical kit partner • The classic ‘buy vs. build’ decision in Information Technology • Option 1 selected for TeraGrid and Next Gen Abilene

21. Conclusions – Abilene future • Backbone upgrade project underway • Partnership with Qwest extended thru 2006 • Juniper T640 routers selected for backbone • 10-Gbps backbone  deployment starts this fall • Advanced service foci • Native, high-performance IPv6 • Enhanced, differentiated measurement • Network resiliency • Incremental, non-disruptive transition • Complementary to and collaborative with NSF’s TeraGrid

22. www.internet2.edu