CENIC, PACIFIC WAVE, NLR, IEEAF: Infrastructure for Research and Education

CENIC, PACIFIC WAVE, NLR, IEEAF: Infrastructure for Research and Education John Silvester Chair of the CENIC BoardVice-Provost for Scholarly Technology, University of Southern California LISHEP: Digital Divide and HEPGRID Workshop Rio de Janeiro, Brasil February 19th, 2004

CENIC History • CalREN and DCP • CalREN ONI • CalREN Today • National LambdaRail (aka Lightrail) • Pacific Wave • IEEAF • Pointers

Education in California – Overview • University of California – 9 (10) campuses • 3 Private Research Universities – Caltech, Stanford, University of Southern California • California State University – 23 campuses • Community Colleges – over 100 • Other independent institutions of higher education – over 100 • K-12 schools – over 9000 • Various government labs and university affiliated research institutes

State of Networking in 1996 • 4-CNET connected the CSU system with extension out to community colleges • Most institutions had their own commodity internet connections • UC operated some private leased lines • No statewide K-12 network • Some individual county and school district networks

Impetus to Establish R&E Network in California • Expansion of the NSF Connections program to facilitate access to the super-computer centers • Broadened to allow access to the high performance backbone (vBNS) for meritorious research • Acted as a stimulus to bring the Research Universities to create a joint proposal • Two initial goals: • Reduce costs of access to high performance national networks by a collaborative approach • Facilitate communication and collaboration between institutions by building a California REesearch Network

How to Proceed? • Decision that the network should be funded by the institutions rather than looking for a handout from the state or relying on Federal Government funding • Startup funds came from a joint proposal on behalf of the R-1’s to the NSF connections program (12 campuses). This allowed the institutions to ease into the burden of covering the network costs over a 3 year period • Founded a non-profit public benefit entity – CENIC – creating some separation from publicly funded entities (UC, CSU) • Lightweight organization (1.5 FTE for first 3 years)

Phase 1 - 1998

CENIC Initial 4-Year Funding Charter Associates $16 M 42% CENIC Partners $2 M 5% Corporate Discounts $14 M 37% Federal Funding $6 M 16%

Developments • Direct funding from UC for a central valley link to link CalREN-north (SF) and CalREN-south (LA-SD) • ISP – took advantage of the aggregated buying power to leverage ISP contracts with multiple vendors – significant savings (additional savings from Quilt pricing) • Peering – took advantage of peering opportunities at PAIX and MAE-LA • DCP – Digital California Project – extend connectivity to (public) K-12 schools

Digital California Project • Funded from California State in FY 2000/01 • $32M through University of California Implementation through CENIC • Program Steering Committee - Advisory board of involved constituents from K-20 • Subsequent budget reductions -- $26M -$21M - $14M – eliminated as direct funding for 2004-05

California R&E Networks in 2000 • CALREN-2 – advanced services network, owned and operated by CENIC (Corporation for Educational Network Initiatives in California) • 4CNet – owned and operated by Cal. State University • DCP – K-12 network- owned and operated by CENIC; being implemented • Los Nettos – dark fiber based CWDM Metro network owned and operated by USC and Caltech - last of the NSFnet regionals.

Phase 2 - 2000

Redesigning CalREN • In late 1999, with the approaching end of current SONET contracts (late 2002), CENIC began thinking about the next generation CalREN • User demand: • Reliable ‘commodity’ network • High bandwidth (IP) network in support of research (Abilene) • Some demand for dedicated resources • Significant demand for experimental and research networks at level 3, level 2 and even level 1 • This formed the thinking for an integrated infrastructure built on dark fiber

CALREN-DC Digital California • IP based network. 2.5-10 GB • Serves-140 H.E institutions; 8000+ elementary and high schools • 8.0 million+ student, faculty and staff users • I2 connectivity and commodity ISP services.

CALREN-HPRHigh Performance Research Network • IP network: 10Gb, potentially several wavelengths • 50+ Research institutions, National Laboratories and San Diego Super-computing Center in California • California component of Internet2 with 10G and OC-12 connections • Serves hundreds of researchers, demanding applications

CALREN-XDExperimental/Development Network • 10.0 Gb Wavelengths and Dark Fiber • Potential for Wavelength Switching and Special Network Configurations • Special applications, e.g. Teragrid • Serves Network Researchers in California Research Institutions – primarily four UC Institutes; USC’s ISI; Stanford; and Caltech

CALREN - today • 3 backbones – one commodity, one production, one research oriented, sharing physical resources where applicable • Integrated at the physical and operations level, separable at the link and network levels • Separate local solution from long-haul solution (due to different possibilities, players) • Combination of dark fiber and wavelengths

CalRENBackbone NetworkPhase 3 - 2003

UC Davis Sacramento Triangle Court Fergus Oakland Sunnyvale Soledad Fresno San Luis Obispo Bakersfield Los Angeles Santa Barbara Tustin San Diego CalREN Waves CalREN DC HPR Teragrid HPR & DC

Digital California Overlay

Campus or Metro Interconnect Calren/DC/HPR/XD POP Architecture CalRen DC HPR XD Long Haul OC48/OC192/10GigE DWDM DWDM 10 Gig E or OC192 Gig E • switch/mux CalREN/DC HPR XD 15500

Historical Review of CalREN 1996 - Initial meetings 1997 - NSF proposal for CalREN2 funded; CENIC incorporated 1998 - CalREN2 operational, connect to vBNS 1999 - Connect to Abilene; ISP service launched 2000 - DCP project launched 2001 - ONI project launched; First DCP nodes operational 2002 - ONI vendors selected; DCP 90% completed 2003 - CalREN-DCP (optical edition) operational 2004 - CalREN-HPR operational 2004 - Pacific Wave linkup 2004 - XD links operational as NLR comes up

Expanding the reach? • Multi-State interest in a larger geographic fiber based infrastructure • Fiber vendor very interested in selling National footprint

National Light Rail - concept • Dark Fiber National footprint • Serves very high-end experimental and research applications including network research • 4 - 10GB Wavelengths initially • Capable of 40 10Gb wavelengths at build-out • Partnership model (including corporate partners)

Many different perspectives • “NLR… … aims to reenergize innovative R&D into next generation networking technologies, protocols, services, and apps.” … is a virtual laboratory.” … will contribute to the (NSF) Cyberinfrastructure that is critical to progress in every field of science & engineering.” … will help fuel the growth of the TeraGrid and computational science in general.” … motivates our planned regional optical network.” … is a hedge timed to hit the trough of the Internet/telecom economy.” … is both an experimental facility and a complex, multi-dimensional experiment in and of itself.” ALL OF THE ABOVE!

Distinguishing features - I • NLR, Inc. has been established as a non-profit organization (May 2003, 501(c)(3) tax-exempt status being sought) • Partnership with industry (CISCO is a major partner/contributor with special interest in promoting network research) • Largest higher-ed owned/managed optical networking & research facility in the world • ~10,000 route-miles of dark fiber (Level 3) • Four 10-Gbps ’s provisioned at outset • First & foremost, an experimental platform for research • Optical, switching & network layers • Research committee (with 2 board seats) • Experimental support center • 50% of capacity available for network research

Distinguishing features - II • Sparse backbone’ topology • Use of high speed Ethernet for WAN transport • 1O Gigabit Ethernet LAN PHY is primary interface • Traditional OC-192 SONET available, too • Additional ’s (up to 40 on a given segment) can be provisioned with pricing tied to incremental cost • Unprecedented level of self-capitalization for national networking initiative by higher ed participants • ~$80M budget for full national backbone (CapEx + 5-yrs OpEx) • Each contribution (typically $5M) assumed to be ‘sunk cost’ • Each participant/node has concurrent responsibility for developing optical networking capabilities and sustaining performance in their region

CENIC Pacific Northwest Gigapop Pittsburgh SC Duke Univ./NCLR MATP/Va. Tech Cisco Systems Internet2 Florida LambdaRail Georgia Tech CIC Pending: Texas Consortium New York Area Consortium Mountain region consortium Current Members and Associates

NLR Project Segment Deployment Schedule (approx) • Chicago to Pittsburgh 11/03 • Sunnyvale to Seattle 4/04 • Pittsburgh to Washington DC 3/04 • Washington DC to Atlanta 5/04 • Atlanta to Jacksonville 8/04 • Seattle to Denver 7/04 • Denver to Chicago 8/04

Related Developments • O(10k) miles of dark fiber have been acquired by the community for national and regional optical networks by CENIC, FiberCo (via Level 3), and others. • ATT makes major donation of fiber to USA Waves • The Abilene Network has been upgraded to a 10-Gbps backbone and supports the research university community through initiatives such as IPv6 deployment and the Observatory. • Internet2 plans to use one NLR wave for development of HOPI (hybrid optical packet interconnect)

U.S. Pacific Coast Peering Collaboration • CENIC (Corporation for Education Network Initiatives in California) and PNWGP (Pacific Northwest Gigapop) have combined efforts to create an advanced, extended peering facility on the U.S. West Coast. • Concept: an extensible, geographically dispersed peering fabric • Result: you connect at any one location on the fabric and have the option to peer with any other participant, regardless of where they are connected

Current Model

Summer 2004 - LA & Seattle

Multi-Node Pacific Wave • The Pacific Wave International Peering exchange facility will offer connection points initially in Los Angeles and Seattle, proximal to submarine cable landing sites on the U.S. Pacific Coast. • Connection points to be connected by 10GE link derived from NLR • Expected operation by Summer 2004

Future?

New Public-Private Partnerships • Global telecomm build-out of technical infrastructure provides new possibilities for economic development • Current market conditions have resulted in capacity which is currently going unused -- cannot be sold. • As a matter of social responsibility, this unused capacity could be made available for stimulating future applications and markets -- by donation for use by research and education institutions.

Example: IEEAF • The IEEAF represents one such partnership whose goal is to obtain donations of international bandwidth to enable a global collaboration in research and education. • Current donations have already linked US and Europe, and are linking US and Asia-Pacific. • This bandwidth helps enable global collaborations in research and education, in the true spirit of the Global Quilt Initiative.

IEEAF Vision: The Global Quilt • A Network of Networks, “stitched together” to create a common single fabric, and shared equally by all. This will be achieved through collaboration and community effort, until it covers the globe. • The IEEAF has no boundaries of “home” territory…..

IEEAF - What is it? • U.S. 501.c.3 Not-for-profit corporation • Formed from original MOU between GEO and CENIC (Corporation for Educational Networking in California) • Vision: Accelerate the global growth of advanced networking (Internet2) to achieve "universal educational access” to: • Enable and stimulate the rapid expansion of research and educational collaboration in many forms between teaching and learning institutions around the world. • Cultivate and promote practical solutions to delivering scalable, universally available and equitable access to suitable bandwidth and necessary network resources in support of these collaborations.

IEEAF - How does it work? • Leverage global deregulation and new entrants into telco business • Leverage private sector business relationships • Geographic Network Affiliates, Inc. (GEO) • Build donations into business deals (contracts) such as no-cost IRUs

IEEAF - What does it do? • Gets donated communications related assets • Makes them available to existing institutions and networking organizations to put to work • Vehicle: Asset Steward Agreement

CENIC, PACIFIC WAVE, NLR, IEEAF: Infrastructure for Research and Education