Internet2 Technology Update

1. Internet2 Technology Update Rick Summerhill Chief Technology Officer, Internet2 rrsum@internet2.edu Internet2 Fall Member Meeting 9 October 2007 San Diego, CA

2. Introduction • This session will provide an introduction and summary of many of the technology investigations and developments underway in the Internet2 community • Technology group on the Internet2 staff • Rick Summerhill, CTO • Eric Boyd, Deputy Technology Officer, concentrating on Network Architecture and Performance • Ken Klingenstein, Senior Director, concentrating on Middleware and Security • Matt Zekauskas, Senior Researcher, concentrating on Network Research • The session is not meant to include an exhaustive list of everything the community is examining, but rather describe the flavor of new technologies under investigation.

3. Collaborations • Almost all technology development is done through a variety of collaborations • Collaborations with members, including campuses, regional networks, and corporate members • Almost all of the development in middleware, for example, is done through these types of collaborations. • Collaborations with the international community or other national networks like ESnet • Much of the work on network performance or architecture includes these types of collaborations. • Collaborations with researchers in academia and corporate members • For example, network research • All of these collaborations are essential to technology development at Internet2

4. Agenda • Some Examples • Performance and Architecture • Network Research • Security and Middleware

5. Examples • Here are a few examples to illustrate how new technologies are undertaken • It is crucial that our community push the boundaries on new developments and investigate new ideas. • In these first few examples, consider • IPv6 • Hybrid Networking and the Dynamic Circuit Network

6. IPv6 • IPv6 has long been an area of emphasis for our community • IPv6 will likely become very important in the near future given recent ARIN discussions and announcements about the exhaustion of the IPv4 address space • The IPv6 initiative is essentially member driven in our community • There is an IPv6 working group that meets regularly at the Joint Techs meetings and there are hands-on workshops to support deployment • Many of our connectors and members have deployed IPv6 from a network centric point of view.

8. IPv6 Deployment • Although many connectors have deployed IPv6, it is difficult to gauge the deployment deep into the campus • IPv6 deployment as a network protocol is fundamentally not difficult • Getting campuses and connectors to support IPv6 on crucial applications, however, is often problematic • For example, mail servers, web servers, authentication servers - supporting (and porting, in some cases) critical applications to IPv6 lags • We encourage you to participate in the IPv6 working group to help set strategic direction for Internet2 in the future

9. Hybrid Networking • There has been tremendous interest from all communities associated with Internet2 to examine services that utilize lower layers of the protocol stack along with IP at layer 3 • This has become known as “hybrid networking” • It is motivated by applications from the research and education community that require greater capabilities • High bandwidth flows (for example, flows that come close to saturating links in the shared IP backbone) • Flows with special requirements related to quality of service, for example jitter requirements • On the Internet2 network, this takes the following form of an IP network together with the Dynamic Services Network:

11. Nodes

12. The Dynamic Circuit Network • A Network using protocols different from the normal IP protocols • A similar model as an IP network, but with different basic elements - dedicated circuits rather than shared data flows • Create Circuits (data paths) in seconds for periods of hours to days between hosts • Hosts might be individual hosts or routers on the IP network • Tremendous international collaboration on this project - GÉANT2, ESnet and Internet2 • Innovative work involving exchange of topology, path computation and scheduling and signaling using web services • Demonstration of how this works in the first plenary session

13. Technology Update:Architecture and Performance Eric Boyd eboyd@internet2.edu

14. CI Components Bulk Transport 2-Way Interactive Video Applications Real-Time Communications …. Applications call on Network Cyberinfrastructure Phoebus …. …. …. Network Cyberinfrastructure Performance Infrastructure / Tools Middleware Control Plane Measurement Nodes Network Control Plane Nodes

15. Internet2 DCN and HOPI I2 HOPI: Force10 E600 I2 DCS: Ciena CoreDirector 10 Gigabit Ethernet 10 Gigabit Ethernet 10 Gigabit Ethernet OC192 SONET/SDH 1 Gigabit Ethernet or SONET/SDH 1 Gigabit Ethernet

16. Internet2 DCN “Circuits” • Physical Connection: • 1 or 10 Gigabit Ethernet • OC192 SONET • Circuit Service: • Point to Point Ethernet (VLAN) Framed SONET Circuit • Point to Point SONET Circuit (future) • Bandwidth provisioning in 100 Mbps increments • How do Clients Request? • Client must specify [VLAN ID|ANY ID|Untagged], SRC Address, DST Address, Bandwidth • Request mechanism options are Web Service API, Web Page, phone call, email • What is the definition of a Client? • Anyone who connects to an ethernet or SONET port on an Ciena Core Director; could be RON, other wide area networks, domain specific applications

17. Internet2 DCN Circuit IntraDomain Circuit Request • Source Address • Destination Address • Bandwidth • VLAN TAG (None | Any | Number) • User Identification (certificate) • Schedule Dynamically Provisioned Dedicated Resource Path (“Circuit”) Internet2 IDC api api To IDC Client B XML Ethernet Mapped SONET or SONET Circuits Client A USER API Internet2 DCN Service • api can run on the client, or in a separate machine, or from a web browser Actual Network Path

18. 2 2 Internet2 DCN Circuit InterDomain • No difference from a client (user) perspective for InterDomain vs IntraDomain USER API A A 1 XML RON Dynamic Infrastructure Ethernet VLAN RON Dynamic Infrastructure Ethernet VLAN Internet2 DCS Ethernet Mapped SONET A. Abstracted topology exchange 1. Client Service Request 2. Resource Scheduling 5. Service Instantiation (as a result of Signaling)

19. Internet2 DCN Current Status DCN Infrastructure Deployed DCN Control Plane deployed and under test available for use for early adopters General DCN availability planned for January 2008 Instructions for those interested in using Internet2 DCN or in deploying their own dynamic network will be made available soon c

20. Phoebus Current Status • Developed at University of Delaware (Martin Swany) • Transport Middleware • Configuration per route/host/user • UDT for inter-depot communication • Transparent operation (library, iptables) • Simple file transfer tool (scp) • Transparently use Phoebus/Dynamic Circuits • Leverage Control Plane • Allocate dynamic circuits across Oscars (DCN, others) • Authentication and Authorization (currently primitive) • Future: Utilize Measurement Infrastructure • Help find best routes, provide information about paths and achievable bandwidth

21. Internet2 Active Measurement Tools • OWAMP (Latency) • v3.0c (RFC 4645 version) available now • Regular tests between all routers, and on-demand • BWCTL (Throughput) • v2.0 version under development • Regular tests between all routers and on-demand • NDT (User Diagnostic) • v3.4.1 available now • Latest version added better logging and error handling • NPToolkit (Active Measurement Tool Package) • v1.7 available now • Knoppix Live-CD bootable system

22. Internet2 Passive Measurement Tools • Circuit Status Service (E2EMON) • v1.0 • Internet2 implementation of European tool • Circuit Status service, Link Status service, Topology service • Netflow • Anonymized, available to researchers

23. Internet2 Measurement Framework • Why do we need an end-to-end measurement framework? • Most organizations can do monitoring and diagnostics of their own network • Networking is becoming an increasingly cross-domain effort • Monitoring and diagnostics must also become a cross-domain effort • What is perfSONAR? • A set of protocols and schemas for implementing a service-oriented architecture for sharing and controlling network performance tools • A community of users and developers (Internet2, ESnet, GEANT2, and RNP) • A set of software (the sample implementation)

24. Internet2 perfSONAR Current status • perfSONAR UI v0.9 available • Java release v2.1 available • perfSONAR-PS • Perl versions of perfSONAR services written by Internet2, ESnet, FNAL, SLAC, and UDel • Now Available: Micro-releases of Circuit Status Service, Link Status Service, Lookup Service, Topology Service, SNMP MA • Under Development: Micro-releases of perfSONOBUOY, and PingER • perfSONAR-PS bundle release planned for early ‘08

40. Technology Update:Network Research Matt Zekauskas matt@internet2.edu

41. Research Support in Internet2 Research on the network • Learning from measurements • Ability to test new theories, protocols and components Research using the network • All kinds, not just “network research” • Much tends to be “big science”, but it also spans a wide range including new methods of interaction and learning

42. Philosophy Internet2 does not do network research per se, but seeks to facilitate and support research projects led by faculty at member institutions • Make accessible network resources readily available to this community • Participate in research collaborations and provide support for proposals • Integrate research findings into the evolution of Internet2 network initiatives and services

43. Making Resources Available Primarily through Internet2 Observatory Two pieces • Measurements of Internet2 Network made available • Measurements for operations • Measurements specifically for research • Opportunity to collocate equipment where it makes sense to do so

44. Existing Measurement Capabilities One way latency, jitter, loss • IPv4 and IPv6 (“owamp”) Regular TCP throughput tests – ~1 Gbps • IPv4 and IPv6; On-demand available (“bwctl”) • ~10GE now also possible (Myricom and Dell 1950, must ask) SNMP • Octets, packets, errors; collected 1/min Flow data • Addresses anonymized by 0-ing the low order 11 bits Routing updates • Both IGP and BGP - Measurement device participates in both Router configuration • Visible Backbone – Collect 1/hr from all routers Dynamic updates • Syslog; also alarm generation (~nagios); polling via router proxy

45. Dataset Use Major consumption • Flows • Most popular (but also one that must be asked for) • Routes • Configuration • Nick Feamster (while at MIT) • Dave Maltz (while at CMU) Papers in SIGCOMM, INFOCOM Hard to track folks that just pull data off of web sites

46. Current Collocation VINI, a Planetlab followon • Will provide some sort of private network • Congruence with routed network useful 100x100: programmable network processors • Again, want private interconnect • More details in Research talk Phoebus • Break TCP sessions to allow hosts that are not tuned or on flawed networks to effectively use wide-area network • May also take advantage of circuits or non-TCP

47. Current Research Collaborations Ultralight (NSF) • Research support for upcoming LHC Physics data flows • Project led by Caltech 100x100 (NSF) • Focused on understanding the technical & economic requirements for providing 100-Mbps connectivity to 100 million U.S. homes • Project led by CMU, Stanford and Rice Hybrid Multi-layer Network (DoE) • Look at interoperability issues with new dynamic circuit networks. Data plane interoperability, control plane interoperability… • Project led by U New Mexico, USC ISI; includes ESnet and UltraScienceNet

48. Other, More Ad-Hoc, Collaborations Buffer sizing project (Stanford): • Reduce buffers available to router interfaces (software controlled) • Take an anonymized but correlated packet trace • Look for throughput and latency anomalies Rapid raw SNMP to test link capacity measurement programs Occasionally run programs on behalf of researchers on backbone machines

49. Small Grant Participation Network Measurement for International Connections • I’m PI, but work is done in close collaboration with Matt Mathis (who also has a small grant) and the International Research Network Connection PIs. • Research current state and propose solutions • Suggest common measurements • Identify areas for improvement • Work to establish a program-wide measurement group

50. Futures Work with Research Advisory Council to determine futures Restart some focus on outreach and dialog that was begun under a different small grant on the use of Internet2 facilities for research Provide the best possible data from our network, and facilitate other opportunities that come our way Come see the Network Research update late this afternoon for more details on current activity