Enabling New Applications with Optical Circuit-Switched Networks Xuan Zheng April 27, 2004

1. Enabling New Applications with Optical Circuit-Switched Networks Xuan Zheng April 27, 2004

2. Outline • Background and problem statement • Proposed RESCUE service • Application I: High-speed optical Dial-Up Internet access service using RESCUE circuits • Application II: end-to-end RESCUE circuits to improve file transfer delays • Implementation of application II • Summary

3. Background • Current optical network architectures • Current optical network applications • Leased access circuits for enterprise users • High-speed inter-switch/inter-router circuits

4. Gaps between User Needs and Current Network Solutions • Access link bottleneck problem • Date rates of access links are still slow. • Access links are often heavily utilized. • TCP limitations • TCP is not suited for High-Delay-Bandwidth-Product (HDBP) networks because of its congestion control scheme. • Hard to create end-to-end connections to provide QoS for interactive real-time applications • Current Internet is connectionless.

5. Prior work • In packet-switched networks • Packet-switched ring (RPR) is proposed for access links • Increasing the circuit rate does not help a lot if the packet loss rate remains high. • TCP enhancements are proposed to achieve high end-to-end TCP throughputs • HighSpeed TCP, Scalable TCP, FAST TCP, etc. • Did not touch the shared nature of Internet; no end-to-end QoS guarantee. • QoS in IP based networks • IntServ, DiffServ, TCP switching, etc. • Implemented at IP routers instead of end hosts. • Not scalable, especially when traffic is large.

6. Prior work • In circuit-switched networks • Traditionally, bandwidth-on-demand is primarily focused on inter-switch/inter-router circuits in service provider networks. • Fast restoration and rapid provisioning • Centralized resource management with human interventions • Latest efforts on bandwidth-on-demand • UCLP in Canarie network, ESnet, etc. • Provide user-controlled end-to-end optical circuit provisioning • Still centralized approach • Applications are limited to the elephant data transfer and other eScience applications in a small community • Too costly • Does not scale for commodity service

7. Problem Statement • Design new network architectures exploiting advances in optical switching technologies to bridge the gaps between user needs and network limitations. • High-speed circuit switches • Dynamic distributed control with signaling/routing protocols

8. Other Enterprises Proposed Architecture: Reconfigurable Ethernet/SONET Circuits for End Users (RESCUE) Software upgrade Second NIC SETUP SUCCESS SUCCESS Second leased line SETUP

9. RESCUE: An “Add-on” Service to Primary Internet Access • Two paths between two entities: the primary TCP/IP path and an Ethernet/SONET circuit. Packet-switched Internet End host I End host II Optical Circuit-switched Network • “Parallel-hybrid” architecture vs. traditional “sequential-hybrid” architecture

10. RESCUE: Applications • High-speed optical Dial-Up Internet access service • End-to-end file transfers • Gap #1 • Gap #2

11. ARP table Routing table Map MAC addresses Map IP address to Enterprise building to newly setup newly setup RESCUE circuit RESCUE circuit User space Application + Ethernet RESCUE software hosts Internet service OS provider Dial-Up server NIC 2 NIC 1 (signaling + Optical circuit-switched configuration access network Ethernet software) Ethernet switch/IP switch/IP router router Primary Internet leased access circuit SONET From other end hosts MSPP RESCUE circuit SONET for Dial-Up service MSPP Ethernet Interface Application I: Dial-Up Internet Access Service using RESCUE Circuits

12. Application II: End-to-end RESCUE Circuits to Improve File Transfer Delays • Use new transport protocols other than TCP on end-to-end RESCUE circuits

13. Application II: Analytical Basis for the Routing Decision - Delay Analysis

14. Application II: Analytical Basis for the Routing Decision - Delay Analysis

15. Application II: Analytical Basis for the Routing Decision - Delay Analysis

16. Application II: Analytical Basis for the Routing Decision -Delay Analysis Tprop= 0.1ms Tprop= 50ms

17. Pb=0.3 + Ploss=0.01 Crossover file size=180KB Application II: Analytical Basis for the Routing Decision - Delay Analysis For example:

18. Application II: Analytical Basis for the Routing Decision - Utilization Analysis Symmetric three-link network model

19. 93% 84% Application II: Analytical Basis for the Routing Decision - Utilization Analysis Access link utilization uaccess Core link utilization ucore

20. Analytical Basis for the Routing Decision • In low propagation-delay environments • Delay-based decision • Crossover file size depends upon the link rates and the loading conditions on the two paths • In high propagation-delay environments • Utilization-based decision • A lower bound is needed for crossover file size

21. Implementation of Application II • End-host RESCUE software • A high-speed transport protocol module for end-to-end file-transfer applications, • A routing decision module, • A signaling module.

22. High-speed Transport Protocol: Design Rationale • Flow control: rate-based scheme to achieve high circuit utilization. • Implementation is not trivial. • Error control: selective-Automatic-Repeat-reQuest (selective-ARQ) scheme to achieve a high efficiency. • Negative Acknowledgements (NAK) because of the guaranteed in-sequence delivery of data blocks on dedicated circuits. • Positive Acknowledgements (ACK) are still needed to update sender’s retransmission buffers. • Dual communication paths • Use primary TCP/IP path to transport reverse-path control messages. • Our transport solution: Fixed Rate Transport Protocol (FRTP).

23. High-speed Transport Protocol: FRTP Specification • The model of FRTP connections

24. High-speed Transport Protocol: An Implementation of FRTP protocol • FRTP is implemented as an application-level process using a combination of UDP and TCP.

25. High-speed Transport Protocol: An Implementation of FRTP protocol • Experimental environment: • Connections: Two Dell Precision 650 workstations connected via a Dell PowerConnect Gigabit Ethernet switch. • Hardware configurations: • A 2.4-GHz Intel CPU connected to a 533-MHz front-side bus (34Gbps CPU bandwidth), • An E7505 chipset with 512MB of DDR 266MHz memory (17Gbps memory bandwidth), • An 80GB ATA/100 7200 RPM EIDE disk drive with 2MB cache (400Mbps average access rate measured by Bonnie [66]), and, • A 64bit/100MHz PCIx bus for the GbE NIC (6.4Gbps network bandwidth). • The operating systems: RedHat Linux 9 with version 2.4.20-30.9 kernel.

26. High-speed Transport Protocol: An Implementation of FRTP protocol • Experimental results with default settings • 256KB UDP buffer size, 1500Bytes DATA packet size, 40MB FRTP buffer size, and 8MB block size for disk I/O operations. FRTP throughput FRTP packet-loss rate

27. High-speed Transport Protocol: An Implementation of FRTP protocol • Impact of UDP buffer size • 500Mbps sending rate, 1500Bytes DATA packet size, 40MB FRTP buffer size, and 8MB block size for disk I/O operations. FRTP throughput FRTP packet-loss rate

28. High-speed Transport Protocol: An Implementation of FRTP protocol • Impact of FRTP DATA packet size • 500Mbps sending rate, 256K UDP buffer size, 40MB FRTP buffer size, and 8MB block size for disk I/O operations. FRTP throughput FRTP packet-loss rate

29. Routing Decision Module Design

30. Signaling Module Design • A RSVP-TE implementation

31. Contributions • New network architecture • “Parallel-hybrid” instead of traditional “sequential-hybrid” • Dedicated end-to-end high-speed connectivity between end hosts • Distributed, dynamic end-to-end circuit provisioning instead of centralized resource management. • Objective: a large-scale network providing commodity services • High aggregate network utilization • Commodity services: the elephant data transfer as well as small data transfer • High traffic load -> high utilization -> low cost • Call blocking mode with packet-switched back-up paths. • High circuit utilization • Superfast provisioning: distributed + hardware signaling • High-speed rate-based flow control • Leveraging current conditions of Ethernet and SONET • Circuit-switched SONET are widely deployed in wide-area networks. • Ethernet dominates local-area networks.

32. Publications from this work • Journal papers: • M. Veeraraghavan and X. Zheng, “A Reconfigurable Ethernet/SONET Circuit Based Metro Network Architecture,” IEEE JSAC on Advances in Metropolitan Optical Networks (Architectures and Control), 2004. • M. Veeraraghavan, X. Zheng, W. Feng, Hojun Lee, E. Chong, and H. Li, “Scheduling and transport for file transfers on high-speed optical circuits,” JOGC on High Performance Networking, 2004. • Conference papers: • X. Zheng, M. Veeraraghavan, and H. Lee, “Using Dial-Up Optical Circuits to Address the Access Link Bottleneck Problem,” Under revision based on reviews from Infocom 2004. • Best Student Paper Award, M. Veeraraghavan, X. Zheng, H. Lee, M. Gardner, and W. Feng, “CHEETAH: Circuit-switched High-speed End-to-End Transport ArcHitecture,” Proceeding of Opticomm 2003, Dallas, TX, Oct. 13-16, 2003. • T. Moors, M. Veeraraghavan, Z. Tao, X. Zheng, R. Badri, Experiences in automating the testing of SS7 Signaling Transfer Points, International Symposium on Software Testing and Analysis (ISSTA), July 22-24, 2002, Via di Ripetta, Rome - Italy. • Magazine paper: • M. Veeraraghavan, D. Logothetis, and X. Zheng, “Using dynamic optical networking for high-speed access,” Optical Networks Magazine, special issue on “Dynamic Optical Networking around the Corner or Light Years Away?”, vol. 4, no. 5, pp. 30-40, Sep. 2003. • Workshop papers: • M. Veeraraghavan, H. Lee, and X. Zheng, “File transfers across optical circuit-switched networks,” PFLDnet 2003, Geneva, Switzerland, Feb. 3-4, 2003.

33. Questions? Thanks! 