1 / 15

UDTv4: Improvements on Performance and Scalability

UDTv4 is a protocol designed for fast, reliable, and secure data transfer. It features a three-layer design, improved resource management, and faster connection setup time. This version also includes better security features. Evaluation and future plans are discussed.

rgarvin
Download Presentation

UDTv4: Improvements on Performance and Scalability

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. UDTv4: Improvements on Performance and Scalability Yunhong Gu and Robert Grossman National Center for Data Mining Univ. of Illinois at Chicago

  2. Overview • Previous Versions • Use Case (Sector & Sphere) • Protocol Design & Implementation • Evaluation

  3. UDT: A Brief History • Started in 2001 (SABUL) • Bulk data transfer: control information over TCP, data over UDP • Naïve congestion control • UDTv2 • Both control and data over UDP • Fast, fair, and friendly congestion control • UDTv3 • Configurable congestion control • Partial reliability & messaging

  4. Use Case: Sector & Sphere • http://sector.sf.net • Distributed storage/file system • Distributed computing framework (data processing)

  5. Sector & Sphere Cloud Computing • http://www.opencloudconsortium.org • 120-node over wide area (Baltimore, Chicago, and San Diego) • 10Gb/s inter-site bw, 2Gb/s inter-node bw • Data access: user may upload and download data • Data exchange: data or computing result can be moved between any pair of nodes

  6. Sector & Sphere Cloud Computing • Large number of concurrent connections • Connections join and leave frequently and randomly • Data transfer happens in complicated patterns

  7. UDTv4: What’s New? • Three-layer design • UDP multiplexer • UDT flow • UDT connection • Support more concurrent UDT connections • Resource management • Sharing congestion control instance • Faster connection setup time • Better security

  8. UDTv4: Protocol Design

  9. UDTv4: Protocol Design

  10. Connection Record Index/Cache • Store historical and statistical information about each peer address • RTT, bw, loss rate, etc. • Garbage collection

  11. Implementation • Scalability and resource management • Queue management for UDT flows • Sending queue: O(logN), N is the number of flows • Receiving queue: O(1) • Buffer sharing and auto-sizing • Security • Rendezvous connection setup • Defend buffer overflow, SYN flooding, malicious control or data packets

  12. Evaluation: single connection

  13. Evaluation: concurrent connection

  14. Current Status and Future Plan • Current version 4.2 • Version 4.3 to be released before SC08 • Congestion control sharing not available in stable release yet • Internet Draft (version 2), will try IETF standardization after v5

  15. For More Information • Web: http://udt.sf.net Open source project (C++) • Email: Yunhong Gu gu@lac.uic.edu

More Related