Peer-Assisted Delivery : The Way To Scale Internet Video To The World

1. Peer-Assisted Delivery : The Way To Scale Internet Video To The World Jin Li, Principal Researcher (Collaborators: Cheng Huang, Keith Ross) Communication and Collaboration Systems Microsoft Research

2. Introduction: Internet Video on the Rise

3. Internet Video is on the Rise • Video streams served increased 38.8% in 2006 to 24.92 billion (Source: AccuStream iMedia Research) • 53 web-video startup in 2006 (US), $521M VC funding (Source: DowJones VentureOne) • Major studio goes into Web Video • $410M video ads revenue in 2006 and grow by 89% in 2007 (0.6% of $74B TV ad market, Internet ads $16.4B in 2006, expect to grow 19% in 2007) [source: Emarketer]

4. Internet Video is Growing: in Popularity & Quality video quality evolution popularity evolution

5. Internet Video Delivery: Data Center vs. CDN vs. P2P

6. Just Build More Powerful Data Center?

7. Data Center Capacity • VHS quality video streaming: 500 kbps (H.264) • 200,000 viewers = 100 Gbps • Data center capacity: Tera Grid (UIUC) • 30 petabyte of storage • 40 Gbps backbone: 80k video viewers

8. CDN Is Not The Answer • Akamai • 20,000 servers, 900 point of presence, 71 countries • 400Gbps bandwidth • Network optimized for latency • Limelight • 25 point of presence, hundreds servers per presence • 1,000Gbps bandwidth • Akamai+Limelight: 2.8 million viewers. • Current TV audience • Olympics: 2.5 billion viewers • Each viewer may have his/her own interest (different sport event, athlete nationality, etc.)

9. Peer Assisted Delivery is the Way To Scale bandwidth • Economical to run • Saves server/CDN bandwidth, disk I/O, CPU, memory • Robust • no single point of failure in network • Super-scalable • system capacity increases with number of nodes CPU memory peer resource hard drive

10. P2P Benefit Consumers: Better Video Quality, More Selection

11. Case Study 1: On Demand Internet Video

12. Peer Assisted Delivery: Mode Live Messenger FolderShare Groove File Sharing On Demand Streaming (Interactive TV) broadcast

13. Peer Buffer Map: File Sharing Peer 1 Peer 2 Peer 3 Peer 4 Peer 5 Peer 6

14. Peer Buffer Map: Broadcast Peer 1 Peer 2 Peer 3 Peer 4 Peer 5 Peer 6

15. Peer Buffer Map: On Demand Peer 1 Peer 2 Peer 3 Peer 4 Peer 5 Peer 6

16. MSN VoD Service • Traces from the on-demand service of MSN Video • 9-month period: Apr. – Dec. 2006 • 520M streaming requests • 59,000 unique videos

17. Peer-assisted VoD Model • Guaranteed QoS: always available server • Performance metric: server bandwidth • Peers upload what / when they are watching • conservative assumption servers in data centers/CDN

18. Peer Bandwidth • Download BW is measured by Windows Media Server • no accurate measurement beyond 3.5Mbps • Upload BW is inferred • Average upload: 500+ kbps Modem ISDN DSL1 Ethernet Cable DSL2

19. Bandwidth Allocation Policies • ask • ask • ask • ask server 3 2 • Assumptions • peers always start watching from beginning • VoD: earlier peers upload to later peers • 1st policy: no-prefetching • only satisfy demand for smooth playback, do not further build up the buffer • used by commercial live streaming companies to offer VoD 3 1 2 4 1 servers in data centers arrival

20. Bandwidth allocation policies (2) water-leveling: greedy: 4 • Prefetching – to utilize remaining upload capacity • 2nd policy: water-leveling • 3rd policy: greedy • Lower bound • allow later peers upload to earlier ones • no arrival order constraints 3 2 2 4 1 servers in data centers arrival

21. Observations on Policies(Simulated: Peer Poisson Arrival) • Prefetching is crucial • “free” to increase video bitrate • “balanced mode” is most difficult • S ≈ D • Greedy policy works best • lowest server load • very close to bound more available upload

22. Server BW Reduction – Two Videos • select top two most popular videos • ~800,000 views during April, 2006 • significant server bandwidth reduction using peer assistance • less server BW even increase quality 3 times (@3x bitrate) gold stream silver stream

23. Server BW reduction – two videos P2P @3x • select top two most popular videos • ~800,000 views during April, 2006 • significant server bandwidth reduction using peer assistance • less server BW even increase quality 3 times (@3x bitrate) gold stream silver stream

24. Server BW reduction – all videos • 12,000+ videos • server bandwidth reduction in all categories • 1.23Gbps  36.9Mbps (97%) • 1.23Gbps  770Mbps @3X bitrate (38%) April 2006

25. Locality Aware P2P Delivery

26. P2P Traffic Today • 1999 to present: fueled by Napster, KaZaA, eDonkey and BitTorrent 50-65% of downstream traffic is P2P, 75-90% of Upstream traffic is P2P. CacheLogic Research Internet Protocol Breakdown 1993 - 2006

27. Internet Traffic on the Rise • Internet traffic trend: grow at a compound monthly average of 7.4% in 2006 Internet traffic doubles per year Traffic at Amsterdam Internet Exchange (AMS-IX)

28. Locality to the Rescue • Internet Hierarchy • AS • ISP POP • Home/corporation • Branch office of a corporation • Delivery content in a locality aware fashion • Beyond ISP aware delivery

29. Internet : Grand View

30. Impact on ISPs • Economics of ISP relationships • sibling relationship • several ISPs belong to same org • peering relationship • mutual beneficial free agreement (to certain extent) • transit relationship • one ISP pays another Tier 1 ISP transit peering entity boundary peering Tier 2 ISP Tier 2 ISP AS AS AS AS sibling sibling entity boundary

31. Inside ISP

32. ISP POP (Point of Presence)

33. Inside Home/Branch Office neighborhood home corporation Branch office

34. Identify Peers Locality • Information used • External IP address • Internal IP address • Subnet mask • Peer locality • ISP (AS) • ISP POP • Home/corporation • Corporate branch office • Peers are considered closer if they are in a smaller common neighborhood

35. MAP External IP Address to AS Using BGP dump

36. Identify POP • POP neighborhood • Identify one peer that is directly connected to the Internet at some point of time • Collect its external IP address and the subnet mask • Infer the subnet neighborhood where other peers belong, even if they are not directly connected to the Internet

37. Below POP • Home/corporation neighborhood • All peers with the same external IP address • Corporation branch office • All peers with the same external IP address, and on the same internal subnet (based on subnet mask)

38. Locality Aware Topology Building • Preferentially link peers within the same ISP neighborhood • Say if we need to establish 20 connections • We assign 50% of links to be within branch office neighborhood • If there are less peers than the allocated links, we simply put the unused links back to the pool • We then assign 50% of unused links to be within home/office neighborhood • The next 50% of unused links are assigned within POP neighborhood • The next 50% of unused links are assigned within AS neighborhood • The rest of the links are used for cross-AS connections

39. Example

40. Locality Aware P2P Scheduling • Preferentially deliver content to peers within closer neighborhood • Propagate neighborhood availability information • Exchange with a outside peer preferentially content that is not available in the neighborhood

41. Preliminary Result: ISP Friendly • Without ISP-friendly • Much more cross sibling than peering boundary • Significant crossing boundary traffic Without ISP-friendly

42. Preliminary Result: ISP Friendly • Pure ISP-friendly • 1 video  5000+ separate distributions • still surprising reductions but unnecessarily conservative • ISP could help by sharing information cut cross boundary traffic completely

43. Conclusions

44. Conclusion • Peer assisted delivery is the way to go for mass content delivery over the world • Peer assistance can significantly reduce server bandwidth requirement • Demonstrated in real world for file sharing & broadcast • Shown in our work for on demand streaming • Locality aware P2P delivery is the way to scale