New Directions in Peering for Tier-2 and Content Providers

1. New Directions in Peering for Tier-2 and Content Providers Jeb R. Linton Staff Network Engineer, EarthLink jrlinton@ieee.org

2. Introduction and Acknowledgements • Who am I and what am I doing here? • Acknowledgements

3. Topics to be Covered • Peering Motivations and Economics • The Cost of Peering - History • Disruptions - What do they mean? • New Low Transit Costs • New Tier-1 Peering Initiative • Metropolitan Multipoint Private Peering • FUD • (Teaser: What Happens Next?)

4. Peering Motivations and Economics

5. The Role of Peering • To Large Tier-1s • Mission-Critical • To the Rest of Us • Used to Decrease Cost of Transit • Used to Increase Quality of Routes • Usually not mission-critical * * Your Mileage May Vary

6. The Cost of Peering - History

7. Peering Cost – the Legacy NAPs • 2-3 years ago, cost justification for NAP peering was easy based on the high cost of Transit See Bill Norton’s whitepaper: A Business Case for ISP Peering for the origin and explanation of this method of cost justification. Bill’s graphing method will be used throughout.

8. Peering Cost – the Legacy NAPs, cont’d Disruption #1: Rapid drop in Transit cost makes legacy NAPs much less desirable

9. Peering Cost – Private Lines as Supplement • Evolution: NAP Peering is supplemented with Private Peering.

10. Peering Cost –Private Lines for Many Peers • Problem: Private Peering to many Peers can be difficult because of the repeated expenditure and low circuit utilization efficiency

11. Peering Cost – Tier-1 Peering Initiative Disruption #2: Large Tier-1 Providers create the Next-Generation Peering Initiative • Common facilities are chosen, to which each major Tier-1 will acquire fiber rights • Allows excellent scalability while minimizing repeated cost • Independence, Fault Tolerance, and Provisioning time improve

12. Next-Generation Tier-1 Peering: Before • N*(N-1)/2 Links in each of 7 Cities • Dependence on RBOCs • Slow Provisioning • Mostly OC-3 and OC-12 • Poorly adapted to use of Dark Fiber Spr ATT UU L3 Gen Q CW

13. Next-Generation Tier-1 Peering: After • One-time Dark Fiber provisioning to a single facility • Each Tier-1 maintains routers in the same room • Cheap, fast cross-connects for OC48 or higher peering Spr ATT UU L3 NexGen Peering Center Gen Q CW

14. Peering Cost – Appearance of Metro Ethernet Disruption #3: CheapMetro Ethernet services connect to multiple peers over the same port using Point-to-Point and Multipoint VLANs

15. Disruptions – What do they mean?

16. Disruption: Low-Cost Transit • Legacy Peering methods lose popularity – NAP attachments and Private Line Peering links are no longer saving money. • Some companies retain NAP presence just to meet contractual requirements, or for marketing reasons. • Increased pressure to find cheaper Peering methods.

17. Next-Gen Tier-1 Peering: Implications • Large Tier-1s draw smaller Tier-1s to new Centers for Peering • Largest Tier-2 and Content providers drawn to the Centers hoping to peer with Tier-1s • Smaller Content, Tier-2s, Enterprises and others are drawn to new Centers for competitive Transit and some Peering What happens to all the other collocation?

18. Price Disruption: Metro-Ethernet • Most major Collocation spaces in “NFL Cities” are already Metro-Ethernet enabled • Pressure to replace individual Private Lines (DS-3, OC-3, OC-12) with Metro-Ether where possible. • Legacy NAP viability may depend on ability to accommodate Ethernet links • Many are ATM-based and charge heavily for collocation • Some have adapted or may yet adapt

19. Metro Multipoint Private Peering: Before (The situation for Smaller Tier-1, Tier-2, and Content) • Private Peering • NAP Peering • In-Building Peering if you’re lucky. • Low efficiency • High Cost • Slow Provisioning Those Guys ATM OC3 Old NAP Private DS3s ME YOU HIM

20. Metro Multipoint Private Peering: After (The situation for Smaller Tier-1, Tier-2, and Content) • Cross-Connect to an Ethernet port in your current facility • Multiple VLANs for Private and Public Peering, and Transit • Link to NexGen Peering Centers? Those Guys HIM ME Public Peering VLAN Transit Provider YOU ? NexGen Peering Center • Efficient, Cheap, and Fast

21. Metro Multipoint Private Peering • Addresses some limitations of Private Peering • Multiple Peer links on the same physical port • Private, Public, and Transit share same ports • Many configurations and payment options • Pay by port, by utilization, or by allocation • P-to-P VLANs per peer, or semi-public Multipoint VLANs • Bandwidth allocated per port or per VLAN • Technical options: Broadcasts, Switch Config, Failover, etc.

22. Metro Multipoint Private Peering • Probably won’t scale to tens of Gbps • May scale further once 10Gig-E is available • Scaling is better addressed by Dark Fiber/Lambda connections to the NexGen Peering Centers • Optimum scalability only reached if all local peers use the same Metro-Ether Provider (!)

23. FUD* * Fear, Uncertainty, and Doubt

24. FUD – The Tier-1 Peering Initiative • Q. Is this Unfair? Non-Inclusive? Anti-Competitive? • A. No… it’s just common sense, and anyone can join in. • Q. Will this increase Transit prices? • A. Competition should improve Transit pricing in the Peering Centers, and have no effect outside them.

25. FUD – The Tier-1 Peering Initiative • Q. What about my old CoLo? • A. Thanks to cheap metro, most large Collocation centers are still viable, but you may want to get a foot in the door at the new peering centers. • Q. Are the Tier-1’s really committed to this? • A. Yes… It’s the only viable long-term option, and they don’t want to waste the investment they’ve already made.

26. FUD - Metro-Multipoint Private Peering • Q. Early Entry is scary… why not wait? • A. Very low commitment is needed; can start below 10M • Q. Does the service have the robustness of Private Line? • A. All vendors use diversity wherever available. Most use same or similar configuration to the standard SONET/Lambda services we all rely on

27. FUD – Metro-Multipoint Peering, cont’d • Q. What about the financial stability of the vendors? Isn’t the cheap pricing based on gaining market share? We saw what happened in the last year… What about today’s vendors? (Applies to Metro-Ether and Peering-Collocation vendors) • A. Reduce your risk with short contract terms…Multiple vendors means you can switch if need be.

28. FUD – Metro-Multipoint Peering, cont’d • Q. What technical gotchas should we be aware of? • A. Management overhead similar to old NAP, and Broadcast and Congestion issues need to be resolved Bottom line: Mitigate risk with short contract terms, and do your own financial analyses… Caveat Emptor!

29. Points of Contact: Earthlink • Jeb Linton (Architecture/Peering Strategy) – jeblinton@corp.earthlink.net • Josh Fleishman (Peering Coordinator) – flyman2@corp.earthlink.net

30. Backup: What Happens Next?

31. Backup: What Happens Next • In “NFL Cities”, expect most large Internet companies (and many small ones) to migrate to the NexGen Peering Centers. • In all areas with Metro-Ethernet, expect off-net Collocation Centers to die off. • Expect ATM-based NAPs to continue losing popularity. • Metro-Ethernet will predominate for new P-to-P Private Peering links and links to the NexGen Peering Centers. • Metro-E will be used for Multipoint Peering. (Services already emerging from several vendors) • One Metro-E vendor will begin to predominate for Peering in each Metro area. • Peers will migrate from MM-PP to the Centers as they grow.

32. Backup: What Happens Next… the Big Picture • Large Internet companies interested in significant Peering will migrate toward the NexGen Peering Centers • Mid-sized companies will use both: • NexGen Peering Centers, for Tier-1 and High-Speed peering • Metro-Multipoint for smaller peers, to maximize savings • Small companies will: • Start with a Metro Ether port for Transit, B-B, etc. • Add peering VLANs on the same port for cost savings • Point-to-Point for B-B links and opportunistic private peering • Metro-Multipoint for additional cost savings • Use Metro Ethernet as a bridge to the NexGen Peering Centers as they grow

33. Backup - Terminology (for convenience only) • Tier-1 – Providers of Internet Access to Autonomous Systems (a.k.a. “Leased Transit Service”, “Transit”) • Tier-2 – Providers of Internet Access to Individuals • NexGen Peering Centers – Collocation Centers chosen as common locations for Peering by the Largest Tier-1 Providers

34. Backup - Who wants to peer with whom? • Largest Tier-1 • Each Other • Other Tier-1 • Largest Tier-1, each other, large Tier-2, and some Content • Large Tier-2 • Content, all Tier-1, and each other • Content • Everybody but each other • Other (Who is Other?) • Academe, Enterprises, R&D, etc… the Usual Suspects

35. Other Similar Largest Tier-1 Other Similar Other Similar Other Tier-1 Largest Tier-2 Content-Heavy Traffic Mostly Outbound Eyeball-Heavy Traffic Mostly Inbound Content Other Tier-2 Other Similar Other Backup: The “Peering Promiscuity Chart” Arrows point toward Targeted Peers

36. Backup – Metro Ethernet

37. Backup – Point-to-Point Metro Ethernet

38. Backup – Multipoint Metro Ethernet

39. Backup - Evaluating Available Methods of Peering • Questions to Ask: • How much Peering traffic is locally available using this peering method? • How much does this Peering Method cost per Mb for the expected traffic level? • What does the company pay per Mb for Transit in this metro area? • Things to consider: • Should be half the cost of Transit (per Mb) or better • Methods of Peering are complimentary, not exclusive