1 / 24

Opportunistic Multipath Forwarding in Publish/Subscribe Systems

Opportunistic Multipath Forwarding in Publish/Subscribe Systems. Reza Sherafat Kazemzadeh AND Hans-Arno Jacobsen Middleware Systems Research Group University of Toronto. Agenda. The content-based publish/subscribe model Characteristics and challenges Our approach Overlay neighborhoods

keegan
Download Presentation

Opportunistic Multipath Forwarding in Publish/Subscribe Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Opportunistic Multipath Forwarding in Publish/Subscribe Systems Reza SherafatKazemzadehANDHans-Arno Jacobsen Middleware Systems Research Group University of Toronto

  2. Agenda • The content-based publish/subscribe model • Characteristics and challenges • Our approach • Overlay neighborhoods • Adaptive multipath forwarding • Experimental evaluation results Opportunistic Multipath Forwarding

  3. Content-Based Pub/Sub Model P P P P Publish P P Pub/Sub S S S S S P S S S S Subscribers Publishers Opportunistic Multipath Forwarding

  4. Content-Based Pub/Sub Model • Many to many communication between a large number of publishers/subscribers • Selective delivery based on subscription matching • Unicast (one) • Multicast (some) • Broadcast (all) • Variations in traffic patterns makes it difficult to design an optimal overlay network P P Publish P Pub/Sub S S Subscribers Publishers Opportunistic Multipath Forwarding

  5. Current Pub/Sub Overlays Forwarding paths in the overlay are constructed in fixed end-to-endmanner (no/little path diversity) This results in a high number of “pure forwarding” brokers P P ✗ P ✓ ✗ ✗ S A B ✓ C D E #msgs delivered #msgs sent System Yield = = 1/5 = 20% Opportunistic Multipath Forwarding

  6. Summary of Our Goals • Construct a highly connected overlay mesh that provides high path diversity between publishers and subscribers • Avoid pure forwarders by allowing brokers to make fine-grained forwarding decisions based on individual publications and their matching sets • Improve system yield, efficiency, scalability and delivery delay • Support dynamic adaptive routing based on live traffic patters while avoiding high costs of full overlay reconfiguration Opportunistic Multipath Forwarding

  7. Forwarding Strategies • Fixed end-to-end (baseline) Total msgs: 6 • Forwarding strategy 1 Total msgs: 5 • Forwarding strategy 2 Total msgs: 3 p * * * * * * * * * * * * * * * A A A B B B C C C D D D * * * p p Opportunistic Multipath Forwarding

  8. Our Approach in a Nutshell Δ=3 Δ=2 Δ=1 A S Opportunistic Multipath Forwarding

  9. Path Computation forForwarding Strategies Strategy 1 Strategy 2 * * * * * * * * * * * * * * A A A A B B C C C C D D * * * * p B B D D p Opportunistic Multipath Forwarding

  10. Experimental Evaluations • We have implemented the algorithms and performed large-scale experimental evaluations with up to 500 brokers • Datasets • Synthesized based on Zipf distribution • Social networking traces from Facebook • We measured performance of the system in terms of: • Overlay mesh connectivity • Delivery delay • Maximum system throughput • System yield • Publication propagation path length • Memory and CPU overhead Opportunistic Multipath Forwarding

  11. Path Diversity in Overlay Mesh Number of Available paths Network size:250Delta=3 Path diversity: 10% of brokers w/ 1000 paths Network size:120Delta=3 Path diversity: 20% of brokers w/ 100 paths Graph is based on a snapshot of the state of links in a running system Opportunistic Multipath Forwarding

  12. 115% throughput enhancement Strategy 2 Fixed-end-to-end Publication Hop Count Experiment setup: 120 Brokers Publish rate is 1,800 msgs/sec and number of deliveries: 73,000 (in 5 min) Opportunistic Multipath Forwarding

  13. Conclusions • Brokers build a highly connected overlay mesh and make fine-grained forwarding decisions for each publication in order to avoid pure forwarding neighbors • We used the notion of overlay neighborhoods to enable local traffic profiling and avoid high costs of overlay reconfiguration • Our approach enhances system’s efficiency and yield, and ultimately improves its scalability and performance Opportunistic Multipath Forwarding

  14. Thanks Questions! Opportunistic Multipath Forwarding

  15. Pub/Sub Characteristics • Many to many communication between a large number of publishers/subscribers • Selective delivery based on subscription matching • Unicast (one) • multicast (some) • broadcast (all) • Traffic patterns depends on workload application and may change over time Client Broker Opportunistic Multipath Forwarding

  16. Related Work: Overlay Reconfiguration Broker overlay is “reconfigured” by addition and removal of links between brokers Advantages • Forwarding path may be improved bringing some publishers and subscribers closer together Disadvantages • Some forwarding paths between publishers and subscribers may indeed suffer • Resulting overlay still relies on fixed end-to-end paths • Reconfiguration is costly and requires full or partial re-propagation of subscriptions Re-configure     [1] Virgillito, A., Beraldi, R., Baldoni, R.: On event routing in content-based publish/subscribe through dynamic networks, FTDCS ‘03[2] Virgillito, A., Beraldi, R., Baldoni, R.: On event routing in content-based publish/subscribe through dynamic networks. In: FTDCS. (2003) Opportunistic Multipath Forwarding

  17. Links Management Overlay Network • Links types • Primary links • Soft links – shortcut links • Candidate links – expeditionary links • Traffic profiling • In intervals of T sec, brokers count thenumber of pubs sent over each links • Gain function gain(A,B,T) = pub_traffic_during_T * dist(A,B) • Soft link selection • Profiling input • Link state measured • Broker load information exchanged Soft link Primary link Candidate link C B A B D Link quality Neighbor load High rank Links ranking high → low A B C D E F G E F … Opportunistic Multipath Forwarding

  18. Master v. Working Routing Data Structures Master Routing Tables • Overlay map captured by brokers’ Δ-neighborhoods are relatively static Master overlay Map (MOM) • Brokers link connectivity change dynamically, brokers need an efficient way to compute forwarding paths over the changing set of links  Working Overlay Map (WOM) • MOM is a concise representation of the primary overlay that only contains brokers with a direct link construct Working Routing Tables Opportunistic Multipath Forwarding

  19. Master v. Working Subscription Tables Working Overlay Map Master Subscription Table • Similar to WOM, brokers adapt their subscription tables based on the current set of available links: Working Subscription Table (WST) Working Subscription Table Between Brokers Sets Beyond Brokers Sets Behind Brokers Sets Opportunistic Multipath Forwarding

  20. System Yield (measure of efficiency) Opportunistic Multipath Forwarding

  21. Publication Hop Count Experiment setup: 120 Brokers Sparse and dense workloads Publish rate of 1,800 msgs/sec  Deliveries: 73,000 in 5 min Opportunistic Multipath Forwarding

  22. Publication Hop Count Experiment setup: 120 Brokers Sparse publication/subscription workload Publish rate of 1,800 msgs/sec  Deliveries: 73,000 in 5 min Opportunistic Multipath Forwarding

  23. Publication Hop Count Sparse Matching Workload Dense Matching Workload Multi-path forwarding is more effective in sparse workloads Opportunistic Multipath Forwarding

  24. Impact of Broker Fanout on Subscription Covering Experiment setup: 500 brokers Fanout of 5-25 Opportunistic Multipath Forwarding

More Related