Oblivious AQM and Nash Equilibria

1. Oblivious AQM and Nash Equilibria D. Dutta, A. Goel and J. Heidemann USC/ISI USC USC/ISI IEEE INFOCOM 2003 - The 22nd Annual Joint Conference of the IEEE Computer and Communications Societies Presented By Sharon Mendel Game Theory in Networks Seminar 25/01/2006

2. Before we Begin… Today’s Internet - Motivation Oblivious AQM & Nash Equilibria

3. Before we Begin… Active Queue Management ?? • A congestion control protocol (e.g. TCP) operates at the end-points and uses the drops or marks received from the Active Queue Management policies (e.g. Drop-tail, RED) at routers as feedback signals to adaptively modify the sending rate in order to maximize its own goodput. Oblivious AQM & Nash Equilibria

4. Before we Begin… Transport Control Protocol • TCP is the dominating transport layer protocol in the Internet and accounts for over 90% of the total traffic. • The TCP Protocol is well defined, robust and congestion-reactive (thus stable). • The end-to-end congestion control mechanisms of TCP have been a critical factor in the robustness of the Internet. • It is widely believed that if all users deployed TCP, networks will rarely see congestion collapses and the overall utilization of the network will be high. Oblivious AQM & Nash Equilibria

5. Before we Begin… Today’s Internet • There are indications that the amount of non-congestion-reactive traffic is on the rise. • Most of this misbehaving traffic does not use TCP. • e.g. Real media, network games, other real time multimedia applications. • The unresponsive behavior can result in both unfairness and congestion collapse for the Internet. • The network itself must now participate in controlling its own resource utilization. * Some of the previous slides: “Promoting the Use of End-to-End Congestion Control in the Internet”, S. Floyd and K. Fall, IEEE/ACM Transactions on Networking Vol. 7 1999. Oblivious AQM & Nash Equilibria

6. Before we Begin… The Paper’s Motivation • TCP (and in fact, any transport protocol) does not guarantee good performance in the face of aggressive, greedy users (who are willing to violate the protocol to obtain better performance). • Protocol Equilibrium – A protocol which leads to an efficient utilization and a somewhat fair distribution of network resources (like TCP does), and also ensure that no user can obtain better performance by deviating from the protocol. • If protocol equilibrium is achievable, then it would be a useful tool in designing robust networks. Oblivious AQM & Nash Equilibria

7. More Introduction Oblivious AQM and Nash Equilibria Oblivious AQM & Nash Equilibria

8. More Introduction Oblivious AQM • Oblivious (stateless) AQM scheme – a router strategy that does not differentiate between packets belonging to different flows. • Stateful schemes – e.g. Fair-Queuing. • Stateful Schemes offer good performance, but oblivious schemes are easier to implement. Oblivious AQM & Nash Equilibria

9. More Introduction Popular AQM Schemes (1) • Congestion avoidance is achieved through packet dropping. • Drop-Tail – Buffers as many packets as it can and drops the ones it can't buffer: • Distributes buffer space unfairly among traffic flows. • Can lead to global synchronization as all TCP connections "hold back" simultaneously, hence networks become under-utilized. DT Oblivious AQM & Nash Equilibria

10. More Introduction Popular AQM Schemes (2) • RED – Random Early Dedication - Monitors the average queue size and drops packets based on statistical probabilities: • If the buffer is almost empty, all incoming packets are accepted; As the queue grows, the probability for dropping an incoming packet grows; When the buffer is full, the probability has reached 1 and all incoming packets are dropped. • Considered more fair than tail drop - The more a host transmits, the more likely it is that its packets are dropped. • Prevents global synchronization and achieves lower average buffer occupancies. RED Oblivious AQM & Nash Equilibria

11. More Introduction Oblivious AQM and Nash Equilibria • The paper studies the existence and quality of Nash equilibria imposed by oblivious AQM schemes on selfish agents: • Existence • Efficiency • Achievability Oblivious AQM & Nash Equilibria

12. Content • Introduction • The Model • Existence • Efficiency • Achievability • Summary and Future Work Oblivious AQM & Nash Equilibria

13. The Model The Markovian Internet Game Oblivious AQM & Nash Equilibria

14. The Model The Internet Game • Players – The end-to-end selfish traffic agents. • Each player has a strategy which is to control the average rate he tries to push through the network. • Users’ Performance Metric – goodput. • Rules – set by the AQM policies (AQM schemes in routers). • Nash Equilibrium – No selfish agent has any incentive to unilaterally deviate from it’s current state. • Oblivious AQM scheme leads to a protocol equilibrium only if it imposes a Nash equilibrium on the selfish users. • Paper’s focus– AQM schemes that guarantee bounded average buffer occupancy regardless of the total arrival rate. Oblivious AQM & Nash Equilibria

15. The Model The Markovian Internet Game • The agents generate Poisson traffic. • Does not accurately model Internet traffic, yet a reasonable first step. • Each user controls its own offered load = the average Poisson traffic rate. • The system is modeled as a M/M/1/K queue: • Average service time - Without loss of generality assumed to be unity. • Buffers capacity - K=B. • No assumptions are made on the selfish protocol (i.e. TCP, AIMD etc’). Oblivious AQM & Nash Equilibria

16. The Model The M/M/1/K Internet Game • n – Number of users – players. • λi – The Poisson average arrival rate of player i. • Ui – Utility function of player i. • μi – Goodput , Ui = μi . • p – AQM router drop probability due an average aggregated load (offered load) of λ and an average service time of unity. • A symmetric Nash equilibrium - Ensures that every agent has the same goodput at equilibrium. • Unless mentioned otherwise, quantities such as the rates, goodput and throughput are averages (Poisson traffic sources). Oblivious AQM & Nash Equilibria

17. DT RED The Model Nash Equilibrium Conditions (1) • No agent can increase their goodput, at Nash equilibrium, by either increasing or decreasing their throughput: • A symmetric Nash equilibrium: • Oblivious AQM scheme, hence functions of router state (drop probability, queue length) are independent in i: Utility Function in Nash Equilibrium : Nash Equilibrium Condition :Nash Equilibrium Satisfying Condition –Necessary and sufficient Oblivious AQM & Nash Equilibria

18. The Model Nash Equilibrium Conditions (2) • λNash , μNash,pNash - Theaggregate throughput (offered load), goodput, drop probability respectively. • The Nash equilibrium imposed by an AQM scheme isefficient if the goodput of any selfish agent isbounded below when the throughput (offered load) of the same agent isboundedabove: Nash Equilibrium Efficiency Condition : c1 c2some constants pNash is bounded Oblivious AQM & Nash Equilibria

19. Existence Are there oblivious AQM schemes that impose Nash equilibria on selfish users? Oblivious AQM & Nash Equilibria

20. Existence Drop-Tail Queuing • p - Drop Probability = Probability to find a full system. • From Queuing Theory: • Theorem 1: There is NO Nash Equilibrium for selfish agents and routes implementing Drop-Tail queuing. • Proof: applying the condition for Nash equilibrium: QED. Oblivious AQM & Nash Equilibria

21. If lq <minth 0 p = If minth lq  maxth 1 Otherwisw Existence Random Early Detection • Approximated steady state RED model: • From Queuing Theory, at steady state: • RED Router, at steady state: Steady State: “Faster network simulation with scenario pre-filtering” Tech. Rep., USC/ISI Tech Report 550, November 2001. lq – Queue length p – Drop Probability λ – Aggregated Offered Load lq maxth Oblivious AQM & Nash Equilibria

22. Existence RED and Nash Equilibria • Theorem 2: RED Does NOT impose a Nash equilibrium on uncontrolled selfish agents. • Proof: applying the condition for Nash equilibrium: • Summary: • RED punishes all flows with the same drop probability. • The nature of the drop function is considerably gentle. • Misbehaving flows can push more traffic and get less hurt (marginally). • There is no incentive for any source to stop pushing packets. QED. RED is oblivious: Nash Equlibria does not exist. Oblivious AQM & Nash Equilibria

23. 0 If lvq <minth p = If minth lvq  maxth Otherwise 1 Existence Virtual Load RED • Virtual Load RED model: • Theorem 3: VLRED imposes a Nash Equilibrium on selfish agents if • Proof: • Throughput at Nash equilibrium is independent of minth: lvq – M/M/1 Queue length when facing the same load : Oblivious AQM & Nash Equilibria

24. Efficiency If an Oblivious AQM scheme can impose a Nash equilibria, is that equilibria efficient, in terms of achieving high goodput and low drop probability? Oblivious AQM & Nash Equilibria

25. Total Offered Load Normalized Data Goodput minth = 0 maxth =50 n – 1,..,50 Number Of Flows Efficiency Example - Efficiency of VLRED • Proof: Applying Nashequilibrium satisfying andefficiency conditions: Theorem 4: VLRED is not efficientimposing a Nash Equilibriumon selfish agents α, β - some constants nμ = θ(lvq2) nμ/α lvq2 The goodput falls to zero asymptotically. QED. Oblivious AQM & Nash Equilibria

26. Total Offered Load Goodput Normalized Data Drop Probability Number Of Flows Efficiency Efficient Nash AQM scheme • Assume – Totaldesirable offered loadat Nash equilibrium: • Oblivious mechanism can ensure an efficient Nash equilibria under selfish behavior of users. ENAQM drop probability is bounded Oblivious AQM & Nash Equilibria

27. Achievability How easy is it for players (users) to reach the equilibrium point? orHow can we ensure that agents actually reach the Nash equilibrium state? Oblivious AQM & Nash Equilibria

28. Achievability Ensuring a Nash equilibrium by an Oblivious AQM • λ*i – Offered load at equilibria when the number of agents is i. • Δi = λ*i -λ*i-1 = iααsome constant • p = ƒ(λ*i) non decreasing and convex. • Applying Nash equilibrium satisfying and efficiency conditions: • From convexity of p*i: • From efficiency: • The equilibrium imposed by any oblivious AQM strategy is (very) sensitive to the number of agents, thus making it impractical to deploy in the Internet. • Agents need the help of the router to reach the equliibria. c1, c2 - some constants The sensitivity coefficient falls faster than the inverse quadric. c - constant Oblivious AQM & Nash Equilibria

29. Summary and Future Work Overview Now what? – Future Work Some further work Oblivious AQM & Nash Equilibria

30. Summary and Future Work Overview • Introduction – Today’s Internet. • The proposed model – Markovian (M/M/1/K) Game • Existence – • Drop tail and RED cannot impose a Nash equilibra. • VLRED imposes a Nash equilibra.But the equilibrium points do not have a very high utilization. • Efficiency - ENAQM imposes an efficient Nash equilibra. • Achievability - Equilibrium points in oblivious AQM strategies are very sensitive to the change in the number of users. • It may be hard to deploy oblivious schemes that do have Nash equilibria without the explicit help of a protocol. Oblivious AQM & Nash Equilibria

31. Summary and Future Work Now What ? - Future Work • VLRED – Explore why the Nash equilibria do not result in good network utilization. • Conjecture – VLRED’s drop function becomes very harsh as we reach equilibria. • Study gentler versions of VLRED and determine whether such modification can still impose Nash equilibria. • Design protocols which lead to efficient network operation, such that no user has any incentive to unilaterally deviate from the protocol – can it be done ? – The Protocol Equilibrium Question. Oblivious AQM & Nash Equilibria

32. Summary and Future Work Some Further work (1) • “Towards Protocol Equilibrium with Oblivious Routers” D. Dutta, A. Goel and J. Heidemann, IEEE INFOCOM 2004. • “…In this paper, we show that if routers used EWMA to measure the aggregate rate, then the best strategy for a selfish agent to minimize its losses is to arrive at a constant rate. Even though the protocol space is arbitrary, our scheme ensures that the best greedy strategy is simple, i.e. send with CBR. Then, we show how we can use the results of an earlier paper to enforce simple and efficient protocol equilibria on selfish traffic agents…” Oblivious AQM & Nash Equilibria

33. Summary and Future Work Some Further work(2) • “Pricing Differentiated Services A Game -Theoretic Approach”, E. Altman, D. Barman, R. El Azouzi, D. Ros, B. Tuffin, (accepted for publication in Computer Networks, 2005). • “…The goal of this paper is to study pricing of differentiated services and its impact on the choice of service priority at equilibrium. We consider both TCP connections as well as non controlled (real time) connections…. We first study the performance of the system as a function of the connections’ parameters and their choice of service classes. We then study the decision problem of how to choose the service classes. We model the problem as a noncooperative game. We establish conditions for an equilibrium to exist and to be uniquely defined. We further provide conditions for convergence to equilibrium from non equilibria initial states. We finally study the pricing problem of how to choose prices so that the resulting equilibrium would maximize the network benefit…” • “…The paper (“Oblivious AQM and Nash Equilibria”) restricted itself to symmetric users and symmetric equilibria and the pricing issue was not considered. In this framework, with a common RED buffer, it was shown that an equilibrium does not exist. An equilibrium was obtained and characterized for an alternative buffer management that was proposed, called VLRED. We note that in contrast to (Oblivious AQM and Nash Equilibria”), since we also include in the utility of CBR traffic a penalty for losses we do obtain an equilibrium when using RED…” Oblivious AQM & Nash Equilibria

34. Thank You ! Oblivious AQM & Nash Equilibria

35. Appendixes Stateful Schemes – Fair Queuing From Queuing Theory Nash Equilibrium Conditions Approximated Steady State RED Model VLRED Model, Theorem 3 - Nash Equilibrium Existence - Proof Efficient Nash AQM Drop Probability Oblivious AQM & Nash Equilibria

36. Introduction - Appendix Stateful Schemes – Fair Queuing - Nagle’s Algorithm • Gateways maintain separate queues for packets from each individual source. • The queues are serviced in a round-robin manner. • Nagle’s algorithm, by changing the way packets from different sources interact, does not reward, nor leave others vulnerable to, anti-social behavior. “Analysis and Simulation of a Fair Queuing Algorithm”, A.Demers, S. Keshav and S.J. Shenker, ACM SIGCOMM, 1989. Oblivious AQM & Nash Equilibria

37. λ λ λ λ λ 0 1 K-1 K 2 μ μ μ μ μ From Queuing Theory • M/M/1/K Queuing system - Poisson arrivals, Exponentially distributed service times, one server and finite capacity buffer: • PASTA - Poisson Arrivals See Time Averages – For a queuing system, when the arrival process is Poisson and independent of the service process:The probability that an arriving customer finds i customers in the system Equals The probability that the system is at state i. • πi – Probability that the system is in state i, Using birth-death model: • Block Probability - Oblivious AQM & Nash Equilibria

38. The Model – Appendix Nash Equilibrium Condition • Substituting 2,3 in 5 we obtain: Nash Equilibrium Satisfying Condition : Oblivious AQM & Nash Equilibria

39. Approximated RED Transfer Function Drop prob. Standard RED Transfer Function Drop prob. Normalized Offered Load Approximated RED Buffer Occupancy Avg. Buffer Occupancy Avg. Buffer Occupancy Normalized Offered Load Existence - Appendix Approximated Steady State RED Model “Faster network simulation with scenario pre-filtering” Tech. Rep., USC/ISI Tech Report 550, November 2001. Example: minth = 10 maxth =20 n – 1,..,50 Pmax = 0.3 Oblivious AQM & Nash Equilibria

40. 0 If lvq <minth p = If minth lvq  maxth Otherwise 1 Existence - Appendix VLRED Model, Theorem 3 - Nash Equilibrium Existence - Proof (1) • Virtual Load RED model: • Proof of Theorem 3: VLRED imposes a Nash Equilibrium on selfish agents if • Assume the drop probability can be written as a continues function for all lvq<maxth: • Nash equilibrium condition: lvq – M/M/1 Queue length when facing the same load : Oblivious AQM & Nash Equilibria

41. Existence - Appendix VLRED Model, Theorem 3 - Nash Equilibrium Existence - Proof (2) • Substituting and simplifying we obtain: • This is true if (by Substituting n=1) Oblivious AQM & Nash Equilibria

42. Efficiency - Appendix Efficient Nash AQM Drop Probability • Total desirable offered load at Nash equilibrium: • Substituting y2=1-λ and then integrating: • k is determined so when there is one user, the drop probability is zero as long as his offered load is less than unity • The offered load at Nash equilibria is bounded drop probability at equilibria is bounded (proved by substitution). k - arbitrary constant to be determined Oblivious AQM & Nash Equilibria