Beyond selfish routing: Network Games

1. Beyond selfish routing:Network Games

2. Network Games • NGs model the various ways in which selfish agents strategically interact in using a network • They aim to capture two competing issues for agents: • to minimize the cost they incur in creating/using the network • to ensure that the network provides them with a high quality of service

3. Motivations • NGs can be used to model: • social network formation (edge represent social relations) • how subnetworks connect in computer networks • formation of P2P networks connecting users to each other for downloading files (local connection games) • how users try to share costs in using an existing network (global connection games)

4. Setting • What is a stable network? • we use a NE as the solution concept • we refer to networks corresponding to Nash Equilibria as being stable • How to evaluate the overall quality of a network? • we consider the social cost: the sum of players’ costs • Our goal: to bound the efficiency loss resulting from selfishness

5. A warming-up network game: The ISPs dilemma C, S: peering points s1 two Internet Service Providers (ISP): ISP1 e ISP2 t2 C S ISP1 wants to send traffic from s1 to t1 t1 ISP2 wants to send traffic from s2 to t2 s2 Long links incur a per-user cost of 1 to the ISP owning the link Each ISP can use two paths: either passing through C or not

6. A (bad) DSE C, S: peering points s1 t2 Cost Matrix C S t1 ISP2 s2 ISP1  PoA is (4+4)/(2+2)=2 Dominant Strategy Equilibrium

7. Our case study:Global Connection Games

8. The model • G=(V,E): directed graph, k players • ce: non-negative cost of e E • Player i has a source node si and a sink node ti • Strategy for player i: a path Pi from si to ti • The cost of Pi for player i in S=(P1,…,Pk) is shared with all the other players using (part of) it: costi(S) =  ce/ke ePi this cost-sharing scheme is called fair or Shapley cost-sharing mechanism

9. The model • Given a strategy vector S, the constructed network will be N(S)= i Pi • The cost of the constructed network will be shared among all players as follows: • Notice that each user has a favorable effect on the performance of other users (so-called cross monotonicity), as opposed to the congestion model of selfish routing cost(S)= costi(S) =  ce/ke=  ce ePi i i eN(S)

10. Goals • Remind that given a strategy vector S, N(S) is stable if S is a NE • To evaluate the overall quality of a network, we consider its social cost, i.e., the sum of all players’ costs • A network is optimal or socially efficient if it minimizes the social cost

11. Be optimist!: The price of stability (PoS) • Definition (Schulz & Moses, 2003): Given a game G and a social-choice minimization (resp., maximization) function f (i.e., the sum of all players’ payoffs), let S be the set of NE, and let OPT be the outcome of G optimizing f. Then, the Price of Stability (PoS) of G w.r.t. f is:

12. Some remarks • PoA and PoS are (for positive s.c.f. f) •  1 for minimization problems •  1 for maximization problems • PoA and PoS are small when they are close to 1 • PoS is at least as close to 1 than PoA • In a game with a unique NE, PoA=PoS • Why to study the PoS? • sometimes a nontrivial bound is possible only for PoS • PoS quantifies the necessary degradation in quality under the game-theoretic constraint of stability

13. An example 3 s2 1 3 1 1 3 1 2 s1 t2 t1 4 5.5

14. An example 3 s2 1 3 1 1 3 1 2 s1 t2 t1 4 5.5 optimal network has cost 12 cost1=7 cost2=5 is it stable?

15. An example 3 s2 1 3 1 1 3 1 2 s1 t2 t1 4 5.5 …no!, player 1 can decrease its cost cost1=5 cost2=8 …yes, and has cost 13! is it stable?  PoA  13/12, PoS ≤ 13/12

16. An example 3 s2 1 3 1 1 3 1 2 s1 t2 t1 4 5.5 …a best possible NE: cost1=5 cost2=7.5 the social cost is 12.5  PoS = 12.5/12

17. Addressed issues in GCG • Does a stable network always exist? • Does the repeated version of the game always converge to a stable network? • How long does it take to converge to a stable network? • Can we bound the price of anarchy (PoA)? • Can we bound the price of stability (PoS)?

18. Theorem 1 Any instance of the global connection game has a pure Nash equilibrium, and best response dynamic always converges. Theorem 2 The price of anarchy in the global connection game with k players is at most k. Theorem 3 The price of stability in the global connection game with k players is at most Hk, the k-th harmonic number.

19. The potential function method For any finite game, an exact potential function is a function that maps every strategy vector S to some real value and satisfies the following condition: • S=(s1,…,sk), s’isi, let S’=(s1,…,s’i,…,sk), then (S)-(S’) = costi(S)-costi(S’). A game that does possess an exact potential function is called potential game

20. Lemma 1 Every potential game has at least one pure Nash equilibrium, namely the strategy vector S that minimizes(S). Proof: consider any move by a player i that results in a new strategy vector S’. Since (S) is minimum, we have: (S)-(S’) = costi(S)-costi(S’)  0 player i cannot decrease its cost, thus S is a NE. costi(S)  costi(S’)

21. Convergence in potential games Observation: any state S with the property that (S) cannot be decreased by altering any one strategy in S is a NE by the same argument. This implies the following: Lemma 2 In any finite potential game, best response dynamic always converges to a Nash equilibrium Proof:best response dynamic simulates local search on .

22. …turning our attention to the global connection game… Let  be the following function mapping any strategy vector S to a real value [Rosenthal 1973]: (S) = eN(S) e(S) where (recall that ke is the number of players using e) e(S) = ce · H = ce · (1+1/2+…+1/ke). ke

23. Lemma 3 ( is a potential function) Let S=(P1,…,Pk), let P’i be an alternative path for some player i, and define a new strategy vector S’=(S-i,P’i). Then: (S) - (S’) = costi(S) – costi(S’). Proof: • It suffices to notice that: • If edge e is used one more time in S: (S+e)=(S)+ce/(ke+1) • If edge e is used one less time in S: (S-e)=(S) - ce/ke • (S) -(S’) = (S) -(S-Pi+P’i) = (S)– ((S) - ePi ce/ke + eP’ice/(ke+1))= costi(S) – costi(S’).

24. Existence of a NE Theorem 1 Any instance of the global connection game has a pure Nash equilibrium, and best response dynamic always converges. Proof:From Lemma 3, a GCG is a potential game, and from Lemma 1 and 2 best response dynamic converges to a pure NE.

25. Price of Anarchy: a lower bound k s1,…,sk t1,…,tk 1 optimal network has cost 1  best NE: all players use the lower edge PoS is 1  worst NE: all players use the upper edge PoA is k

26. k k k k     i=1 i=1 i=1 i=1 Upper-bounding the PoA Theorem 2 The price of anarchy in the global connection game with k players is at most k. Proof: Let OPT=(P1*,…,Pk*) denote the optimal network, and let i be a shortest path in G between si and ti. Let w(i) be the length of such a path, and let S be any NE. Observe that costi(S)≤w(i) (otherwise the player i would change). Then: cost(S) = costi(S)≤ w(i)≤ w(Pi*)≤ k·costi(OPT) = k·cost(OPT).

27. PoS for GCG: a lower bound >o: small value t1,…,tk 1/k 1/(k-1) 1/3 1 1/2 sk-1 . . . s1 s2 s3 sk 1+ 0 0 0 0 0

28. PoS for GCG: a lower bound >o: small value t1,…,tk 1/k 1/(k-1) 1/3 1 1/2 sk-1 . . . s1 s2 s3 sk 1+ 0 0 0 0 0 The optimal solution has a cost of 1+ is it stable?

29. PoS for GCG: a lower bound >o: small value t1,…,tk 1/k 1/(k-1) 1/3 1 1/2 sk-1 . . . s1 s2 s3 sk 1+ 0 0 0 0 0 …no! player k can decrease its cost… is it stable?

30. PoS for GCG: a lower bound >o: small value t1,…,tk 1/k 1/(k-1) 1/3 1 1/2 sk-1 . . . s1 s2 s3 sk 1+ 0 0 0 0 0 …no! player k-1 can decrease its cost… is it stable?

31. PoS for GCG: a lower bound >o: small value t1,…,tk 1/k 1/(k-1) 1/3 1 1/2 sk-1 . . . s1 s2 s3 sk 1+ 0 0 0 0 0 A stable network k social cost:  1/j =Hk  ln k + 1 k-th harmonic number j=1

32. Lemma 4 Suppose that we have a potential game with potential function , and assume that for any outcome S we have cost(S)/A  (S)  B cost(S) for some A,B>0. Then the price of stability is at most AB. Proof: Let S’ be the strategy vector minimizing  (i.e., S’ is a NE) Let S* be the strategy vector minimizing the social cost we have: cost(S’)/A  (S’)  (S*)  B cost(S*)  PoS ≤ cost(S’)/cost(S*) ≤ A·B.

33. Lemma 5 (Bounding  ) For any strategy vector S in the GCG, we have: cost(S)  (S)  Hk cost(S) Proof:Indeed: (S) = eN(S) e(S) = eN(S) ce· Hke  (S)  cost(S) = eN(S) ce and (S) ≤ Hk· cost(S) = eN(S) ce· Hk.

34. Upper-bounding the PoS Theorem 3 The price of stability in the global connection game with k players is at most Hk, the k-th harmonic number Proof:From Lemma 3, a GCG is a potential game, and from Lemma 5 and Lemma 4 (with A=1 and B=Hk), its PoS is at most Hk.