Distributed Multiagent Resource Allocation In Diminishing Marginal Return Domains

1. Distributed Multiagent Resource Allocation In Diminishing Marginal Return Domains Yoram Bachrach(Hebew University) Jeffrey S. Rosenschein (Hebrew University)

2. Outline • Multiagent Resource Allocation (MARA) • General problem • Applications • Centralized and decentralized mechanisms • Selfish behavior challenge • Specific restricted domain • VCG solution in restricted domain • Allocation by interaction • Market motivation behind method • Allocation protocol and suggested strategies • Convergence to optimal allocation • Strategic and selfish behaviour • Expected time to convergence • Conclusions and future research

3. Multiagent Resource Allocation • Allocating resources to users • Scarce resources • Selfish agents with private information • Both users and resource owners • An allocation maps resources to users

4. MARA Applications Industrial procurement Satellite resources Tasks in manufacturing systems Grid computing RF spectrum and coverage …

5. MARA Domain Properties • Divisible / Indivisible • Can parts of a single resource be allocated to several agents? • Sharable / Non-Sharable • Can a resource be allocated to several agents simultaneously? • Single-Unit / Multi-Unit • Are there bundles of identical resources? • Transferable / Non Transferable Utility • Can agents compensate by transferring utility among them?

6. MARA Approaches • Attempt to maximize social welfare • Other possible goals – Maximin, fairness, … • There may be more than one optimal allocation • Centralized mechanisms • A central mechanism gets the agents’ preferences and chooses an outcome • Decentralized approaches • Agents actively participate in choosing the outcome • Problem – agents are selfish and attempt to maximize their own utility

7. Centralized Mechanisms • The mechanism must elicit the agents’ private information about allocations • But agents may manipulate to increase their own utility • We are interested in incentive compatible mechanisms • Agents reply truthfully, under a certain rational behavior • Rational behavior captured in a game theoretic solution concept • Vickery-Clarke-Groves (VCG) approach • Tax agents to make truth telling is a dominant strategy • Strategyproof, allocatively efficient but only weakly budget balanced

8. Distributed Mechanisms • Central mechanisms may not be appropriate in distributed environments • Hard to establish a trusted central authority • Scalability concerns – the central mechanism may be a performance bottleneck • Have agents interact among themselves to choose the allocation • Need to define the protocol for interaction • Selfish agents may still manipulate

9. Specific Domain • Set of identical agents • Each agent only requires a single resource, and does not benefit from being allocated more than one resource • Set of resources • Cannot be divided among agents • Can be shared among agents • Diminishing marginal production • The total utility of the agents who are allocated a certain resource drops as more agents use that resource

10. Diminishing Marginal Return 10 14 15 5 7 5 10 7 5

11. Diminishing Marginal Return Total production is 10 10 14 15 5 7 5 10 7 5

12. Diminishing Marginal Return Total production increases to 14 10 14 15 5 7 5 10 7 5

13. Diminishing Marginal Return Total production increased by 4 when adding a single agent Marginal production of 4 10 14 15 5 7 5 10 7 5

14. Diminishing Marginal Return Total production increased by 1 when adding a single agent Marginal production of 1 10 14 15 5 7 5 10 7 5

15. What needs to be decided? • A mechanism must decide: • An allocation – which agent gets which resource • We want to maximize the social welfare – total production • Utility transfers • Agents gain utility due to the allocation • Resource owners receive nothing • Resource owners hold the private information • Eliciting this information requires incentivizing the resource owners to report their production function • Requires giving resource owners some of the utility • We assume the total production across all the resources can be redistributed in any way

16. VCG in Restricted Domain • Easy to compute an optimal allocation • Resources report total production functions • Find maximal social welfare by a greedy algorithm • Assign to the resource with maximal marginal production • Induce truthfullness by VCG tax • Requires establishing a trusted central authority • Trust and security issues, central bottleneck, … • Weakly budget balanced – some of the total production is kept in the mechanism and not distributed

17. Allocation by Interaction • Define a protocol for interaction between agents and resource owners • Simulate a market for services • Interaction proceeds in discrete time rounds • Each round determines both an allocation and transfers • Design protocol and suggest interaction strategies so that the optimal allocation is always reached • Challenges • Achieve the optimal allocation despite selfishness • Make sure the optimal allocation is reached quickly

18. Interaction Protocol Currently on R1, getting utility 5 R1 R2 Round Payment (5) R3

19. Interaction Protocol Currently on R1, getting utility 5 R1 R2 Resource Request R3

20. Interaction Protocol R1 R2 Payment Bid (10) R3

21. Interaction Protocol Switch to R2 with utility 10 R1 R2 Accept R3

22. Interaction Protocol Stay on R1, with utility 5 R1 R2 Decline R3

23. Interaction Protocol Currently on R2 with utility 10 R1 R2 Round Payment 10 R3

24. Interaction Protocol Currently on R2 with utility 5 R1 R2 Payment Change (5)

25. The Resource Owner’s Perspective Production – 12 Payments – 10 Utility – 2 13 12 Production – 13 Payments – 12 Utility – 1 4 5 4 5 4

26. Chosen Allocation • The interaction decides both the allocation and redistribution of the utility • Agents are allocated the last resource whose bid they accepted • Agents get the utility as in the last payment bid they accepted • Resource owners keep the reminder of the production on the resource not redistributed to the agents • The allocation may change at the end of every round • An allocation is stable if once reached it never changes • Depends on the strategies of the participants • Agents and resource owners

27. Suggested Strategy - Agents • Each round, randomly choose a resource and request using the resource • If the bid in that resource is better than the current bid, switch to that resource (accept) • If the bid is lower than the current resource offers, stay with current resource

28. Suggested Strategy – Resource Owners • Keep the agents’ share of the utility in the level of the marginal production on the resource • On round start, offer all the agents allocated to the resource the current last marginal production • Answer resource requests with bid of the next marginal production on the resource • If accepted, set the bid for all the agents to the new marginal production by a Payment Change message • If declined – do nothing

29. Resource Owners - Example MP = 4 MP = 1 10 14 15 1 4 1 10 4 1

30. Protocol Stable Allocation • Given a set of strategies for the agents and resource owners, a protocol stable allocation is one that, once reached, never changes • Under these strategies, no interaction results in an agent switching to a different resource • Protocol stable under the suggested strategies • No agent is ever given a bid higher than what he is currently getting on his current resource • Resource owners bid the next marginal production • There is no resource where the next marginal production is greater than the current marginal production on other resources • Similar to greedily allocating agents to resources according to marginal production

31. Convergence to Optimum • Under the suggested strategies, the chosen allocation always converges to the optimal allocation • Monotonic improvement • If an agent switches resources, the social welfare increases • Stability in optimum • The optimal allocation is protocol stable • No “local” optima – protocol stable is optimal • If a non optimal allocation is chosen, there is a possible round where an agent switches resources • What about strategic behavior?

32. Strategic Behavior • Agents and resource owners have to follow the protocol, but not the suggested strategies • Might obtain higher utility by choosing a different strategy • Agents may accept a bid lower than what they currently have • Resource owners may suggest a bid different than the current marginal production • Higher, to attract more agents • Lower, to give a lower share of utility to the agents • Is such strategic behavior rational for self interested agents?

33. Strategic Agents (Our domain) • If an agent gained from strategic behavior, we still reach an optimal allocation • If a single agent has deviated from the suggested strategy and gained utility • Gained utility: a protocol stable allocation has been reached, in which the agent gets a higher utility • Then the reached protocol stable allocation is also optimal

34. Strategic Resource Owners • Resource owners who set too high a bid • Attract more agents but pay more and lose utility • Resource owners who set too low a bid • Pay less, but lose agents to competing resources • who offer higher bids • When the domain is competitive for resource owners, such a manipulation is irrational • Highly competitive settings • Condition that occurs mostly in environments where there are many resources with similar marginal production values • Similar resources or slight changes in marginal production

35. Strategic Resource Owners • In our specific domain • Diminishing marginal return • Highly competitive for resource owners • If a resource owner gained from strategic behavior, we still reach an optimal allocation • If a single resource owner has deviated from the suggested strategy and gained utility • Gained utility: a protocol stable allocation has been reached, in which the resource owner gets a higher utility • Then the reached protocol stable allocation is optimal

36. Convergence Time • When agents and resource owners behave rationally, we converge to an optimal allocation • But how quickly is the optimal allocation reached? • Under the suggested strategies • Expected time to convergence: • Bound on convergence time: • Quick polynomial convergence

37. Related Work • TFG-MARA survey • Y. Chevaleyre, P. E. Dunne, U. Endriss, J. Lang, M. Lemaître, N. Maudet, J. Padget, S. Phelps, J. A. Rodríguez-Aguilar, and P. Sousa. Issues in Multiagent Resource Allocation. • Distributed mechanism design approaches • J. Feigenbaum and S. Shenker. Distributed algorithmic mechanism design: Recent results and future directions. • Scheduling domains • B. Heydenreich, R. Muller, and M. Uetz. Decentralization and mechanism design for online machine scheduling. • Negotiations over resources • U. Endriss, N. Maudet, F. Sadri, and F. Toni. Negotiating socially optimal allocations of resources. • T. W. Sandholm. Contract types for satisficing task allocation.

38. Conclusions • A distributed approach to resource allocation in a specific domain • Achieves optimal allocation (maximal social welfare) • No central authority required • All utility divided among agents and resource owners • “Strongly budget balanced” • Quick convergence • Can a similar approach be applied to other domains (or more general domains)?