SIF8072 Distributed Artificial Intelligence and Intelligent Agents

1. Lecture 3: Negotiation Reaching Agreements SIF8072 Distributed Artificial IntelligenceandIntelligent Agents http://www.idi.ntnu.no/~agent/ 30 January 2003 Lecturer: Sobah Abbas Petersen Email: sap@idi.ntnu.no

2. Lecture Outline • Recap Auctions • Negotiation • Mechanisms, protocols, strategies • Negotiation domains (task-oriented and worth-oriented) • Monotonic Concession Protocol • Zeuthen Strategy • Coordination – The Contract Net Protocol

3. References • Curriculum: Wooldridge: ”Introduction to MAS”, • Negotiation: Chapter 7 • Coordination, Contract Net Protocol: Chapter 9 • Recommended Reading (Not curriculum): • J. Rosenschein and G. Zlotskin, Rules of Encounter, MIT Press, 1994, ISBN 9 780262 181594, • Chapters 1, 2 and 3 • R. Davis and R. G. Smith, Negotiation as a Metaphor for Distributed Problem Solving, (A. H. Bond and L. Gasser eds.) Readings in Distributed Artificial Intelligence, Morgan Kaufmann Publishers, 1988, p. 333-356.

4. Negotiation • ”The process of several agents searching for an agreement”e.g. about price. • Reaching consensus ”Rules of Encouter” by Rosenchein and Zlotskin, 1994

5. auctioneer bidders Price bidder auctioneer Recap - Auctions • An Auction takes place between an auctioneer and a collection of bidders. • In most settings, the auctioneer desires to maximise the price; bidders desire to minimise the price. • Types of Auctions: • English auction • Dutch auction • First-price sealed bids • Vickrey (Second-price sealed bids) • Useful for allocating goods. But too simple for many other settings.

6. Recap - Limitations of Auctions • Only concerned with the allocation of goods; • Not adequate for settling agreements that concerns matters of mutual interest. • Negotiation

7. Negotiation Example 1 Consider two companies A and B decide to form an alliance to develop a software product for the shipbuilding industry. Company A has the shipbuilding market and knowledge in that domain. Company B is a software company that specialises in CAD systems and CAE. They negotiate to agree upon the financial assets, the skills and technology contributed by each company, how much of the new product each company will own and who will market the product and provide first-line support for the product.

8. Negotiation Example 2 Consider two agents, each controlling a telecommunications network with associated resources such as communication lines, microwave links, routing computers, short and long-term storage devices. The load that each agent has to handle varies over time. The agents negotiate about resource sharing.

9. Negotiation • May involve: • Exchange of information • Relaxation of initial goals • Mutual concession

10. Mechanisms, Protocols, Strategies • Negotiation is governed by a mechanism or a protocol: • defines the ”rules of encounter” between the agents • the public rules by which the agents will come to agreements. • Given a particular protocol, how can a particular strategy be designed that individual agents can use?

11. Mechanisms Design • Mechanism design is the design of protocols for governing multi-agent interactions. • Desirable properties of mechanisms are: • Convergence/guaranteed success • Maximising social welfare • Pareto efficiency • Individual rationality • Stability • Simplicity • Distribution

12. Negotiation Components • Any negotiation setting will have 4 components: • Negotiation set: represents the space of possible proposals that agents can make • Protocol: defines the legal proposals that agents can make • Collection of strategies: (one for each agent) determines what proposals the agent will make • Rule: to determine when an agreement has been reached

13. Proposal Counter Proposal Agenti concedes Agenti Agentj Negotiation Process 1 • Negotiation usually proceeds in a series of rounds, with every agent making a proposal at every round. • Communication during negotiation:

14. Point of Acceptance/ aggreement Proposals by Aj Proposals by Ai Negotiation Process 2 • Another way of looking at the negotiation process is:

15. Complex Negotiations • Some attributes that make the negotiation process complex are: • Multiple attributes: • Single attribute (price) – symmetric scenario. • Multiple attributes – several inter-related attributes, e.g. buying a car. • The number of agents and the way they interact: • One-to-one, e.g. single buyer and single seller . • Many-to-one, e.g. multiple buyers and a single seller, auctions. • Many-to-many, e.g. multiple buyers and multiple sellers.

16. Negotiation Domains: Task-oriented • ”Domains in which an agent’s activity can be defined in terms of a set of tasks that it has to achieve”, (Rosenschein & Zlotkin, 1994) • An agent can carry out the tasks without interference from other agents • All resources are available to the agent • Tasks redistributed for the benefit of all agents

17. Task-oriented Domain: Example Imagine you have 3 children, each of whom needs to be delivered to 3 different schools each morning. Your neighbour has 4 children who also need to be taken to school. Delivery of each child is a task. Assume that one of your children and one of your neighbour’s children both go to the same school. It obviously makes sense for both children to be taken together and only you or your neighbour needs to make the trip.

18. Task-oriented Domain: Definition • Can be defined as a triple: • T,Ag,c • T: finite set of all possible tasks • Ag: set of negotiating agents • C: cost of executing each subset of tasks • How can an agent evaluate the utility of a specific deal? • Utility represents how much an agent has to gain from the deal. • Since an agent can achieve the goal on its own, it can compare the cost of achieving the goal on its own to the cost of its part of the deal. • If utility<0, it is worse off than performing tasks on its own.

19. Deals in Task-oriented Domains • Conflict deal: if agents fail to reach an agreement: • where no agent agrees to execute tasks other than its own. • utility = 0 • A deal that is not dominated by any other deal is pareto optimal. • A deal is individual rational if it weakly dominates the conflict deal.

20. Negotiation Set in Task-oriented Domains Utility for agent i Negotiation set: (pareto optimal+ Individual rational) B A C Utility of conflict Deal for agent i The circle delimits the space of all possible deals E Conflict deal D Utility for agent j Utility of conflict Deal for agent j

21. Let’s take a minute…… • Give an example in the Task-oriented domain? • Discuss with your neighbour(s) about this domain and how you can agree upon sharing the tasks.

22. Ai best deal Aj best deal The Monotonic Concession Protocol (MCP) 1 • Negotiation proceeds in rounds • On round 1, agents simultaneously propose a deal from the negotiation set. • Agreement is reached if one agent finds that the deal proposed by the other agent is atleast as good or better than its proposal.

23. The Monotonic Concession Protocol 2 • If no agreement is reached, then negotiation proceeds to another round of simultaneous proposals. • In round u+1, no agent is allowed to make a proposal that is less preferred by the other agent than the deal proposed at time u. • If neither agent concedes, then negotiation terminates with a conflict deal.

24. The Monotonic Concession Protocol 3 • Advantages: • Symmetrically distributed (no agent plays a special role) • Ensures convergence • It will not go on indefinitely • Disadvantages: • Agents can run into conflicts • Inefficient – no quarantee that an agreement will be reached quickly

25. Key Questions 3 key questions to be answered: • What should an agent’s first proposal be?It’s most preferred deal. • On any given round, who should concede?The agent least willing to risk conflict. • If an agent concedes, then how much should it concede?Just enough to change the balance of risk.

26. Maximum loss from conflict How much am I willing to riskaconflict? Maximum loss from concession Conflict deal Ai best deal Aj best deal The Risk Factor One way to think about which agent should concede is to consider how much each has to loose by running into conflict at that point.

27. The Zeuthen Strategy • Uses the risk evaluation strategy • Suppose you have conceded a lot. Then: • Your proposal is now close to conflict deal. • You are more willing to risk conflict. • An agent will be more willing to risk conflict if the difference in utility between its current proposal and the conflict deal is low. • Degree of willingness to risk a conflict can be defined as: utility i loses by conceding and accepting j’s offerutility i loses by not conceding and causing conflict Riskti =

28. About MCP and Zeuthen Strategies • Advantages: • Simple and reflects the way human negotiations work. • Stability – in Nash equilibrium – if one agent is using the strategy, then the other can do no better than using it him/herself. • Disadvantages: • Computationally expensive – players need to compute the entire negotiation set. • Communication burden – negotiation process may involve several steps.

29. Negotiation Domains: Worth-oriented • ”Domains where agents assign a worth to each potential state (of the environment), which captures its desirability for the agent”, (Rosenschein & Zlotkin, 1994) • agent’s goal is to bring about the state of the environment with highest value • we assume that the collection of agents have available a set of joint plans – a joint plan is executed by several different agents

30. 100 0 9 12 16 Worth-oriented Domain: Example 2 agents are trying to set up a meeting. The first agent wishes to meet later in the day while the second wishes to meet earlier in the day. Both prefer today to tomorrow. While the first agent assigns highest worth to a meeting at 16:00hrs, s/he also assigns progressively smaller worths to a meeting at 15:00hrs, 14:00hrs…. By showing flexibility and accepting a sub-optimal time, an agent can accept a lower worth which may have other payoffs, (e.g. reduced travel costs). Worth function for first agent Ref: Rosenschein & Zlotkin, 1994

31. Worth-oriented Domain: Definition • Can be defined as a tuple: • E,Ag,J,c • E: set of possible envirinment states • Ag: set of possible agents • J: set of possible joint plans • C: cost of executing the plan

32. Worth-oriented Domains and Multiple Attributes • If you want to pay for some software, then you might consider several attributes of the software such as the price, quality and support – multiple set of attributes. • You may be willing to pay more if the quality is above a given limit, i.e. you can’t get it cheaper without compromising on quality. • Pareto Optimal – Need to find the price for acceptable quality and support (without compromising on some attributes).

33. How can we calculate Utility? • Weighting each attribute • Utility = {Price*60 + quality*15 + support*25} • Rating/ranking each attribute • Price : 60, quality : 20, support : 20 • INSPIRE uses rating • Using constraints on an attribute • Price[5,100], quality[0-10], support[1-5] • Try to find the pareto optimum

34. Utility Agentj Point of acceptance Agenti time No. of negotiations Utility Graphs 1 • Each agent concedes in every round of negotiation • Eventually reach an agreement

35. Utility Agentj Agentj finds offer unacceptable Agenti time No. of negotiations Utility Graphs 2 • No agreement

36. Let’s take a minute…… • Give an example in the Worth-oriented domain? • Discuss with your neighbour(s) the kinds of attributes that will play a role in the negotiation process.

37. Argumentation 1 • The process of attempting to convince others of something. • Why argument-based negotiations: • Limitations of game-theoretic approaches • Positions cannot be justified – Why did the agent pay so much for the car? • Positions cannot be changed – Initially I wanted a car with a sun roof. But I changed preference during the buying process.

38. Argumentation 2 • 4 modes of argument: • Logical - ”If you accept that A and A implies B, then you must accept that B” • Emotional - ”How would you feel if it happened to you?” • Visceral - One argumentation participant stamps their feet and show the strength of their feelings • Kisceral - Appeals to the intuitive

39. 100 0 9 12 16 Negotiation - Summary • Task-oriented domains: • Monotonic Concession Protocol • Could result in a conflict deal • Zeuthen strategy – takes into account the risk of running into a conflict • Risk – distance between current position and conflict deal • Worth-oriented domains • Multiple set of attributes • Pareto optimality • Argumentation Worth function

40. Coordination ”The process by which an agent reasons about its local actions and the (anticipated) actions of others to try and ensure that the community acts in a coherent manner.” Jennings,1996

41. References - Curriculum • Wooldridge: ”Introduction to MAS”, • Chapter 9 • N. R. Jennings. ”Coordination Techniques for Distributed Artificial Intelligence”,in: G. M. P. O'Hare, N. R. Jennings (eds). Foundations of Distributed Artificial Intelligence, John Wiley & Sons, 1996, pp. 187-210.

42. Coordination Example Consider an interaction between two robots, A and B, operating in a warehouse. The robots have been designed by different companies, and they are stacking and unstacking boxes to remove certain goods that have been stored in the building. They need to coordinate their actions to share the work load and to avoid knocking into each other and dropping the boxes.

43. Reasons for Coordination • Preventing anarchy or chaos • Dependencies between agents’ actions. • Need to meet global constraints • No individual has sufficient competence, resources or information to solve the entire problem. • Efficiency

44. Cooperative Distributed Problem Solving (CDPS) CDPS studies how a loosely-coupled network of problem solvers can work together to solve problems that are beyond their individual capabilities. Each problem solving node is capable of sophisticated problem solving and can work independently, but the problems faced by the nodes cannot be solved without cooperation.

45. Benevolent and Self-interested Agents • Important to distinguish between • Benevolent agents • Share the same goal - our best interest is their best interest. • No potential for conflicts • Problem-solving in benevolent systems => CDPS • Self-interested agents • Agents will act to further their own interests, possibly at the expense of others • Potential for conflicts • Self-interested agents => MAS

46. Success Criteria for a MAS • 2 criteria to assess the success of a MAS: • Coherence: How well the (multi-agent) system behaves as a unit. • May be measured in terms of solution quality, resource usage, etc. • Coordination: The degree to which agents can avoid extraneous activity such as synchronisation and alignment of activities. • Indicator – Conflicts between agents where agents destructively interfere with one another.

47. Coordinated System • In a perfectly coordinated system: • Agents may not need to explicitly communicate, they may be mutually predictable. • May maintain good models of each other. • Agents will not accidently hit each other’s subgoals while trying to achieve a common goal.

48. Main issues in CDPS • Problem decompositionfor distribution • Synthesis of sub-problem results to obtain the solution • Optimisation of the problem-solving activities of the agents • Techniques for the coordination of the agents’ activities

49. The 3 Stages of CDPS 1. Problem decomposition 2. Subproblem solution 3. Answer synthesis Ref: Smith & Davis, 1980

50. A1 A2 A3 Task and Result Sharing • Task sharing: • when a problem is decomposed into subproblems and allocated to different agents. • Result sharing: • When agents share information relevant to their subproblems. Task 1 Task 1.1 Task 1.2 Task 1.3