Agents: Pros and Cons

1. Agents: Pros and Cons Keita Fujii Jennifer Rhough

2. Papers • Agents that Reduce Work and Information Overload (P. Maes, p. 525-536) • Presenting Through Performing: On the Use of Multiple Lifelike Characters in Knowledge-Based Presentation Systems (E. André, T. Rist; IUI-2000, pp. 1-8) • Embedding Critics in Design Environments (G. Fischer, 537-561) • Multimodal Interaction for Distributed Interactive Simulation (P. Cohen et al., 562-571) • Animated Conversation: Rule-based Generation of Facial Expression, Gesture and Spoken Intonation for Multiple Conversational Agents (J. Cassell, J.et al., p. 582-591) • Direct manipulation vs. interface agents§§ (B. Shneiderman, P. Maes; Interactions 4, 1997, p. 42-61)

3. These papers focus on agents that: • Support task performance • Perform tasks on behalf of users • Present information • Enable integration of complex software systems • Create interfaces possessing anthropomorphic communicative abilities • integrated speech, facial displays, gesture

4. Agents that Reduce Work and Information Overload • Motivation • We increasingly use computers for our everyday activities • Increasing number of untrained users • Dominant Interaction Metaphor • Direct manipulation vs. Indirect management

5. Building Agents • Two problems to overcome • Competence • How, when and what • Trust • Comfort levels in delegating tasks • Integrate into existing interfaces • Way of operating should be easily understandable

6. “Semi-autonomous Agents” Example: Email sorting agent Consists of a collection of user programmed rules Competence not dealt with Trust Do you trust your own skills? “Knowledge-based approach” Interface agent supplied with extensive domain specific knowledge Competence issues Trust issues Past Approaches

7. Another approach • Hypothesis is that under certain conditions the agent can “program itself” • Two conditions need to be fulfilled • Use of application involves repetitive behavior • This behavior is potentially different for all users

8. Personal Assistant Metaphor • Assists user by: • Hiding complexity of difficult tasks • Performs tasks on user’s behalf • Trains/teaches user • Helps different users collaborate • Monitors events and procedures

9. The learning approach • Requires less work from the end-user and application developer • Is a solution to the trust problem • Allows agents to reason their behavior • Agents can more easily adapt to the user over time and become customized to individual/organizational preferences and habits • Helps in transferring info, habits, and know-how among the different users of a community.

10. How the agent acquires competence

11. Electronic Mail Agent (Maxims) • Learning technique is memory-based learning • Learns to prioritize, delete, forward, sort, and archive mail on behalf of user by “looking over the shoulder” of the user • Agent memorizes generated situation-action pairs • Situations described by features

12. Maxims • Agent will compare new with memorized situations and tries to find a set of nearest neighbors to base its action • distance metric – weighted sum of differences for the features; weight determined by agent • agent analyzes its memory for correlation bet features and actions taken • From vs Date • measures confidence in prediction

13. Maxims • Two user defined thresholds • Do-it • Will take action • Tell me • Will ask and wait confirmation

15. Maxims • Slow start problem • user can instruct agent explicitly • default or hard-and-fast rules, use of “wildcard” fields • Multi-agent collaboration • Confidence is below “tell-me” so ask other agents by sending part of description via email • Learns trustworthy sources • Preliminary user approval • Report feeling comfortable delegating tasks • Users want to be able to instruct agent to disregard behavior

16. Meeting Scheduling Agent • Same software agent as above but attached to a meeting scheduling package • assists user in scheduling of meetings (accept/reject, schedule, reschedule, negotiate times)

17. News Filtering Agent • User creates “news agents” and initialize by giving it +/- examples of articles • User can give feedback on portions of articles recommended • No social filtering • Limitations • Users rely on it too much - still responsible for finding less predictably interesting articles • Restriction to keywords only, no semantic analysis

18. Entertainment Selection Agent • Social filtering • Relies solely on correlations between different users • Problems • Users can rely too much and not enter new information on items discovered themselves • How to jumpstart the system so agents notice correlations • Users can rely too much and not enter new information on items discovered themselves • Virtual Users

19. Some questions to ask • How to guarantee user’s privacy? • How can heterogeneous agents collaborate? • Should the user be held responsible for the agent’s actions?

20. Presenting through Performing: On the Use of Multiple Animated Characters in Knowledge-Based Presentation Systems • Based on observation that vivid and believable dialogues are a means to present information to an audience • Use of animated characters • Ability to express emotions in a believable way • Provide means of conveying conversational signals • Users rate presentations by characters as lively and engaging

21. Rationale • Presentation teams vs. face to face • Easier to convey differing points of view • Debates between two characters • Allows reinforcement • Single members function as indices to help user classify information • Also used to convey meta-information • Some people feel uncomfortable when addressed directly by an agent

22. Related Work • Virtual human-like weather reporter • One agent for presenting information • Bank teller and employee • Restricted to Q&A type dialogue between two agents • Mr. Bengo • Resolutions of disputes with judge, prosecutor, and lawyer (controlled by user) • Exhibits basic emotions but not through linguistic style

23. Designing Presentation Dialogues • Choose dialogue type • Sales dialogue and soccer commentary • Define roles • Sales – seller and buyer • Define characters to occupy roles • Personality and emotional traits • Gestures, linguistic style • Distinguishable by expertise, audio/visual appearance, interests

24. Generation of Dialogue • Actors with scripted behaviors • Actors in a play • Knowledge to be communicated known a priori • Possible to vary dialogue by expressions, gestures, emotions • Autonomous actors • Agents draw from dialogue strategies to meet a certain goal (can be different) • Reactive and difficult to ensure coherence

25. Inhabited Market Place

26. Inhabited Market Place • Scripted • Ordinary product database • each example with n attributes • Attributes also grouped according to the values of the character • safety, economy, comfort, prestige, environmental considerations, etc

27. Central planning component Knowledge is represented by plan operators handle the dialogue and allocation of dialogue agents. NAME: “DiscussValue1” GOAL: PERFORM DiscussValue $attribute; PRECONDITION: FACT polarity $attribute $dimension “neg”; FACT difficulty $attribute $dimension “low”; FACT Buyer $buyer; FACT Negative $buyer; FACT Seller $seller; BODY: PERFORM NegativeResp $buyer $dimension; PERFORM RespNegResp $seller $attribute $dimension; Design of Information Dialogues

28. Generation Example Agent Role Personality factors Interests Robby seller extravert, agreeable sportiness Peedy buyer introvert, disagreeable environment Peedy: How much gas does it consume? Robby: It consumes 8 liters per 100 km. Peedy: Isn’t that bad for the environment? ;;;negative comment because it is disagreeable, less direct speech ;;;because it is introvert Robby: Bad for the environment? It has a catalytic converter. It is made of recyclable material. ;;;questions the negative impacts and provides counter arguments

29. RoboCup Soccer Games • Semi-autonomous agents • triggered by events occurring in the scene & dialogue from other agent • rapidly changing environment • Gerd and Matze • Characterized by sympathy of team, level of extraversion, openness, and two emotional dispositions, excitability, and valence

30. Dialogue Input and Templates • Basic input is obtained by the soccer server • delivers player location and orientation, ball location, score, play modes (goal kicks, throw-ins) • info is pieced together at a more conceptual level to provide material for characters • Templates extracted from 13.5 hours of actual soccer reports and characterized by features like verbosity, bias, formality • Selection of template filtered by • situational needs like time • remove templates that were recently used • keep those that are aligned with character’s attitude • keep those aligned with character’s personality

31. Generation Example Agent Attitude Personality factors Gerd in favor of team Kasunga extravert, open Matze neutral introvert, not open Gerd: Kasunga kicks off ;;;recognized event: kick off Matze: Andhill 5 ;;;recognized event: ball possession, time pressure Gerd: We’re live from an exciting game, team Andhill in red versus Kasunga in yellow ;;;time for background information (…) Gerd: ball hacked away by Kasunga 4 ;;;recognized event: shot, flowery language since it is creative

32. Conclusions • Testing • Users found the scenarios entertaining and amusing • Eager to test the effect of role castings on the generated presentation • Implies people might learn more about a subject matter because they are willing to spend more time with a system • Questions • How to actively involve the user, either as a co-presenter or by providing feedback during performance • Optimal number of roles and casting

33. Embedding Critics in Design Environments • The critiquing approach • Growth of human knowledge • Helps in error elimination • Promotion of mutual understanding of all participants • Computer based critiquing applied to design • Critics recognize and communicate debatable issues • Suited for design tasks where • Knowledge of design domain is incomplete/evolving • Design knowledge is distributed • Problem requirements can only be partially specified

34. Shortcomings that hinder the ability to say the “right” thing at the “right” time • Lack of domain orientation • Insufficient facilities for justifying critic suggestions • Lack of an explicit representation of user’s goals • No support for different perspectives • Timing problems • Passive vs. active critics

35. HYDRA-KITCHEN • Design creation tools • Construction component • Analogous to the Paint program • Includes palette of domain-oriented design units (e.g. sinks, stoves) • Critics are tied to units and relationships between units • Specification component • Allows designers to describe abstract characteristics of their design • Dynamic • Used to tailor critic’s suggestions and explanations

36. HYDRA-KITCHEN II • Design information repositories • Argumentative hypermedia component • Consists of issues, answers, and arguments about decisions made in the design • Identifies pros and cons of a suggestion and helps users to understand consequences of following a suggestion • Catalog component • Collection of previously constructed designs • Can be used by critics as examples illustrating solutions

39. Generic Critics • Enabled by placing design units into the construction area • Reflects knowledge that applies to all designs • Defined through property sheets that specify rules and relations • Users can add and modify

41. Specific critics • Enabled by the partial specification • Fine tune generic critics • Detects inconsistencies between design specification and construction • Situation specific physical characteristics • Size/shape of kitchen, owner’s height • Specified requirements • Abstract domain concepts like safety or efficiency

43. Interpretive critics • Enabled by the currently active design perspective • Examines the design from different viewpoints • Electrician, plumber, city inspector, interior designer • Inheritance network - inherit other critics • Can then add additional rules and modify inherited ones

44. Some advantages • Embedding allows access the work state and time delivery of information that is relevant to the current task • Support for different perspectives • Critic suggestions supported by domain-oriented design environment • Design environment allows explicit representation of user’s goals • Locating relevant information • Large information space • People are lazy or unaware

45. HYDRA-KITCHEN Remarks • Learning on demand • Integrate learning into work • Immediate gratification • Relevant to task • End user modifiability • How to “seed” with domain knowledge • System builders not domain experts