Emotions

1. Emotions • Many thinkers disagree on answers to these questions: • Is surprise an emotion? • (Some people say “always”, others say “only in certain cases”.) • If you love your country, is that an emotion, or an attitude? • What if your love for your country is far from your present thoughts? • Can you have an emotion without being aware of it? • E.g. jealousy, infatuation • Does an emotion have to have some externally observable/measurable physiological manifestation? • Can a fly feel pain, or have emotions? • Is there a stage at which a human fetus becomes able to have emotions? • E.g. able to worry about how the birth will go? • Could a disembodied mathematician have emotions? • E.g. feel disappointment at finding a flaw in a proof? • There is no consensus about what emotions are

2. Prejudices That Confuse the Investigation • Some people assume: • That minds must be “embodied” and emotions must involve bodies • Ignoring the possibility of passionate disembodied mathematicians. • That emotions are required for intelligence. • That emotions evolved because they are useful • Maybe some did. It does not follow that all are useful. Many may be side-effects of other useful mechanisms. • That you can’t have an emotion without being conscious of it. • What about the person who is obviously infatuated or jealous but unaware of the fact? • That it would be a “bad thing” if robots could have emotions. • Could they possibly do worse things than humans do to other humans?

3. A Tip for Doing Conceptual Analysis • If someone puts forward a definition of mental state or process X, the question: • “Does this also apply to X in flies, rats, chimpanzees, newborn infants?” • often reveals that the definition was not based on sufficient thought.

4. Affect – Webster’s Collegiate Dictionary Noun: The conscious subjective aspect of an emotion considered apart from bodily changes. Verb: Applies to the acting of a stimulus strong enough to produce a noticeable response or reaction or modification Synonyms: INFLUENCE, TOUCH, IMPRESS, SWAY

5. Emotions are a Subclass of “Affective” States • Affective states are of many kinds. They include not only what we ordinarily call emotions but also states involving desires, pleasures, pains, goals, values, ideals, attitudes, preferences, and moods. • The general notion of “affective state” is very hard to define but very roughly it involves using some kind of information that is compared (explicitly or implicitly) against what is happening, sensed either internally or externally. • –When there’s a discrepancy some action is taken, or tends to be taken to remove the discrepancy by acting on the sensed thing: affective states involve a disposition to change reality in some way to reduce a mismatch. • – If the information is part of a percept or a belief, then detecting a discrepancy tends to produce a change in the stored “reference” information.

6. Emotions are a Subclass of “Affective” States There is a more primitive type of control state which does not use any sort of description or representation that can be compared with reality, but merely generates action, or has a disposition to produce action (including resisting change). Many innate behaviors are like that. Are these “affective” states? There’s no right definition of such a vague notion. Emotions (from Latin emovere meaning moving, displacing) are, as its etymology suggests, manifest reactions to those affective conditions that, due to their intensity, move us to some kind of action.

7. Affective Computing Group – MIThttp://affect.media.mit.edu/ “Affective Computing is computing that relates to, arises from, or deliberately influences emotions. Emotion is fundamental to human experience, influencing cognition, perception, and everyday tasks such as learning, communication, and even rational decision-making. However, technologists have largely ignored emotion and created an often frustrating experience for people, in part because affect has been misunderstood and hard to measure. Our research develops new technologies and theories that advance basic understanding of affect and its role in human experience. We aim to restore a proper balance between emotion and cognition in the design of technologies for addressing human needs.”

8. Definition – Whatis?comhttp://whatis.techtarget.com/ Affective computing is human-computer interaction in which a device has the ability to detect and appropriately respond to its user's emotions and other stimuli. A computing device with this capacity could gather cues to user emotion from a variety of sources. Facial expressions, posture, gestures, speech, the force or rhythm of key strokes and the temperature changes of the hand on a mouse can all signify changes in the user's emotional state, and these can all be detected and interpreted by a computer. A built-in camera captures images of the user and algorithms are used to process the data to yield meaningful information. speech recognition and gesture recognition are among the other technologies being explored for affective computing applications.

9. A Control-basedConception of Emotion • What is there in common between • – a crawling woodlouse that rapidly curls up if suddenly tapped with a pencil, • – a fly on the table that rapidly flies off when a swatter approaches, • – a fox squealing and struggling to escape from the trap that has clamped its leg, • – a child suddenly terrified by a large object rushing towards it, • – a person who is startled by a moving shadow when walking in a dark • passageway, • – a rejected lover unable to put the humiliation out of mind • – a mathematician upset on realizing that a proof of a hard theorem is fallacious, • – a grieving parent, suddenly remembering the lost child while in the middle of • some important task?

10. A Control-basedConception of Emotion Proposed Answer: In all cases there are at least two sub-systems at work in the organism, and one of them, a specialized sub-system, somehow interrupts or suppresses or changes the behavior of others, producing some alteration in (relatively) global (internal or external) behavior of the system.

11. A Control-basedConception of Emotion WE CAN MAKE IT MORE PRECISE BY: – spelling out different kinds of information-processing control architectures in which such things (e.g. global interrupts or modulations of processing) can occur – showing how different varieties of states with these general features can arise in different architectures. Different sorts of emotions (and other affective states) arise out of different sorts of: – Interacting sub-systems –Ways one can interrupt or modulate another – Functional roles and side-effects

12. Affect and Architecture • As with emotions, which affective states are possible in an organism or machine will depend on the information-processing architecture of the whole system. • We’ll consider three architectural layers: • reactive • deliberative • meta-management • and the different sorts of emotions that can be associated with them.

13. States and Processes • There are at least three different classes of mental phenomena • commonly referred to as “emotions”. • Primary emotions • Evolutionarily oldest – depend only on reactive mechanisms • Secondary emotions • Deliberative mechanisms generating these evolved later • Tertiary emotions • Newest and rarest: involve disruption of meta-management, e.g. loss of control of attention. These are usually not distinguished from secondary emotions • These rather vaguely defined categories must be re-defined in terms of the information-processing architectures (virtual machine architectures) that make them possible. • E.g., an animal without deliberative mechanisms cannot have secondary emotions.

14. Primary Emotions • Examples of primary emotions familiar in humans • Being startled by a loud noise • Being frozen in terror as boulder crashes towards you, • Being nauseated by a horrible smell • In primary emotions, sensor states and/or internal reactive states trigger a fast • but stupid reactive “alarm” mechanism that produces global changes in motors • and internal reactive states. • – Simple versions occur even in insects: when flee, fight, feed, freeze, or mate responses override other processes. (The five Fs!) • – In humans these primary emotions often have sophisticated accompaniments that cannot occur in most other animals capable of having primary emotions. • E.g. when we are aware of having them we are using meta-management mechanisms that are not needed for primary emotions. • Often the primary emotion will immediately trigger some other kind, e.g. apprehension, a secondary or tertiary emotion.

15. Secondary Emotions • Examples — You are: • Afraid the bridge you are crossing may give way • Relieved that you got to the far side safely • Afraid the bridge your child is crossing may give way • Worried about what to say during your interview • Undecided whether to cancel your holiday in ... • Enjoying the prospect of success in your endeavor • Secondary emotions are triggered by events in a deliberative sub-system. Some of • these are triggered by thinking about what might happen, what might have • happened, what did not happen, etc., unlike primary emotions which are triggered only by actual occurrences. • So secondary emotions require deliberative capabilities with ‘what if’, i.e. • counterfactual, representational and reasoning capabilities. These are very subtle • and complex requirements.

16. Counterfactual Reasoning Counterfactual: statement about the consequences of things which happen to be false. In a case where proposition P is contrary to fact, counterfactual reasoning involves drawing inferences about what P would imply if it were true. Because, by definition, a counterfactual is a conditional in which the antecedent is false, logically speaking all counterfactuals are true statements. This has led some philosophers to argue that counterfactuals are not in fact cases of material implication. This, in turn, has prompted the construction of formalisms with which to represent differences between this type of hypothetical reasoning and material implication.

17. Tertiary Emotions • Examples — You are: • Infatuated with someone you met recently • Overwhelmed with grief • Riddled with guilt about betraying a friend • Full of excited anticipation of a loved one’s return • Full of longing for your mother • Basking in a warm glow of pride after winning an election • Obsessed with jealousy about a colleague’s success • These involve disruption of high level self monitoring and control mechanisms. I.e., there is loss of control of thought processes. Thus they cannot occur in animals and machines that are incapable of having such control. • An architecture including meta-management capabilities is required for tertiary emotions.

18. Emotions and Architectures • In all of the categories (primary, secondary, tertiary emotions) there • is a subsystem that produces some relatively global changes in the rest of the system, or in much of it. • They differ in • – What kind of subsystem does the disrupting • – Where the information comes from that triggers the disrupting (e.g. does it come from a deliberative layer, or only sensors and internal states of a reactive layer?) • – Which parts of the system are disrupted, e.g. is there externally visible behavior or only internal disruption? Which internal parts? • – Also there are differences in kind of semantic content, time scale, what can and cannot suppress the disruption, whether learning is involved, etc.

19. Classes Not Mutually Exclusive • All three kinds of emotional processes can coexist in complex situations. • As a result of this, the emotions labeled in ordinary language, e.g. “fear”, “anger”, “relief”, “distress”, cannot simply be classified as primary, secondary or tertiary. Often they are a mixture. • People involved in long and tiring adventure trips often describe multiple emotions at the end. E.g. they may be simultaneously: • Glad to have succeeded in their aims • Regretful at not having done better • Sad that the trip is over • Relieved that some threat did not materialize (e.g. running out of fuel) • Glad to see their families again • Hoping to be selected for their national team • Desperately longing for a good meal • Worried about an injury incurred on the trip

20. Architectural Underpinnings • Different architectural underpinnings are required for different categories of emotions. • Primary emotions: • Require sensors linked to fast reactive mechanisms that can sometimes trigger rapid global signal patterns sent to motors and other sub-systems. • Secondary emotions (central and peripheral): • Require signals from deliberative mechanisms to fast reactive mechanisms that can, under certain conditions, trigger rapid global reactions. • Tertiary emotions (with and without peripheral effects): • Presuppose self-monitoring self-controlling meta-management systems that can be disrupted or modulated by other sub-processes.

21. Further Refinements • Finer emotional distinctions can be made when we understand the underlying architecture better. • E.g. • – “Purely central” vs “partly peripheral” secondary emotions. • – Second-order emotions (being ashamed of feeling jealous). • – Deliberately induced emotions (teacher who – reluctantly – allows himself to get angry to achieve control of a difficult class) • – Emotions that involve constant activity (plotting, fretting, fuming, ranting). • – Emotions that vary in intensity over time. • – Long term, mostly dormant, emotions, e.g. jealousy, grief. (Often ignored.)

22. Important Human Emotions • Socially important human emotions involve rich concepts and knowledge and high level control mechanisms (architectures). • Example: longing for someone or something: • Semantics: • To long for something you need to know of its existence, its remoteness, and the possibility of being together again. • Control: • One who has deep longing for X does not merely occasionally think it would be wonderful to be with X. In deep longing, thoughts are often uncontrollably drawn to X. Moreover, such longing may impact on various kinds of high level decision making as well as the focus of attention. • Physiological processes (outside the brain) may or may not be involved.

23. Summary So Far 1. We can reduce conceptual muddles regarding emotion by trying to use architecture-based concepts. 2. Different architectures are relevant in different contexts (e.g. infants, adults, other animals). So we need to explore different families of concepts (e.g. for describing infants, chimps, cats, people with brain damage). 3. Finding out which architectures are relevant is a hard research problem. One suggestion is that humans have three architectural layers that manifest themselves not only centrally but also in perception and action sub-systems. Most other animals have only a subset. 4. At least three (and several more if we look closely) classes of affective states and processes can be distinguished, related to different architectural layers. 5. Many other concepts (e.g. “learning”, “belief”, “motivation”, “intentional action”) can be refined on the basis of hypothesised architectures.

24. What Use are Emotions • • In 1994 Antonio Damasio, a well known neuroscientist, published his book Descartes’ Error. • He argued that emotions are needed for intelligence, and accused Descartes and many others of not grasping that. • • In 1996 Daniel Goleman published Emotional Intelligence: Why It Can Matter More than IQ, quoting Damasio with approval. • • Likewise Rosalind Picard a year later in her book Affective Computing. • • Since then there has been a flood of publications and projects echoing • Damasio’s claim, and many researchers in Artificial Intelligence have become convinced that emotions are essential for intelligence, so they are now producing many computer models containing a module called ‘Emotion’. • • Before that, serious researchers had begun to argue that the study of emotions and affect had not had its rightful place in psychology, and cognitive science, but the claims were more moderate. • E.g. a journal called Cognition and Emotion was started in 1987.

25. What Use are Emotions • Damasio’s argument rested heavily on two examples: • • Phineas Gage: In 1848, an accidental explosion of a charge he had set blew his tamping iron through his head – destroying the left frontal part of his brain. • “He lived, but having previously been a capable and efficient foreman, one with a well-balanced mind, and who was looked on as a shrewd smart business man, he was now fitful, irreverent, and grossly profane, showing little deference for his fellows. He was also impatient and obstinate, yet capricious and vacillating, unable to settle on any of the plans he devised for future action. His friends said he was No longer Gage.” • http://www.deakin.edu.au/hbs/GAGEPAGE/Pgstory.htm

26. What Use are Emotions • • Elliot, Damasio’s patient (‘Elliot’ was not his real name.) Following a brain tumor and subsequent operation, Elliot suffered damage in • the same general brain area as Gage (left frontal lobe). • Like Gage, he experienced a great change in personality. Elliot had been a successful family man, and successful in business. After his operation he became impulsive and lacking in self-discipline. He could not decide between options where making the decision was important but both options were equally good. He perseverated on unimportant tasks while failing to recognize priorities. He had lost all his business acumen and ended up impoverished, even losing his wife and family. He could no longer hold a steady job. Yet he did well on standard IQ tests. • http://serendip.brynmawr.edu/bb/damasio/

27. What Follows? Both patients appeared to retain high intelligence as measured by standard tests, but not as measured by their ability to behave sensibly. Both had also lost certain kinds of emotional reactions.

28. Damasio’s Argument • In a nutshell, here is the argument Damasio produced which many people in many academic disciplines enthusiastically accepted as valid: • There are two factual premises from which a conclusion is drawn. • P1 Damage to frontal lobes impairs emotional capabilities • P2 Damage to frontal lobes impairs intelligence • C Emotions are required for intelligence • Is this a valid argument? • One can argue that the conclusion does not follow from the premises. • Whether the conclusion is true is a separate matter.

29. Compare This Argument • We ‘prove’ that cars need functioning horns in order to start, using two premises on which to base the conclusion: • P1 Damaging the battery stops the horn working in a car • P2 Damage to the battery prevents the car starting • C A functioning horn is required for the car to start • Does C follow from P1 and P2?

30. A Critical Observation • The following should occur to the thoughtful listener: • • two capabilities A and B could presuppose some common mechanism M, so that • • damaging M would damage both A and B • • without either of A or B being required for the other. • For instance, even if P1 and P2 are both true, you can damage the starter motor and leave the horn working, or damage the horn and leave the starter motor working!

31. Why Buy Damasio’s Argument A possible explanation for the surprising fact that so many intelligent people so easily accept what appears to be an invalid argument is sociological: they are part of a culture in which people want the conclusion to be true. There seems to be a wide-spread (though not universal) feeling, even among many scientists and philosophers, that intelligence, rationality, critical analysis, problem-solving powers, are over-valued, and that they have defects that can be overcome by emotional mechanisms. This leads people to like Damasio’s conclusion. They want it to be true. And this somehow causes them to accept as valid an argument for that conclusion, even though they would notice the flaw in a structurally similar argument for a different conclusion (e.g. the car horn example).

32. On the Other Hand ... In fact Damasio produced additional theoretical explanations of what is going on, so, in principle, even though the quoted argument is invalid, the conclusion might turn out to be true and explained by his theories.

33. Expressive Interfaces User Frustration Anthropomorphism in Interaction Synthetic Characters (Agents) Emotions and Interfaces

34. functionality and performance aspects are not always sufficient to make users comfortable expressive interfaces can convey additional information to the user affective computing has to be used with care, it gets annoying very easily Motivation

35. be aware of the effects that the use of expressive computing methods can have in user interaction identify appropriate techniques that enhance the user’s comfort level with the system balance the quality, quantity, and expressiveness of feedback from the system to the user identify and avoid common mistakes that can lead to user frustration Objectives

36. How Interfaces Affect Users • Expressive interfaces • how the ‘appearance’ of an interface can elicit positive responses • Negative aspects • how computers frustrate users • Anthropomorphism and interface agents • The pros and cons • Designing synthetic characters

37. HCI has generally been about designing efficient and effective systems Recently, move towards considering how to design interactive systems to make people respond in certain ways e.g. to be happy, to be trusting, to learn, to be motivated Affective Aspects

38. Color, icons, sounds, graphical elements and animations are used to make the ‘look and feel’ of an interface appealing Conveys an emotional state In turn this can affect the usability of an interface People are prepared to put up with certain aspects of an interface (e.g. slow download rate) if the end result is very appealing and aesthetic Expressive Interfaces

39. Microsoft pioneered friendly interfaces for technophobes - ‘At home with Bob’ software Friendly Interfaces

40. Microsoft Bob was designed to be a user friendly interface for Microsoft Windows version 3.1. Bob included various office suite programs such as a calendar, a finance application, and a word processor. The user interface was designed to be helpful to novice computer users, but many saw its methods of assistance as too cute and involved. Each action, such as creating a new text document, featured the step-by-step tutorials no matter how many times the user had been through the process. Users were assisted by cartoon characters whose appearance was usually vaguely related to the task. These characteristics earned Bob the 7th place in PC World (March 26, 2006) Magazine's list of the 25 worst products of all time. Microsoft Bob

41. Bob featured a diverse selection of "guides," representing an early form of the Office Assistant and Microsoft Agent technologies. Each guide possessed its own unique "personality" and had its own array of animations. Microsoft Bob

42. Rover, the yellow dog Blythe, a firefly from New York City Chaos, a fluffy cat from France Digger, an Irish earthworm Hopper, a blue stuffed bunny rabbit Java, a coffee-drinking dragon from Guatemala Ruby, a parrot Shelly, a turtle that wears a backpack and carries a walking stick Scuzz, a guitar-playing rat Speaker, a black computer speaker Will, a cartoon of William Shakespeare The Dot, a red ball Baudelaire, a gargoyle from a gothic cathedral Bob’s Buddies

43. 3D metaphors based on familiar places (e.g. living rooms) Agents in the guise of pets (e.g. bunny, dog) were included to talk to the user Make users feel more at ease and comfortable Friendly Interfaces

44. Users have created emoticons - compensate for lack of expressiveness in text communication: Happy :) Sad :< Sick :X Mad >: Very angry >:-( Also use of icons and shorthand in text and instant messaging has emotional connotations, e.g. I 12 CU 2NITE User-created Expressiveness

45. rational aspect the system does not do what the user wants it to do tolerable if it is (more or less) expected emotional aspect the user is upset about this mismatch with the mental model the user has disregard of the user’s perspective by the designer may affect the system as a whole even if only a small part causes the frustration may be difficult to recover from User Frustration

46. identify an interaction with a computer system that really frustrates you what are the causes for your frustration: inability to do something unexpected outcome “the idiots who wrote the program” try to contrast this with an example where the system does not work for you, but it is not frustrating for you Exercise: My Most Frustrating Computer Interaction

47. Many causes: When an application doesn’t work properly or crashes When a system doesn’t do what the user wants it to do When a user’s expectations are not met When a system does not provide sufficient information to enable the user to know what to do When error messages pop up that are vague, obtuse or condemning When the appearance of an interface is garish, noisy, gimmicky or patronizing Causes for User frustration

48. “The application Word Wonder has unexpectedly quit due to a type 2 error.” Why not instead: “the application has expectedly quit due to poor coding in the operating system” Shneiderman’s guidelines for error messages include: avoid using terms like FATAL, INVALID, BAD Audio warnings Avoid UPPERCASE and long code numbers Messages should be precise rather than vague Provide context-sensitive help Error Messages

49. Website Error Message

50. “The requested page /helpme is not available on the web server. If you followed a link or bookmark to get to this page, please let us know, so that we can fix the problem. Please include the URL of the referring page as well as the URL of the missing page. Otherwise check that you have typed the address of the web page correctly. The Web site you seekCannot be located, butCountless more exist.” More Helpful Error Message