Unveiling Contextual Influences on Emotions: The Kuleshov Effect Neurobiological Study

1. Dean Mobbs PhD MRC-Cognition and Brain Sciences Unit Social Neuroscience Approach to Social Cognition May 22-25, 2008 Ghent

2. Overview • The Social Brain – Two Core Areas?. • TP, Context and Faces Perception • TP anatomy • TP in Social Cognition and ToM • Working Model of the Social Brain • Some Future Directions • Conclusions

3. Neural Systems in Social Cognition • Core Areas: • Medial prefrontal cortex (mPFC) = interpersonal norms and scripts. ToM. • Temporo-parietal junction (TPJ) = Goals intentions and desires of others (?). • What about the temporal pole!! • Right Temporal Pole (TP) = Special role in ToM, social schemas and concepts?? • - drop-out, ROI “hypothesis driven” approach and lack of tasks/theory about the TP . Van Overwalle, 2008

4. Context in Social Judgements: The Kuleshov Effect • Soviet filmmaker Lev Kuleshov demonstrated that the manipulation of context can alter an audiences’ perceptions of an actor’s facial expressions, thoughts and feelings. • Juxtaposing identical archived clips of actor Ivan Mozzhukhin’s face with either a scene of a funeral or a child playing led the audience to infer Mozzhukhin’s emotional disposition as subtly melancholic or elative, respectively

5. Empirical studies of the Kuleshov effect • Although Kuleshov’s observations were of a anecdotal nature, subsequent research has supported impact of contextual framing by showing that an observer can be influenced to perceive. • Neutral faces as happy or sad (Wallbott, 1988). • Angry facial expressions as fearful (Carroll and Russell, 1996). • Screams as joyful (Goldberg, 1951).

6. We adapted the “Kuleshov Effect” paradigm to elucidate the neurobiological basis of contextual influences on emotional attributions • Methods • Volunteers were explicitly rated emotional expression and mental-state (i.e. what the actor is thinking and feeling) from identical faces juxtaposed with negative, neutral and positively-valanced contexts Mobbs et al. SCAN 2006

7. Context is most effective when a facial expression is source clarity of the face is low and source clarity of the context is high*, we also used neutral faces and faces displaying subtly fearful and happy facial expressions Bottom-up Morphs Top-down Bottom-up 100% Happy 100% Fear Ambiguous *Ekman, Friesen and Ellsworth, (1982)

8. Methods… • Pseudo-candid photomanipulation • To emphasize a link between the actors’ faces and the contextual movie and reduce demand characteristics we used a pseudo-candid photomanipulation with subjects being led to believe that the actors’ expressions were in response to viewing a juxtaposed movie *Ekman, Friesen and Ellsworth, (1982)

9. Task Example Short Movie Response Rate Mobbs et al. SCAN 2006

10. Results: Main Effects Mobbs et al. SCAN 2006

11. Results: Simple Interactions Mobbs et al. SCAN 2006

12. fMRI Results: Parametric Analysis Regions associated with increased + attribution Regions associated with increased - attribution O’Doherty et al. NN (2001) Mobbs et al. SCAN 2006

13. Conclusion • Contextual framing is likely to rely on activation of stored knowledge derived from real-world experiences. • It has been theorized that the right temporal pole serves as a repository for contextual schema or frames. • Involved in storage of contextual information, particularly when of emotional significance. • Connections with the distinct regions of the PFC may result in retrieval of contextual and episodic information (Badre et al 2005; Dobbins et al 2002).

14. Temporal Pole • Right TP - binds together diverse aspects of nonverbal social-emotional communication (prosody, gesture, facial expression, autonomic responses to social stimuli, social context, etc) that must be linked for us to detect sarcasm, irritation, sadness, etc. in others. • Left TP binds processing streams related to word and object meaning (shape, colour, size, texture, and stored word representations must be linked for us to recognize and name - William Seeley – personal communication).

15. Temporal Pole and Ventral-Stream STS FFA Top-down retrieval of social and emotional information Ventral Stream Identity – structural components of the Face

16. Temporal Pole Connectivity mPFC STS FFA Obfc Hypothalamus Olson, Plozker and Ezzyat, 2007

17. Left Temporal Pole in Face Processing? Rothstein et al Nature Neuroscience, 2005 Impaired familiarity with preserved recollection after anterior temporal-lobe resection that spares the hippocampus (Bowles et al. 2007)

18. Right Temporal Pole in Face Processing? Studies on primates have shown the TP to contain faces and eye direction responsive cells Calder et al, Current Biology, 2007 Kriegeskorte et al, PNAS, 2007

19. Role for the Temporal Pole in Social Cognition? • Social attributions are likely to rely on activation of stored contextual schemata derived from experience (Levanthal and Scherer, 1987; Bar, 2004). • Acquired lesions in the vicinity of the right TP can result in the loss of recognition of famous scenes, loss of memory for events and loss of person related knowledge (Tranel et al., 1997; Kitchener et al., 1999; Gorno-Tempini et al., 2004). • Patients with right, but not left, TP atrophy due to the tv-FTD exhibit changes in personality and socially appropriate behaviour (Thompson et al., 2003). • Where are social/cultural conventions stored?

20. Temporal Pole in Social Conceptualization Social Concepts (honor–brave) minus animal concepts (nutritious–useful) Positive (honour–brave) – negative (tactless–impolite) social concepts Zahn et al. PNAS 2007

21. TP rCBF change associated with the perception of ToM animations vs Random animations (Castelli et al 2000)

22. Temporal Pole and ToM Olson, Plozker and Ezzyat, 2007

23. Simulation Theory? • What would I do or expect someone to do in that situation? • Another interpretation is that the TP has some role in encoding personal memories • (Nakamura and Kubota, 1995). • Theory of mind tasks = TP utilizes personal memories to comprehend the state of mind of others (Moriguchi et al., 2006). • Strongest correlation with appropriateness was the TP

24. Evidence of anterior temporal atrophy in college-level soccer players. (Adams et al. Clin J Sport Med. 2007) http://www.zidaneheadbut.co.uk/

25. Right Temporal Pole Social Concepts People Knowledge Emotional episodic memory Social attention Congruency Social Schemas Appropriate/expected social behaviour ToM

26. Working Model of Social Cognition Low-level Perception of Socially Relevant Information and Attention High-level Perception of Social Information Low level social signals (e.g. FFA =faces, EBA Bodies etc R TPJ= Goals intentions and desires of others, ToM STS= biological motion, Social attention, eye direction, intentions mPFC = interpersonal norms and scripts. ToM Stored knowledge/cache R TP = Socioemotional scripts. ToM L TP = Familiarity – History Scripts (in group/out group) Low High

27. Working Model of Social Cognition Low-level Perception of Socially Relevant Information and Attention High-level Perception of Social Information Low level social signals (e.g. FFA =faces, EBA Bodies etc R TPJ= Goals intentions and desires of others, ToM STS= biological motion, Social attention, eye direction, intentions mPFC = interpersonal norms and scripts. ToM Stored knowledge/cache R TP = Socioemotional scripts. ToM L TP = Familiarity – History Scripts (in group/out group) Low High

28. Future Research • Dissociate regions using fMRI and lesion studies • Temporal order using MEG • Alternative approach to functional localizers and ROI focused studies • Developmental studies of social cognition • Novel methods to evoke ecologically-valid studies of social cognition • Form a more elaborate model of the social brain

29. Acknowledgements Chris Frith Ray Dolan Andy Calder Tim Dalgleish Bill Seeley And colleagues at the FIL and CBU Social and Affective Neuroscience Group

30. Questions?