Neuronal Mechanisms in Prefrontal Cortex Underlying Adaptive Choice Behavior

1. Wallis, JD Helen Wills Neuroscience Institute UC, Berkeley Neuronal Mechanisms in Prefrontal Cortex Underlying Adaptive Choice Behavior

2. Background • Role of prefrontal cortex (PFC) in reward-guided choice behavior • 2 Questions: • Does PFC encode reward or behavioral sequelae of reward? • Is encoding specific to reward outcome or reflective of abstract value signal?

3. Background • Orbitofrontal cortex (OFC) is a key region in choice behavior • Has functions in emotions and reward • Thought to regulate planning behavior associated with sensitivity to reward and punishment Paul Wicks’ brain

4. OFC • Damage to the OFC leaves cognitive abilities intact, but impairs ability to make decisions • The cautionary tale of Elliot • OFC neurons encode expected rewards • Neurons show response to delivery of juice rewards predicted by a visual stimulus • Useful for decision making

5. DLPFC • Reward encoding neurons are also found in the dorsolateral PFC (DLPFC) • Neurons showed a difference in firing rate depending on large/small expected reward • However, are these neurons encoding the value of the reward or a behavioral correlate of the reward (tensed muscles, heightened accuracy)?

6. Experiment 1 • Comparison of reward encoding in the DLPFC and the OFC • 2 monkeys choose between pictures associated with small/large fruit juice rewards

7. Experiment 1

8. Methods • Each picture associated with delivery of a certain amount of juice • Subjects learn to maximize reward • Reward-picture contingencies reversed after 27 out of 30 successes • Most important neuronal activity after 2nd picture appears

9. Methods • Activity was recorded simultaneously from multiple electrodes in DLPFC and OFC • 167 DLPFC neurons • 134 OFC neurons

10. OFC Results • Spike density histograms from 2 representative OFC neurons • Activity recorded during predictive cue • One neuron showed higher firing rate when the monkey expected 4 drops of juice • Another encoded the reward in parametric fashion • Firing rate not affected by saccade

11. OFC Results Figure 2

12. DLPFC Results • DLPFC neurons show responses related to both reward and saccade • One neuron discriminated between different amounts of juice only during right saccade • Another encoded reward in parametric fashion (positive?) with a greater increase during left saccade

13. DLPFC Results Figure 3

14. Statistics • 2-way ANOVA on mean firing rate with factors of Reward and Saccade • OFC: • 28% significant main effect of Reward • 19% significant interaction with Saccade • DLPFC • 13% significant main effect of Reward • 43% Reward-Saccade interaction

15. Statistics

16. Statistics • Some neurons in both areas had similar properties: • 27% parametric increase with reward size • 15% parametric decrease with reward size • 59% encode specific reward

17. ROC Analysis • Sliding receiver operating characteristic (ROC) analysis of the selectivity time-course revealed differences in encoding of reward between OFC and DLPFC • Probability that an independent observer could predict reward based on firing rate • Starting from 500ms prior to 2nd picture an ROC curve was generated from 10ms increments

18. ROC Analysis • Latency at which selectivity appeared was computed as the point at which the curve exceeded 0.6 • No difference between OFC (mean 426ms) and DLPFC (mean 467ms) • (t-test = 1, d.f. = 110, P > 0.1)

19. ROC Analysis • Selectivity rose more rapidly and peaked earlier in OFC • 80 ms earlier on average • Short latency indicates OFC is encoding reward’s value rather than correlated behavioral/cognitive processes

20. ROC Analysis

21. Summary • Neurons sensitive to expected reward are found in both the OFC and DLPFC • OFC neurons encoded only reward while DLPFC neurons encoded reward and saccade • OFC neurons encoded reward earlier than DLPFC

22. Summary • Therefore OFC is first prefrontal region receiving reward information • From basolateral amygdala • Encodes immediate reward –Winstanley, CA et al • From gustatory and olfactory cortices

23. Baxter, MG and Murray, EA The amygdala and reward

24. Summary • DLPFC is where reward value converges with subjects actions: reward choice

25. Experiment 2 • OFC response indicates to the motor system which action leads to largest reward • However, decision making needs to be more complex • Reward value is determined by • (Reward – Cost) x P of success • To what extent does OFC encode these variables?

26. Experiment 2 • OFC may integrate all variables relevant to decision making to derive an abstract value signal • Neuronal currency • Another study tested whether PFC neurons were capable to responding to multiple parameters

27. Methods • Monkeys were trained to choose between pictures associated with particular rewards • Recordings from OFC, MPFC, and lateral PFC taken simultaneously

28. Methods • 3 variables were manipulated: • Probability: Some pictures predict fixed amount of juice on only a certain proportion of trials • Reward: Some pictures were associated with varying amounts of juice • Effort: Monkey had to earn fixed amount of juice by pressing a lever multiple times

29. Results • 1/3 of PFC neurons responded parametrically to just 1 parameter • Found in all 3 areas • Some neurons responded to a combination of parameters • Progressive increase from LPFC to OFC to MPFC • 16% -> 27% -> 48%

30. Summary • OFC and MPFC combine multiple variables in order to make a decision by deriving abstract value signals • Too difficult to make direct comparisons between all possible choices • Each choice can be valued against a common reference scale (currency) • Example: how many bananas is your car worth?

31. Summary • Abstract value signals allow flexibility and novelty • Simplifies the task of the motor system • Allows instantaneous choice • Patients with OFC and MPFC damage show unusual patterns of decision making • A > B , B > C but A < C

32. Conclusions • Damage to the OFC impairs decision making while leaving other cognitive abilities intact • OFC is implicated in reward info processing • Must differentiate between reward and behavioral sequelae of reward • Short latency of neuronal reward-related responses indicates encoding of reward’s value in OFC

33. Conclusions • In contrast, DLPFC encodes reward info in relation to behavioral responses • PFC neurons also encode other variables related to decision making, including probability of success and effort required • OFC and MPFC neurons are responsible for integrating these variables to derive an abstract value signal