Contemporary Learning Theory

1. Contemporary Learning Theory Dr Pam Blundell Lecture Three

2. Today Finish off the material from last week! Introduce a formal learning model: the Rescorla-Wagner model Work through some predictions of the model Examine the successes of the model

3. Overshadowing Pavlov (1927) p 270 �In one dog a compound simultaneous conditioned stimulus consisted of a tactile and an auditory component, the auditory being considerably weakened. The compound stimulus, when well established, gave 4-4.5 drops of saliva during 20 seconds� isolated action. When used separately the auditory component gave a secretion of 1-1.5 drops and the tactile 2.5-5 drops�

4. Overshadowing Mackintosh (1976) Conditioned suppression procedure � effects of conditioning with two stimuli N+, N+, NL+, NL+, L+

5. Mackintosh 1976

6. Mackintosh 1976 Both Noise and Noise-light compound are well learned about What has been learned about the light (compared with group L+)

8. Mackintosh 1976 Loud noise overshadows learning about the light

10. Mackintosh 1976 Light doesn�t overshadow learning about the loud noise Light does overshadow learning about the less loud noise The more salient an element is, the more it is learned about.

11. Conditioning depends on� Contingency between CS and US Reinforcement probability Temporal relationships Biological relevance of CS and US Which other stimuli are present during conditioning

12. Blocking Kamin 1960�s Does past learning alter new learning?

15. Blocking Conditioning with a previously trained CS blocks new learning A fully predicted CS cannot be learned about

16. Conditioning depends on� Contingency between CS and US Reinforcement probability Temporal relationships Biological relevance of CS and US Which other stimuli are present during conditioning How surprising the US is

17. Pavlovian conditioning: What is learnt? The most popular and widely accepted hypothesis is that during conditioning some associations get established between the elements of the task. Associative learning theory assumes a conceptual nervous system consisting of a set of representational nodes connected by associative links. These nodes can be activated by direct application of the relevant stimulus, and also by way of excitatory associative links when these have been established by prior training

18. What is learnt?

19. Pavlovian conditioning

20. CS could become associated with the response Occurrence of US might reinforce this association (law of effect)

21. CS becomes associated with the US Would mean that US identity is know to the animal

22. S-R theory CR resembles the UR But it seems unlikely that the animal doesn�t know the identity of the US?

23. Holland & Straub 1979

24. Holland & Straub 1979 If animals know the identity of the US, they should show fewer appetitive CRs to the noise in group E than in group C

26. Dwyer 2005 Conditioned taste preferences Presented flavour A in compound with sucrose, flavour B in compound with maltodextrin (carbohydrate), flavour C alone Animals susequently preferred both A and B over C, which had been presented alone (A, B & C were grape, cherry, or tangerine Kool Aid, counterbalenced)

27. Dwyer 2005 Two bottle test Drink more CS+ (A/B) than CS-(C)

28. Dwyer 2005 Palatability can be conditioned � but what is learnt? Devalue each US in turn, by sensory specific satiety Compare consumption of tastes If animals learn about specific reinforcer identity, will prefer the taste paired with the still valued reinforcer

29. Rats do learn about the specific reinforcers, not simply that a taste is nice

30. Summary Excitatory conditioning CS-> US; CS-> CR Compare excitatory and inhibitory conditioning Inhibitory: CS-> !US; CS ?behaviour The nature of associative conditioning Discrete CSs, USs, The conditions necessary for learning Contingency, contiguity, predictive value,

31. Objectives At the end of this lecture, students should be able to: Evaluate the Rescorla-Wagner model Make unique predictions using the Rescorla-Wagner model

32. Reading Rescorla, R.A. & Wagner, A.R. (1972): A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In A. Black and W.F. Prokasy (Eds.), Classical Conditioning II: Current research and theory (64-99). New York: Appleton-Century-Crofts Dickinson p123 - 134 Pearce Ch 3 (part 1) Journal articles!

33. Formal models of associative learning Must be able to account for the conditions necessary for learning Must be falsifiable

34. Common assumptions Behaviour exhibited by the animal depends upon the strength of an association between a CS and a US (V) High associative strength = Strong CR Low associative strength = Weak CR

35. A simple model CS paired with a US Need to compute the change in associative strength (?V) ?V= a � ? a: level of activation of the CS �node� ?: level of activation of the US �node� �: learning rate parameter

36. In this model Need both CS and US to be activated for learning to occur Need to assume that activity in a node will continue for some time after the stimulus presentation, to account for learning

37. A simple model Consider if a=0.5, � =0.5, ?=1

38. Successes of the simple model V increases with each trial Sensitive to different levels of activation of CS and US

39. Failures of the simple model No shape to the learning curve No asymptote

40. A less simple model ?V= a �(?-V)

41. The less simple model Produces a learning curve, with an asymptote (V= ?) Have to assume activity in the nodes persists following presentation of the stimuli Does this model account for all data?

42. Blocking Recall:

43. Blocking Assume N and L nodes are equally activated by presentation of each stimulus, such that aN=aL=0.5 Assume ?=1

44. Group G V-L at asymptote is 0.87

45. Group B At asymptote, VL is 0.87

46. Failure of the model Doesn�t account for the blocking effect Need to incorporate how suprising the US is Rescorla-Wagner model (1972) ?V= a �(?-?V)

47. Group B

48. Overshadowing Consider Mackintosh (1976) Compare learning about L and NL aN=0.8; aL=.2 Assume ?=1

50. Successes of the R-W model Learning curve Extinction curve Generalisation Discrimination Conditioned inhibition Blocking unblocking overshadowing

51. Extinction

52. Generalisation If two stimuli are similar, animals will generalise their responding from one CS to the novel (but similar) CS. Can be accounted for within R-W model by assuming that the stimuli comprise elements AX and BX. If AX+, then when presented with BX, some associative strength activated by X.

53. Discrimination Learning to discriminate between two similar stimuli AX+ BX-

55. Conditioned inhibition Rescorla-Wagner model assumes that if a CS is a conditioned inhibitor, then it has a negative associative strength Consider Zimmer-Hart & Rescorla (1974 � see last lecture) Tone+/ToneLight-

57. Retardation test CI undergoes new conditioning Should be slower to acquire learning than a neutral CS

59. Summation test Performance is governed by the total V present on any trial CI has negative V � so will always transfer to a novel compound

60. A unique prediction of R-W model Blocking with a reduced CS X++ AX+ R-W model predicts A should become an inhibitor (simulate with ?++=1, ?+=0.5)

62. Le Pelley, Oakeshott & McLaren (2005)

64. Summary Importance of surprise in learning Formal model encapsulating this: Rescorla-Wagner model How to apply the Rescorla Wagner model Evaluation of it�s unique predictions