1 / 32

Markov Models

Markov Models. Agenda. Homework Markov models Overview Some analytic predictions Probability matching Stochastic vs. Deterministic Models Gray, 2002. Choice Example. A person is given a choice between ice cream and chocolate. The person can be Undecided. Choose ice cream.

trory
Download Presentation

Markov Models

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Markov Models

  2. Agenda • Homework • Markov models • Overview • Some analytic predictions • Probability matching • Stochastic vs. Deterministic Models • Gray, 2002

  3. Choice Example • A person is given a choice between ice cream and chocolate. • The person can be • Undecided. • Choose ice cream. • Choose chocolate. • There is some probability of going from being undecided to • Staying undecided and giving no decision. • Choosing ice cream. • Choosing chocolate.

  4. Markov Processes • States • The discrete states of a process at any time. • Transition probabilities • The probability of moving from one state to another. • The Markov property • How a process gets to a state in unimportant. All information about the past is embodied in the current state.

  5. State Space SIce Cream SUndecided SChocolate

  6. Transition Probabilities P(SU|SU)=1-( + ) SIce Cream P(SI_C|SI_C)=1 P(SI_C|SU)= SUndecided P(SC|SU)= P(SC|SC)=1 SChocolate Note: The transition probabilities out of a node sum to 1. How can this model be made equivalent to Luce Choice?

  7. Transition Probabilities P(SU|SU)=1-( + ) SIce Cream P(SI_C|SI_C)=1 P(SI_C|SU)= SUndecided P(SC|SU)= P(SC|SC)=1 SChocolate P(SI_C|SU) = v(I_C)/(v(I_C)+v(C)) P(SC|SU) = v(C)/v(I_C)+v(C))

  8. Transparent Responses P(SU|SU)=1-( + ) SIce Cream P(SI_C|SI_C)=1 P(SI_C|SU)= SUndecided P(RI_C|SI_C)=1 P(SC|SU)= P(SC|SC)=1 SChocolate P(Rnone|SU)=1 P(RI_C|SC)=0

  9. Transparent Responses P(SU|SU)=1-( + ) SIce Cream P(SI_C|SI_C)=1 P(SI_C|SU)= SUndecided P(RI_C|SI_C)=.8 P(SC|SU)= P(SC|SC)=1 SChocolate P(Rnone|SU)=1 P(RI_C|SC)=.2

  10. State Sequence Time SU SU SU SC SC Hidden … None None None Choc. Choc. Observed … … t1 t3 t4 t5 t2

  11. Matrix Form of Transition Probabilities To From

  12. Some Analytic Solutions Where =P(SI_C|SU) and =P(SC|SU)

  13. Some Analytic Solutions • What happens if: • +=1? • t=1?

  14. =.25, =.4

  15. More Analytic Solutions

  16. =.25, =.4

  17. Problem? • Why don’t the choices sum to 1?

  18. More Results… • The matrix form is very convenient for calculations. • It is easy to calculate all moments. • More to come with random walks…

  19. Pair Clustering • Batchelder & Riefer, 1980 • Free recall of clusterable pairs. • Implements a Markov model for the probability that a pair is clustered on a particular trial. • Are MPTs Markov models?

  20. Probability Matching • Paradigm • Warning light • Prediction: P(R1), P(R2) • Feedback: P(E1)=, P(E2)=1- • Typical result • P(R1) 

  21. Probability Matching • Can be implemented via a Markov model. • Assume win-stay/lose-shift paradigm • If “correct”, make same prediction • If “incorrect”, shift response with probability . • Associate an “element” with most recent event, but not perfectly.

  22. Next Trial Current Trial RiEj = Response i and then Feedback j.  = Probability of Feedback 1.  = Probability of switching after error.

  23. Markov Property

  24. Light2 Light1

  25. Markov Property P(Honk) = 0 P(Honk) = 0 L1/Go L2/Go .7 .3 .3 L1/Stop L2/Stop .7 P(Honk) = .3 P(Honk) = .4

  26. Markov Property L1/Go L2/Go P(Honk) = .3 P(Honk) = .4 L1/Stop L2/Stop P(Honk) = .8 P(Honk) = .7 L1/Stop Repeat L2/Stop Repeat

  27. Stochastic vs. Deterministic • Stochastic model: The processes are probabilistic. • Deterministic: The processes are completely determined.

  28. Stochastic Models Imply: • Psychological events are uncertain • Even if we had all the knowledge we needed we could still not figure out what a person is going to do next. • Or

  29. Stochastic Models Imply: • The model does not capture all aspects of the behavior in question • Allows the model to focus on certain parts of behavior and ignore others. • You may believe behavior is deterministic, but still rely on a stochastic model. • Allows the modeler to finesse some ignorance. • OR

  30. Stochastic Models Imply: • Some parts of the task are truly random • E.g., feedback schedule from the experimenter in a probability matching task.

  31. Limitation of Stochastic Models • You need to test them on populations of behavior, not individual behaviors. • E.g., I gave Participant X a single choice and she chose ice cream. • Can we test the model against this datum?

More Related