1 / 17

Decision making as a model

Decision making as a model. 4. Signal detection: models and measures. Nice theorems, but how to proceed in practice?. Hard work Get several points on ROC-curve by inducing several criteria (pay-off, signal frequency). Compute hit rate and false alarm rate at every criterion

zenia
Download Presentation

Decision making as a model

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Decision making as a model 4. Signal detection: models and measures

  2. Nice theorems, but how to proceed in practice? • Hard work • Get several points on ROC-curve by inducing several criteria (pay-off, signal frequency) Compute hit rate and false alarm rate at every criterion Many trials for every point! Measure or compute A using graphical methods

  3. Certainly 0 1 2 3 4 5 Certainly no signal a signal Variant: numeric (un)certainty scale: implies multiple criteria – consumes many trials too

  4. Hits 1-H F 1 - ¼ ------- + ----- 1-F H • (H-F)(1+H-F) • = ½ + ¼ ----------------- • H(1-F) FalseAlarms Average of those two areas: A' = • Rough approximation • Area measure for one point: A' H F Iff H>F

  5. B''= -.4 B''= -.07 B''= 0 B''= .07 HIT RATE B''=.4 FALSE ALARM RATE if H = 1, F≠0, F≠1, then B'' = -1 F H Comparable measure for criterion/bias: Grier’s B'' Isobias curves if H = 1 - F then B'' = 0 if F = 0, H≠ 0, H≠1 then B'' = 1 H(1 - H) – F(1 – F) B'' = sign(H - F) ------------------------ H(1 - H) + F(1 – F)

  6. Introducing assumptions • Even when several points ara available, they may not lie on a nice curve Then you might fit a curve, but which one? Every curve reflects some (implicit) assumptions about distributions Save labor: more assumptions  less measurement (but the assumptions may not be justified)

  7. Normal distributions are popular (there are other models!) Simplest model: noise and signal distributions normal with equal variance One point (PH, PFA pair) is sufficient

  8. Example: in an experiment with noise trials and signal trials these results were obtained: Hit rate: .933, False Alarm rate .309 (.067 misses and .691 correct rejections) Normal distributions: via corresponding z-scores the complete model can be reconstructed:

  9. distance: d´ = 2 measure for “sensitivity” z.933 = - 1.5 f z.309 = .5 h .933 h β = ---- = .37 f Measure for bias/criterion .309

  10. several d'-s and corresponding ROC-curves

  11. zΦ(z) = -∞ φdx ∫ 1 φ(z)= e-z2/2√2π Gaussian models: preliminary Standaard normal curve M=0, sd = 1 P z Transformations: Φ(z)  P Φ-1(P) or: Z(P)  z see tabels and standard software

  12. Roc-curve PH = f(PFA) PH PFA λ zH - Z-transformation ROC-curve P  z zH = f(zFA) zFA Nice way to plot several (PFA, PH) points

  13. Plotting with regression line? Regression line zH = a1zFA+ b1, Minimize (squared) deviations zH : Underestimation of a Regresionline zFA= a2zH + b2 Minimize (squared) deviations zFA : Overestimation of a Compromise: average of regression lines ZH = ½(a1+1/a2)ZFA + ½(b1+b2/a2)

  14. Equal variance model: PFA = 1- Φ(λ), = Φ(-λ), zFA = -λ PH = 1 – Φ(-(d' - λ)) = Φ(d'– λ), zH = d' – λ zH =zFA + d' d' = zH –zFA z-plot ROC 45° 0 λ zH d' d' 45° zFA

  15. Criterion/bias: β = h/f = φ(zH)/φ(zF) f h 1 -z2/2 φ(z) = ------ e (standard-normal) √(2π) 1 -zH2/2 φ(zH) = ------ e √(2π) zFA2 – zH2 ------------ 2 Divide: -------------------- = e 1 -zFA2/2 φ(zFA) = ------ e √(2π) To get symmety a log transformation is often applied: log β = log h – log f = ½(z2FA – z2H )

  16. zH – zFA 2zFA c = - ---------- + ----- 2 2 f h λ c Alternatve: c (aka λcenter), distance (in sd) between middle (were h=f) and criterion c = -(d'/2 – λ) zFA = -λ d' = zH - zFA zH + zFA c = - ---------- 2

  17. β c Isobias curves for βen c

More Related