1 / 16

Improving the Quality of Investigative Decision Making

Improving the Quality of Investigative Decision Making. Craig Bennell, Ph.D. Carleton University Ottawa, Ontario, Canada cbennell@connect.carleton.ca. Two-Alternative Decisions. Is this suspect lying? Will this negotiation end in failure? Are these crimes linked?

marva
Download Presentation

Improving the Quality of Investigative Decision Making

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Improving the Quality of Investigative Decision Making Craig Bennell, Ph.D. Carleton University Ottawa, Ontario, Canada cbennell@connect.carleton.ca

  2. Two-Alternative Decisions • Is this suspect lying? • Will this negotiation end in failure? • Are these crimes linked? • Was this bite mark made by an adult? • Is this suicide note fake? • Is this allegation of rape false? • To improve decision-making we need a valid measure of performance...

  3. EXAMINE THE EVIDENCE COUNT THE FREQUENCIES OF DECISION OUTCOMES SET A DECISION THRESHOLD CALCULATE PERCENTAGE CORRECT How Should We Measure Performance?

  4. Relatively Inaccurate Evidence

  5. Relatively Accurate Evidence

  6. ROC ANALYSIS (Bennell, 2005)

  7. ROC ANALYSIS An AUC of .60 indicates that there is a 60% chance that a randomly selected positive case will be higher on evidence X than a randomly selected negative case (Bennell, 2005)

  8. The Accuracy of Diagnostic Systems: The Polygraph CQT and GKT • The Control Question Test • Relevant (R) and Control (C) questions • For each R-C pair a score of -3 to +3 is given • Greater than +5: truthful, Less than -5: deceitful • The Guilty Knowledge Test • Multiple choice questions • One alternative relates to guilty knowledge (GK) • Elevated responses to GK questions: deceitful • How accurate are these procedures?

  9. Median AUC for GKT = .88 Median AUC for CQT = .85 Accuracy scores for 52 lab studies examining the validity of the polygraph (National Academy of Science, 2003)

  10. Identifying Accurate Predictors: The Fake Suicide Note • What discriminates between real and fake notes? • Structure variables • Content variables • Combinations of the two • How can we tell if some are better than others? • How can we determine how much better?

  11. 4 3 2 1 1 - Structure model (AUC = .69) 2 - Content model (AUC = .74) 3 - Optimal model (AUC = .82) 4 - Inclusive model (AUC = .85) Optimal: Average sentence length (shorter in genuine) Positive affect (more in genuine) Probability of false alarms (Jones & Bennell, 2005)

  12. Setting Appropriate Thresholds:Linking Serial Burglaries • Many behaviours can be used • Inter-crime distance is very useful (AUC=.80) • The closer two crimes are to one another the more likely it is that they are linked • When should we decide that crimes are linked? • What is an appropriate threshold?

  13. 3.50km (85% H, 45% FA) 2.50km (72% H, 25% FA) 1.50km (58% H, 11% FA) 1.10km (49% H, 10% FA) (Bennell & Canter, 2002)

  14. Comparing Decision Makers:Bite Mark Identification • It is often unclear as to the source of a bite mark • Non-accidental (adult) • Accidental (child) • How well can people perform on this task? • Is expertise required?

  15. Senior forensic dentists (AUC=.69) Student dentists (AUC=.69) Junior forensic dentists (AUC=.68) General dentists (AUC=.62) Social workers (AUC=.63) Police officers (AUC=.62) (Whittaker et al., 1998)

  16. Conclusions • ROC analysis provides a way of calculating accuracy scores that are not dependent on arbitrary thresholds • ROC analysis also provides a way of identifying thresholds that meet the demands of specific situations • The result for police investigations will likely be improved decision-making accuracy and utility

More Related