1 / 118

Decision-theoretic systems in the real world Decision support

Decisions Under Incomplete Representations & Reasoning Some Reflections Eric Horvitz Microsoft Research SADR Meeting on Decision Making in Complex Environments Cornell University April 5, 2003 horvitz@microsoft.com http://research.microsoft.com/~horvitz/. Overview.

cullen
Download Presentation

Decision-theoretic systems in the real world Decision support

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Decisions Under Incomplete Representations & ReasoningSome ReflectionsEric HorvitzMicrosoft ResearchSADR Meeting on Decision Making in Complex EnvironmentsCornell UniversityApril 5, 2003horvitz@microsoft.comhttp://research.microsoft.com/~horvitz/

  2. Overview • Decision-theoretic systems in the real world • Decision support • Autonomous decision making, embedded processes • Some successes • Decision making under bounded resources • Inference and modeling • Systems immersed in environments Eric Horvitz, April 5, 2003

  3. Fuel Battery p(F) p(B) Gauge p(G|F,B) TurnOver Start p(T|B) p(S|F,T) Graphical Models and Decision Systems • Twenty years of accelerated results, innovation • Bayesian networks and influence diagrams • Parsimonious representation of joint distribution • Theory of independence, d-separation Eric Horvitz, April 5, 2003

  4. D A A R R A D A R D D R D D R Handcrafted and Learned Expert Model Data Learned Model Eric Horvitz, April 5, 2003

  5. Representing Problems of Action • Influence diagrams (Howard & Matheson, 1975) Action A Utility World State H E3 E2 E1 En Eric Horvitz, April 5, 2003

  6. FUTURE CORONARY ARTERY ARTERY STATUS STATUS FUTURE CHEST PAIN CHEST PAIN MYOCARDIAL INFARCTION LIFE QUALITY HEART SURGERY REMAINING LIFE VALUE YEARS MONETARY COST Representing Multiple Dimensions of Value Eric Horvitz, April 5, 2003

  7. Plan {(A1,t1)...(An,tn)} Utility An, tn A3, t3 A1, t1 State of System H, t1 State of System H, tn State of System H, t2 En, t1 En, t4 E2, t1 E2, t4 En, t2 E1, t1 E2, t2 E1, t4 E1, t2 Eric Horvitz, April 5, 2003 Beyond Single Shot Decisions • Sequences of inference, actions, sensing under uncertainty

  8. Reward (t+1) Reward t Action(t+1) Action(t) Observe t+1 Observe t State t+1 State t State t+2 Eric Horvitz, April 5, 2003 Partially Observable Markov Decision Process Representation

  9. Real-World Applications Eric Horvitz, April 5, 2003

  10. Scaling Up Bayesian Networks PULMONARY EMBOLUS HYPOVOLEMIA LV FAILURE ANAPHYLAXIS ANESTHESIA INSUFFICIENT KINKED TUBE SHUNT PAP DISCONNECTION INTUBATION STROKE LVED HISTORY VOLUME VOLUME LV FAILURE VENT MACHINE CATECHOLAMINE VENT ALV CARDIAC VENT SAO2 TPR OUTPUT TUBE VENT LUNG CVP PCWP HEART PA SAT RATE MV SETTING BLOOD PRESSURE ERROR MINUTE CAUTER ERROR VENTILATION FIO2 PRESSURE LOW OUTPUT ARTERIAL CO2 EXPIRED HR BP HR EKG HR SAT CO2 Eric Horvitz, April 5, 2003

  11. Chief Complaint: Abdominal Pain Eric Horvitz, April 5, 2003

  12. Pulomonary/Sleep ApneaIntegration with Physiological Monitoring Eric Horvitz, April 5, 2003

  13. Fault Diagnosis and Troubleshooting Eric Horvitz, April 5, 2003

  14. On the Web… www.microsoft.com Eric Horvitz, April 5, 2003

  15. Very long Very short Control of Computation: Restarts in Satistifiabilty Solvers Great variation in execution time • Very short and very long runs for different randomized runs on same instances • Heavy-tailed distribution (Pareto) Eric Horvitz, April 5, 2003

  16. 100-1000x Decisom Model for Dynamic Restart Policies • To date: success with simple fixed policy: Restart search if run-time is greater than x • Orders of magnitude speedup Time to Solution Time expended before restart Gomes, et al. 1997 Eric Horvitz, April 5, 2003

  17. Learning Bayesian Models + Decision-Theoretic Analysis • Features • Dynamic and static Gomes, et al. 1997 Eric Horvitz, April 5, 2003

  18. Eric Horvitz, April 5, 2003 Horvitz, et al. 2001

  19. Human-Computer Interaction User’s Profile User’s Goals User’s Needs User Activity Eric Horvitz, April 5, 2003

  20. Monitoring Space Shuttle Telemetry Horvitz & Barry, 95 Eric Horvitz, April 5, 2003

  21. Assistance with Software User background Primary goal Hierarchical presentation Consolidation Pivot wizard Chart wizard Repeated chart create/delete Group mode Database defined 3D cell reference Leading spaces External reference Repeated chart change Use query Repeated print / hide Rows Multicell selection Adjacent conceptual granularity Eric Horvitz, April 5, 2003 Horvitz, et al., 98

  22. Profile Profile Inference about a User’s Time-Dependent Goals Profile Goalt-n Goalto Goalt-1 Ei,t-n Ej,t-n Ei,to Ei,t-1 Ej,to Ej,t-1 Time Horvitz, et al. 98 Eric Horvitz, April 5, 2003

  23. Information Filtering & Alerting Horvitz, et al., 03 Eric Horvitz, April 5, 2003

  24. Reasoning about Attention Horvitz, et al., 03 Eric Horvitz, April 5, 2003

  25. Horvitz, et al., 03 Eric Horvitz, April 5, 2003

  26. Decision-Theoretic Control of Dialog Horvitz & Paek, 01 Eric Horvitz, April 5, 2003

  27. Harnessing Multiple Classes of Evidence • Speech recognizer confidence • Correctness of natural language parse as evidence • Visual analysis of pose Paek & Horvitz, 00 Eric Horvitz, April 5, 2003

  28. ASR Reliability Indicator(t-1) Intention, Attention, and Clarification Dialog Dialog or Domain-Level Action(t-1) Dialog or Domain-Level Action(t) Utility(t-1) Utility(t) Context Context Speaker’s Goal(t-1) Speaker’s Goal(t) User’s Spoken Intention(t-1) User’s Spoken Intention(t) Content at Focus (t-1) Content at Focus (t) User Actions(t-1) User Actions(t) ASR Reliability Indicator(t-1) ASR Candidate n Confidence(t-1) ASR Candidate 1 Confidence(t-1) ASR Candidate n Confidence(t) ASR Candidate 1 Confidence(t) . . . . . . Horvitz & Paek, 01 Eric Horvitz, April 5, 2003

  29. yes Engage Repeat reflect Tshoot Expected utility of actions Inferred beliefs Horvitz & Paek, 01 Eric Horvitz, April 5, 2003

  30. overheard Tshoot yes Repeat no Engage reflect Interaction Troubleshooting Horvitz & Paek, 01 Eric Horvitz, April 5, 2003

  31. Assessing Preferences on Outcomes Horvitz & Paek, 01 Eric Horvitz, April 5, 2003

  32. Assessing Frustration with Number of Steps Horvitz & Paek, 01 Eric Horvitz, April 5, 2003

  33. But Still Cannot Solve Important Problems • Intractability per problem size and structure • Little experience with fielding autonomous, immersed decision making systems • Poor understanding of subobtimality associated with incomplete models and inference Toward principles of rational action under resource limitations Eric Horvitz, April 5, 2003

  34. Hardness • Inference is NP-Hard; approximation is NP-Hard • Work on a tapestry of exact and approximate algorithms • Exploit special structure, probe portions of model, restricted probabilistic relationships Eric Horvitz, April 5, 2003

  35. Quickscore TopN S Heckerman 1989 Henrion 1988, 1990 Shwe & Cooper 1990 For QMR-DT network A Tapestry of Approximation Algorithms Approximate methods Exact methods Polytrees Stochastic Search Kim & Pearl, 1983 simulation methods Arc reversal and Branch and bound reduction Olmsted, 1983 Logic sampling Shachter, 1986, 1988. NESTOR Markov simulation Forward propagation Loop Cutset Cooper, 1984 Henrion, 1986 Gibbs sampling conditioning : Set-covering: Pearl, 1987 Pearl, 1986 Peng & Reggia, Moralizing and propag- 1987 ation in a tree of cliques Lauritzen & Spiegelhalter, 1988 FPRAS: Likelihood Importance Bounded conditioning Chavez & Clustering weighting sampling Horvitz and Suermondt, Cooper, Pearl 1988 1988 1989 PIBNET is NP-hard Cooper, 1987 Fung & Shachter & Selective Chang 1989 Peot 1989 conditioning Dagum and Horvitz, 1992 Eric Horvitz, April 5, 2003

  36. Reasoning Across Internal Medicine • QMR-DT: Internal Medicine QMR-DT derived from Internist-1/ QMR KB 534 diseases 40740 arcs 4040 findings Eric Horvitz, April 5, 2003

  37. Toward Principles • What do we do when we cannot solve a decision problem completely? • How do we trade computation time and value of computation? • What is a complete problem? • How do we reason about sensitivity of the MEU of action to refinement effort? • Toward principles of decisions under bounded resources • Simon: Inadequacy of MEU • Good: Type II Rationality Eric Horvitz, April 5, 2003

  38. Implementing Type II RationalityUnder Limited Resources • Need tractable meta-analysis of ideal nature and extent of reasoning • Need characterizable approximation methods • Need good understanding of cost of time, memory Eric Horvitz, April 5, 2003

  39. Understanding Cost of Computation • “Cost:” “… … - C(t)” • Influence of specific classes of cost on value of computation, comprehensive value of ultimate action • Active inference about the structure of cost, expectation Eric Horvitz, April 5, 2003

  40. Cost of Computational Resources Multiple cost contexts Urgency Deadline Urgent Deadline Uncertain Deadline Target • Urgency • Target • Deadline • Uncertain deadline Eric Horvitz, April 5, 2003 Horvitz, 88

  41. Time Criticality for Shuttle Propulsion Barry and Horvitz (1993) 1.0 p(Engine Damaged) 0.5 p(Shrapnel) 0 1 2 3 4 5 Seconds Continue During Propellant Failure * p(Vehicle Lost | Engine Damaged, No Shrapnel,C1) = .5 * p(Vehicle Lost | Engine Damaged, Shrapnel, C1) = .8 Horvitz & Barry, 95 Eric Horvitz, April 5, 2003

  42. 1.0 • Minor Fracture • Hollow viscus Pr(Survive) • Solid organ injury • Major fracture • Intracranial hemorrhage • Major respiratory • Major vascular • Major airway 0.0 0 30 90 60 Delay of emergency center care (minutes) Inferring Costs of Delay • Case of transportation planning for trauma care Horvitz & Seiver, 97 Eric Horvitz, April 5, 2003

  43. Context-Sensitive Costs Eric Horvitz, April 5, 2003 Horvitz & Seiver, 97

  44. Characterizing the Value of Computation Eric Horvitz, April 5, 2003

  45. Idealization.. Utility • Immediate value of result • Net value of result t • Cost of delay Horvitz, 90 Eric Horvitz, April 5, 2003

  46. Object value Two opt, nearest neighbor • • Utility Net value 30 cities 5 10 15 20 25 30 35 40 45 50 55 60 Time (seconds) Horvitz & Klein, 91 Cost of delay Horvitz & Klein, 91 Eric Horvitz, April 5, 2003

  47. Available idle time Uncertain Quality, Cost, Resources Utility • Immediate value of result • Net value of result t Horvitz, 90 Eric Horvitz, April 5, 2003

  48. Models of Type II RationalityExample: Acute Respiratory Failure (ARF) vs. Respiratory Distress (RD) Horvitz, 90 Eric Horvitz, April 5, 2003

  49. Example:Acute Respiratory Failure (ARF) vs. Respiratory Distress (RD) Actions ARF No ARF A: Treat ARF Treat ARF u(A,S) u(A,S) A: Treat RD States S: ARF Treat RD u(A,S) u(A,S) S: RD Horvitz, 90 Eric Horvitz, April 5, 2003

  50. 1.0 0.8 0.6 Probability 0.4 0.2 0.0 0 10 20 30 40 Time (Subproblems solved) Bounding Conditioning Approximation:Acute Respiratory Failure vs. Respiratory Failure p(S|E1… En) Eric Horvitz, April 5, 2003 Horvitz, et al. 89

More Related