Handling Uncertainty

1. Handling Uncertainty John MacIntyre 0191 515 3778 john.macintyre@sunderland.ac.uk COM362 Knowledge Engineering Handling Uncertainty

2. Reasoning with Uncertainty • Why • Uncertain answers • Confidence Factors • Probabilities • Fuzzy Logic COM362 Knowledge Engineering Handling Uncertainty

3. Why? 82% of all statistics are made up on the spot!Vic Reeves COM362 Knowledge Engineering Handling Uncertainty

4. How do Humans Cope with Uncertainty? • Default/assumed reasoning (guess work) • non monotonic reasoning • Missing data • gradual degradation in performance • Uncertain information • Uncertain reasoning • Not always one correct solution • Ranking possible answers COM362 Knowledge Engineering Handling Uncertainty

5. Uncertain answers • One very simple method of dealing with uncertainty • Allow Yes / No / Unknown answers • Process ‘Unknown’ by triggering extra inferencing to determine answer when user cannot • Automatic depending upon sourcing sequence within Aion DS COM362 Knowledge Engineering Handling Uncertainty

6. Confidence/Certainty Factors • Confidence expressed as a number between 0 and 1 • Allows uncertainty to be expressed in information and in reasoning • Not necessarily based on any real evidence! COM362 Knowledge Engineering Handling Uncertainty

7. CFs in Practice • Assume that if A is true, then B is true • But we are only 80% certain that A is true • Clearly we can only be 80% certain that B is true! 0.8 A => B COM362 Knowledge Engineering Handling Uncertainty

8. CFs continued... • What if we were only 80% certain that A=>B, and only 60% certain that B=>C? 0.8 0.8 0.6 A => B => C COM362 Knowledge Engineering Handling Uncertainty

9. CFs continued... • How certain can we be that B and C are true? • Each uncertain step in the reasoning process must make us less certain in the result - this is mimicked by multiplying CFs together • Multiplying fractions reduces the total at each step COM362 Knowledge Engineering Handling Uncertainty

10. CFs continued... • Thus, given we are only 80% certain of A, we can only be 64% certain that B is true • (0.8 * 0.8 = 0.64) • and only 38% certain that C is true • (0.8*0.8*0.6 = 0.38) COM362 Knowledge Engineering Handling Uncertainty

11. CFs continued... • However, two independent pieces of corroborating evidence must make us more certain of the result - so how does this work? 0.8 0.8 Given A => C AND B => C • Given A and B as true how confident are we of C? • (clearly the answer should be higher than 0.8 but less than 1 COM362 Knowledge Engineering Handling Uncertainty

12. CFs continued... • To calculate the answer we invert the rules: 0.2 0.2 Thus A => not (C) and B => not (C) • Then multiply 0.2 by 0.2 to give 0.04 - thus: 0.04 A+B => not (C) • And invert this again to give: 0.96 A+B => C COM362 Knowledge Engineering Handling Uncertainty

13. Disadvantages of CFs • CF’s come from the opinions of one or more experts and thus have very little basis in fact • People are unreliable at assigning confidence or certainty values • Two people will assign very different numbers and will often themselves be inconsistent from day to day COM362 Knowledge Engineering Handling Uncertainty

14. Advantages of CFs • CFs do allow people to express varying degrees of confidence • Easy to manipulate • Can work well when trying to rank several possible solutions • Must be careful not to place too much emphasis on the actual numbers generated COM362 Knowledge Engineering Handling Uncertainty

15. Probabilities • Advantages • Much stronger mathematical foundation for results • Based on precise data • Disadvantages • Only work when required statistical data is available • Slightly more mathematical COM362 Knowledge Engineering Handling Uncertainty

16. Bayes’ Theorem P(E:H) P(H) P(H:E) = P(E:H) P(H) + P(E:not H) P(not H) And also (1 - P(E:H)) P(H) P(H:not E) = (1 - P(E:H)) P(H) + (1- P(E:not H)) P(not H) COM362 Knowledge Engineering Handling Uncertainty

17. Bayes’ Theorem • Thankfully this is not as complex as it looks !!! • Must understand the notation and the principle of two types of probability: • prior probabilities - our previous assumptions or based on previous evidence • posterior probabilities - amended based on new evidence COM362 Knowledge Engineering Handling Uncertainty

18. The Notation P(H) The prior probability of a hypothesis (H) being true P(E:H) The probability of an event (E) being true given the hypothesis (H) is true P(H:E) The probability of the hypothesis (H) being true given that the event (E) is true P(E:not H) The probability of an event (E) being true given that the hypothesis (H) is known to be false COM362 Knowledge Engineering Handling Uncertainty

19. Working with an Example • Imagine we are trying to determine if a patient has the flu • The hypothesis (H) is that they have the flu • The events (E) are the symptoms they present • There are multiple events (symptoms) COM362 Knowledge Engineering Handling Uncertainty

20. Example continued... • In this case the patient has a temperature, a runny nose and is sneezing but they do not have a headache or other symptoms that indicate flu. • How do we determine the specific probability of flu given this particular set of symptoms? COM362 Knowledge Engineering Handling Uncertainty

21. Easy for One Symptom! We can tell from past experience that P(flu) = 0.3 This is easy to measure! Given 1 symptom we can also determine the probability of the hypothesis:- When symptom is true Flu not (Flu) Temperature .4 .6 Runny nose .4 .6 Hot flushes .5 .5 Thus P(flu : Temperature) = 0.4 P(flu : Temperature & Runny nose) = ??? P(flu : Temperature and Hot flushes) = ??? COM362 Knowledge Engineering Handling Uncertainty

22. What do we Know? • We CAN NOT measure • P(H:E1 & E2 & not E3) • We CAN measure • P(H), P(E1:H), P(E2:H) P(E3,H) • P(E1:not H), P(E2:not H), P(E3:not H) • Using Bayes Theorem we can then calculate P(H:E1 & E2 & not E3) COM362 Knowledge Engineering Handling Uncertainty

23. A Different Approach • Need to change the way we approach the problem • Intuitively we fix the symptom and then determine the hypothesis • Instead, lets first fix the hypothesis and then determine the probabilities of the symptoms COM362 Knowledge Engineering Handling Uncertainty

24. Evidence from Data • Two separate populations of patients • 100 people with flu • 100 people without flu • % symptoms in each population Symptom when Flu is true not (Flu) Temperature .7 .5 Runny nose .6 .2 Hot flushes .7 .1 COM362 Knowledge Engineering Handling Uncertainty

25. Using the Evidence • Notice this data is NOT the same as shown previously and the numbers do not necessarily add up to 1 • Collecting this data is easy • We can now repeatedly use the equations given to calculate the probability of flu given a range of symptoms COM362 Knowledge Engineering Handling Uncertainty

26. Applying Bayes • Given P(Flu) = 0.3 and the table above • If a patient has a temperature we can use P(E:H) P(H) P(H:E) = P(E:H) P(H) + P(E:not H) P(not H) to calculate P(flu : temperature) 0.7 * 0.3 P(H:E) = = 0.375 0.7*0.3 + 0.5 *0.7 COM362 Knowledge Engineering Handling Uncertainty

27. Applying Bayes • Given this result we can now calculate P(flu:temperature but not runny nose) • We now use 0.375 as P(H) and the second equation (1 - P(E:H)) P(H) P(H:not E) = (1 - P(E:H)) P(H) + (1- P(E:not H)) P(not H) COM362 Knowledge Engineering Handling Uncertainty

28. Applying Bayes (1 - 0.6) * 0.375 P(H:not E) = (1 - 0.6)* 0.375 + (1- 0.2)*(1-0.375) = 0.23 COM362 Knowledge Engineering Handling Uncertainty

29. Bayes’ Theorem • Bayes’ theorem allows us to calculate the probability of a hypothesis given any combination of symptoms (or events) • The raw data is easy to obtain though we must make sure that the symptoms are independent of each other • Normal practice would assume independence unless common sense suggests otherwise COM362 Knowledge Engineering Handling Uncertainty

30. Bayes’ Theorem • The equations can also be used to determine which symptom has the most significant effect on all potential hypothesis (diagnosis) and also when all significant events have been determined COM362 Knowledge Engineering Handling Uncertainty

31. Advantages/Disadvantages • Bayes theorem is mathematically sound therefore results using this method can be justified • However, this method needs data collected from previous results and will only work where this is available • This is in itself a strong point as the data can be proven where CFs are only expressions of opinion COM362 Knowledge Engineering Handling Uncertainty

32. Fuzzy Logic • Attempts to move away from “hard” values in reasoning • More like the way humans think • Example: what is ‘room temperature’? • 21oC....? • 20, 21, 22oC....? • 19 - 23oC....? • 17 - 24oC....? COM362 Knowledge Engineering Handling Uncertainty

33. Fuzzy Set Theory • Fuzzy set theory: • Define a set “room temperature” • Define the range of values in the set (17-24oC) • Define the strength of membership of each value in the set, using a membership function COM362 Knowledge Engineering Handling Uncertainty

34. Fuzzy Set Membership 1 0 Strength of Membership 17 18 19 20 21 22 23 24 Set Values for “Room Temperature” COM362 Knowledge Engineering Handling Uncertainty

35. Using Natural Terms • Allows use of intuitive terms in constructing rule base, such as: • temperature is ‘high’ • vibration is ‘low’ • load is ‘medium’ • Outputs can also be in these terms, such as: • bearing damage is ‘moderate’ • unbalance is ‘very high’ COM362 Knowledge Engineering Handling Uncertainty

36. Fuzzy Logic Strengths • Strengths of fuzzy logic: • less rules required since single set membership function can cover a large range of values • membership function can be used to represent intuitive knowledge from experts • outputs can be in familiar terms COM362 Knowledge Engineering Handling Uncertainty

37. Fuzzy Logic Limitations • Limitations of fuzzy logic: • still requires writing of many rules • knowledge acquisition and representation problems apply • not adaptive in its pure form • difficult to maintain and upgrade COM362 Knowledge Engineering Handling Uncertainty

38. Applications of Fuzzy Logic • Many applications, especially in domestic appliances: • auto-focusing in cameras • washing machine controllers • microwave controllers • etc etc etc • Development of ‘Neuro-Fuzzy Systems’ - to give some adaptive ability COM362 Knowledge Engineering Handling Uncertainty

39. Conclusions • Humans deal intuitively with uncertainty all the time • Knowledge-based systems can benefit from being able to mimic this • Three different ways of implementing uncertainty • CFs, Bayesian Mechanics, Fuzzy logic • There are other approaches!! COM362 Knowledge Engineering Handling Uncertainty