Probability for Computer Scientists

1. Probability for Computer Scientists CS109 Cynthia Lee

2. Today’s Topics • Last week: • Counting: sum rule, product rule, inclusion/exclusion • Counting: combinations, permutations, multinomial • Probability: set theory, axioms of probability • TODAY: Conditional Probability! • Conditional probability • Bayes Rule • Next time: • Independence

3. Example: Twins

4. Twins! • One benefit of having twins that are opposite genders is that you can dress them up as Luke and Leia • How many of the twins that you know (perhaps including yourself) are opposite-sex? (This is a non-canon scene where Leia wears her hair in the A New Hope style, while Luke has the Return of the Jedi green lightsaber…)

5. Twins! • If male and female are equally likely*, what is P(one male and one female) for fraternal twins? • 1/4 • 1/3 • 1/2 • Other/none/more than one (This is a non-canon scene where Leia wears her hair in the A New Hope style, while Luke has the Return of the Jedi green lightsaber…) * This is actually not quite true in real life.

6. Twins! • If male and female are equally likely, what is P(one male and one female) for fraternal twins? • Remember, probability for equally-likely events is calculated as |E|/|S| • What is S? • {(two males), (two females), (one of each)} • No! We need to consider the twins as individuals, not a pair: • {(F, F). (M, F), (F, M), (M,M)} • What is E? • {(F, M), (M, F)} (This is a non-canon scene where Leia wears her hair in the A New Hope style, while Luke has the Return of the Jedi green lightsaber…)

7. Twins! • If male and female are equally likely, what is P(one male and one female) for fraternal twins? • 1/4 • 1/3 • 1/2 |E|/|S| = 2/4 = 1/2 • Other/none/more than one

8. Twins! • If male and female are equally likely, what is P(one male and one female) for fraternal twins? • 1/4 • 1/3 • 1/2 |E|/|S| = 2/4 = 1/2 • Other/none/more than one Moral of the story: • Female and male are equally likely*, and each of the four possible sex pairings are equally likely, but the counts of males and females are not equally likely! • One of each is more likely than 2 females (or 2 males) * This is actually not quite true in real life.

9. Conditional Probability Given you think this topic is awesome, what is the probability that you will enjoy the homework?

10. Who wants to win cash? • Two people each roll a die, yielding values D1 and D2 • If the sum is 4, you each win $2(srsly)

11. Who wants to win cash? • Two people each roll a die, yielding values D1 and D2 • Let E be the event D1 + D2 = 4 • What is P(E)? • |S| = 36 • |E| = {(1,3), (2,2), (3,1)} • P(E) = 3/36 = 1/12

12. Who wants to win cash? • Two people each roll a die, yielding values D1 and D2 • Let E be the event D1 + D2 = 4 • What is P(E)? • |S| = 36 • |E| = {(1,3), (2,2), (3,1)} • P(E) = |E|/|S| = 3/36 = 1/12 • Now what if one person rolls before the other? You get a clue about whether you are more or less likely to win. • If D1 is 4, 5, or 6, not good 

13. Who wants to win cash? • Two people each roll a die, yielding values D1 and D2 • Let E be the event D1 + D2 = 4 • What is P(E)? • |S| = 36 • |E| = {(1,3), (2,2), (3,1)} • P(E) = |E|/|S| = 3/36 = 1/12 • Let F be the event D1 = 2 • What is P(E given F is already observed)? • |S| given F = {(2,1),(2,2),(2,3),(2,4),(2,5),(2,6)} • |E| given F = {(2,2)} • P(E given F is already observed) = 1/6

14. Conditional Probability • Definitions: Conditional Probability is the probability of an event E, given that event F has already occurred. • Call this “conditioning on F” • Write this: P(E|F) • Say this: “probability of E given F” • F acts as a filter on the sample space and the event space • Sample space is restricted to those elements that are consistent with F: • Event space is restricted to those elements that are consistent with F: Remember, EF means

15. Conditional Probability

16. Back to this example: • Two people each roll a die, yielding values D1 and D2 • You both win if D1 + D2 = 4 If you are the second person to roll, what number do you hope to see on the first roll? • 1 and 3 tie for the best • 1, 2, and 3 tie for the best • 2 is the best • Other/none/more than one

17. Back to this example: • Two people each roll a die, yielding values D1 and D2 • If you are the second person to roll, what number do you hope to see on the first roll? • Let E be the event D1 + D2 = 4 • P(E) = |E|/|S| = 3/36 = 1/12 • P(E | D1=1) = |{(1,3)}| / 6 = 1/6 • P(E | D1=2) = |{(2,2)}| / 6 = 1/6 • P(E | D1=3) = |{(3,1)}| / 6 = 1/6 • P(E | D1=4) = P(E | D1=5) = P(E | D1=6) = |{}| / 6 = 0/6 = 0 • Two winning scenarios involve 1 and 3, but once we pin down which is first, they only have one option from there, making them equally good as 2 for a first roll

18. Generalized Chain Rule

19. Back to this example: • Two people each roll a die, yielding values D1 and D2 • If you are the second person to roll, what number do you hope to see on the first roll? • Let E be the event D1 + D2 = 4 • P(E) = |E|/|S| = 3/36 = 1/12 • P(E | D1=1) = |{(1,3)}| / 6 = 1/6 • P(E | D1=2) = |{(2,2)}| / 6 = 1/6 • P(E | D1=3) = |{(3,1)}| / 6 = 1/6 • P(E | D1=4) = P(E | D1=5) = P(E | D1=6) = |{}| / 6 = 0/6 = 0 • Two winning scenarios involve 1 and 3, but once we pin down which is first, they only have one option from there, making them equally good as 2 for a first roll

20. Bayes Theorem Applied conditional probability

21. Important fact that of which all CS109 students have historically been made aware, and of which I will now make you aware: • The Rev. Thomas Bayes (1702-1761) bore an uncanny resemblance to actor Charlie Sheen. • What are the odds, right? (get it, odds?)

22. Bayes Theorem • Ross book form: • More common form: • Expanded form:

23. Bayes Theorem Examples Applied conditional probability

24. Test accuracy • An HIV test is 98% accurate at detecting HIV • It has a 1% false positive rate • 0.5% of the US population has HIV • Let E = you test positive • Let F = you have HIV • What is P(F|E)? • In other words, given you test positive, what is the probability that you actually have HIV? • First, just do a “gut reaction” prediction of how this math is going to work out: • Pr(F|E) ≤ 1/2 • 1/2 <Pr(F|E) ≤ 3/4 • 3/4 <Pr(F|E)

25. Test accuracy • An HIV test is 98% accurate at detecting HIV • It has a 1% false positive rate • 0.5% of the US population has HIV • Let E = you test positive • Let F = you have HIV • What is P(F|E)? • In other words, given you test positive, what is the probability that you actually have HIV?

26. Test accuracy • Wait, what? If a positive test only means a 1/3 chance that you have the disease, is the test useless? Why get tested? • What if your test is negative? What is the probability that you actually have HIV? • First, just do a “gut reaction” prediction of how this math is going to work out: • Pr(F | EC) ≤ 1% • Pr(F | EC) ≤ 0.5% • Pr(F | EC) ≤ 0.05% • Other/none

27. Test accuracy • Wait, what? If a positive test only means a 1/3 chance that you have the disease, is the test useless? Why get tested? • What if your test is negative? What is the probability that you actually have HIV?