1 / 50

Probability Theory Overview and Analysis of Randomized Algorithms

Probability Theory Overview and Analysis of Randomized Algorithms. Analysis of Algorithms. Prepared by John Reif, Ph.D. Probability Theory Topics . Random Variables: Binomial and Geometric Useful Probabilistic Bounds and Inequalities. Readings. Main Reading Selections:

cais
Download Presentation

Probability Theory Overview and Analysis of Randomized Algorithms

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Probability Theory Overview and Analysis of Randomized Algorithms Analysis of Algorithms Prepared by John Reif, Ph.D.

  2. Probability Theory Topics • Random Variables: Binomial and Geometric • Useful Probabilistic Bounds and Inequalities

  3. Readings • Main Reading Selections: • CLR, Chapter 5 and Appendix C

  4. Probability Measures • A probability measure (Prob) is a mapping from a set of events to the reals such that: • For any event A 0 <Prob(A) < 1 • Prob (all possible events) = 1 • If A, B are mutually exclusive events, then Prob(A  B) = Prob (A) + Prob (B)

  5. Conditional Probability • Define

  6. Bayes’ Theorem • If A1, …, An are mutually exclusive and contain all events then

  7. Random Variable A (Over Real Numbers) • Density Function

  8. Random Variable A (cont’d) • Prob Distribution Function

  9. Random Variable A (cont’d) • If for Random Variables A,B • Then “A upper bounds B” and “B lower bounds A”

  10. Expectation of Random Variable A • Ā is also called “average of A” and “mean of A” = μA

  11. Variance of Random Variable A

  12. Variance of Random Variable A (cont’d)

  13. Discrete Random Variable A

  14. Discrete Random Variable A (cont’d)

  15. Discrete Random Variable A Over Nonnegative Numbers • Expectation

  16. Pair-Wise Independent Random Variables • A,B independent if Prob(A ∧ B) = Prob(A) * Prob(B) • Equivalent definition of independence

  17. Bounding Numbers of Permutations • n! = n * (n-1) * 2 * 1= number of permutations of n objects • Stirling’s formula n! = f(n) (1+o(1)), where

  18. Bounding Numbers of Permutations (cont’d) • Note • Tighter bound = number of permutations of n objects taken k at a time

  19. Bounding Numbers of Combinations = number of (unordered) combinations of n objects taken k at a time • Bounds (due to Erdos & Spencer, p. 18)

  20. Bernoulli Variable • Ai is 1 with prob P and 0 with prob 1-P • Binomial Variable • B is sum of n independent Bernoulli variables Ai each with some probability p

  21. B is Binomial Variable with Parameters n,p

  22. B is Binomial Variable with Parameters n,p (cont’d)

  23. Poisson Trial • Ai is 1 with prob Piand 0 with prob 1-Pi • Suppose B’ is the sum of n independent Poisson trialsAi with probability Pi for i > 1, …, n

  24. Hoeffding’s Theorem • B’ is upper bounded by a Binomial Variable B • Parameters n,p where

  25. procedure GEOMETRIC parameter p ¬ begin V 0 loop with probability 1-p goto exit Geometric Variable V • parameter p

  26. Probabilistic Inequalities • For Random Variable A

  27. Markov and Chebychev Probabilistic Inequalities • Markov Inequality (uses only mean) • Chebychev Inequality (uses mean and variance)

  28. Example of use of Markov and Chebychev Probabilistic Inequalities • If B is a Binomial with parameters n,p

  29. Gaussian Density Function

  30. Normal Distribution • Bounds x > 0 (Feller, p. 175)

  31. Sums of Independently Distributed Variables • Let Sn be the sum of n independently distributed variables A1, …, An • Each with mean and variance • So Sn has mean μ and varianceσ2

  32. Strong Law of Large Numbers: Limiting to Normal Distribution • The probability density function of to normal distributionΦ(x) • Hence Prob

  33. Strong Law of Large Numbers (cont’d) • So Prob (since 1- Φ(x) <Ψ(x)/x)

  34. Advanced Material Moment Generating Functions and Chernoff Bounds

  35. Moments of Random Variable A (cont’d) • n’th Moments of Random Variable A • Moment generating function

  36. Moments of Random Variable A (cont’d) • Note S is a formal parameter

  37. Moments of Discrete Random Variable A • n’th moment

  38. Probability Generating Function of Discrete Random Variable A

  39. Moments of AND of Independent Random Variables • If A1, …, An independent with same distribution

  40. Generating Function of Binomial Variable B with Parameters n,p • Interesting fact

  41. Generating Function of Geometric Variable with parameter p

  42. Chernoff Bound of Random Variable A • Uses all moments • Uses moment generating function By setting x = ɣ’ (s) 1st derivative minimizes bounds

  43. Chernoff Bound of Discrete Random Variable A • Choose z = z0 to minimize above bound • Need Probability Generating function

  44. Chernoff Bounds for Binomial B with parameters n,p • Above mean x >μ

  45. Chernoff Bounds for Binomial B with parameters n,p (cont’d) • Below mean x <μ

  46. Anguin-Valiant’s Bounds for Binomial B with Parameters n,p • Just above mean • Just below mean

  47. Anguin-Valiant’s Bounds for Binomial B with Parameters n,p (cont’d) Tails are bounded by Normal distributions

  48. Binomial Variable B with Parameters p,N and Expectation μ= np • By Chernoff Bound for p < ½ • Raghavan-Spencer bound for any ∂ > 0

  49. Probability Theory Analysis of Algorithms Prepared by John Reif, Ph.D.

More Related