1 / 29

Lesson 11 - 1

Lesson 11 - 1. Significance Tests: The Basics. Knowledge Objectives. Explain why significance testing looks for evidence against a claim rather than in favor of the claim Define null hypothesis and alternative hypothesis Define P -value Define significance level

montyj
Download Presentation

Lesson 11 - 1

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Lesson 11 - 1 Significance Tests: The Basics

  2. Knowledge Objectives • Explain why significance testing looks for evidence against a claim rather than in favor of the claim • Define null hypothesis and alternative hypothesis • Define P-value • Define significance level • Define statistical significance (statistical significance at level α)

  3. Construction Objectives • Explain the difference between a one-sided hypothesis and a two-sided hypothesis. • Identify the three conditions that need to be present before doing a significance test for a mean. • Explain what is meant by a test statistic. Give the general form of a test statistic. • Explain the difference between the P-value approach to significance testing and the statistical significance approach.

  4. Vocabulary • Hypothesis – a statement or claim regarding a characteristic of one or more populations • Hypothesis Testing – procedure, base on sample evidence and probability, used to test hypotheses • Null Hypothesis – H0, is a statement to be tested; assumed to be true until evidence indicates otherwise • Alternative Hypothesis – H1, is a claim to be tested.(what we will test to see if evidence supports the possibility) • Level of Significance – probability of making a Type I error, α

  5. Steps in Hypothesis Testing • A claim is made • Evidence (sample data) is collected to test the claim • The data are analyzed to assess the plausibility (not proof!!) of the claim • Note: Hypothesis testing is also called Significance testing

  6. Hypotheses: Null H0 & Alternative Ha • Think of the null hypothesis as the status quo • Think of the alternative hypothesis as something has changed or is different than expected • We can not prove the null hypothesis! We only can find enough evidence to reject the null hypothesis or not.

  7. Hypotheses Cont • Our hypotheses will only involve population parameters (we know the sample statistics!) • The alternative hypothesis can be • one-sided: μ > 0 or μ < 0 (which allows a statistician to detect movement in a specific direction) • two-sided: μ 0 (things have changed) • Read the problem statement carefully to decide which is appropriate • The null hypothesis is usually “=“, but if the alternative is one-sided, the null could be too

  8. Three Ways – Ho versus Ha 1 2 3 a b b a Critical Regions 1. Equal versus less than (left-tailed test) H0: the parameter = some value (or more) H1: the parameter < some value 2. Equal hypothesis versus not equal hypothesis (two-tailed test) H0: the parameter = some value H1: the parameter ≠ some value 3. Equal versus greater than (right-tailed test) H0: the parameter = some value (or less) H1: the parameter > some value

  9. English Phrases Revisited

  10. Example 1 A manufacturer claims that there are at least two scoops of cranberries in each box of cereal Parameter to be tested: Test Type: H0: Ha: • number of scoops of cranberries in each box of cereal • If the sample mean is too low, that is a problem • If the sample mean is too high, that is not a problem • left-tailed test • The “bad case” is when there are too few Scoops = 2 (or more) (s ≥ 2) Less than two scoops (s < 2)

  11. Example 2 A manufacturer claims that there are exactly 500 mg of a medication in each tablet Parameter to be tested: Test Type: H0: Ha: • amount of a medication in each tablet • If the sample mean is too low, that is a problem • If the sample mean is too high, that is a problem too • Two-tailed test • A “bad case” is when there are too few • A “bad case” is also where there are too many Amount = 500 mg Amount ≠ 500 mg

  12. Example 3 A pollster claims that there are at most 56% of all Americans are in favor of an issue Parameter to be tested: Test Type: H0: Ha: • population proportion in favor of the issue • If p-hat is too low, that is not a problem • If p-hat is too high, that is a problem • right-tailed test • The “bad case” is when sample proportion is too high P-hat = 56% (or less) P-hat > 56%

  13. Conditions for Significance Tests • SRS • simple random sample from population of interest • Normality • For means: population normal or large enough sample size for CLT to apply or use t-procedures • t-procedures: boxplot or normality plot to check for shape and any outliers (outliers is a killer) • For proportions: np ≥ 10 and n(1-p) ≥ 10 • Independence • Population, N, such that N > 10n

  14. Test Statistics Principles that apply to most tests: • The test is based on a statistic that compares the value of the parameter as stated in H0 with an estimate of the parameter from the sample data • Values of the estimate far from the parameter value in the direction specified by Ha give evidence against H0 • To assess how far the estimate is from the parameter, standardize the estimate. In many common situations, the test statistic has the form: estimate – hypothesized valuetest statistic = ------------------------------------------------------------ standard deviation of the estimate (ie SE)

  15. Example 4 Several cities have begun to monitor paramedic response times. In one such city, the mean response time to all accidents involving life-threatening injuries last year was μ=6.7 minutes with σ=2 minutes. The city manager shares this info with the emergency personnel and encourages them to “do better” next year. At the end of the following year, the city manager selects a SRS of 400 calls involving life-threatening injuries and examines response times. For this sample the mean response time was x-bar = 6.48 minutes. Do these data provide good evidence that the response times have decreased since last year? List parameter, hypotheses and conditions check

  16. Example 4 cont μ = 6.7 minutes (unchanged) Parameter: H0: Ha: Conditions Check: 1) : 2) : 3) : μ < 6.7 minutes (they got “better”) SRS stated in problem statement Normality n = 400 suggest CLT would apply to x-bar Independence n = 400 means we must assume over 4000 calls each year that involve life-threatening injuries

  17. Hypothesis Testing Approaches • P-Value • Logic: Assuming H0 is true, if the probability of getting a sample mean as extreme or more extreme than the one obtained is small, then we reject the null hypothesis (accept the alternative). • Classical (Statistical Significance) • Logic: If the sample mean is too many standard deviations from the mean stated in the null hypothesis, then we reject the null hypothesis (accept the alternative) • Confidence Intervals • Logic: If the sample mean lies in the confidence interval about the status quo, then we fail to reject the null hypothesis

  18. -z*α/2 z*α/2 Confidence Interval Approach FTR Region LB UB μ0 Reject Regions Reject Regions x – μ0 Test Statistic: z0 = ------------- z* = invnorm(1-α/2) σ/√n

  19. -zα/2 zα zα/2 -zα Classical Approach Reject Regions x – μ0 Test Statistic: z0 = ------------- σ/√n

  20. P-value • P-value is the probability of getting a more extreme value if H0 is true (measures the tails) • Small P-values are evidence against H0 • observed value is unlikely to occur if H0 is true • Large P-values fail to give evidence against H0

  21. -|z0| |z0| z0 z0 x – μ0 Test Statistic: z0 = ------------- σ/√n P-Value Approach P-Value is thearea highlighted • Probability(getting a result further away from the point estimate) = p-value • P-value is the area in the tails!!

  22. Example 5: P-Values For each α and observed significance level (p-value) pair, indicate whether the null hypothesis would be rejected. • α = . 05, p = .10 • α = .10, p = .05 • α = .01 , p = .001 • α = .025 , p = .05 e) α = .10, p = .45 α < P  fail to reject Ho P < α reject Ho P < α reject Ho α < P  fail to reject Ho α < P  fail to reject Ho

  23. Example 4 cont • What is the P-value associated with the data in example 4? • What if the sample mean was 6.61? x – μ0 6.48 – 6.7 Z0 = ----------- = -------------- σ/√n 0.10 = -2.2 P(z < Z0) = P(z < -2.2) = 0.0139 (unusual !) x – μ0 6.61 – 6.7 Z0 = ----------- = -------------- = - 0.9 σ/√n 0.10 P(z < Z0) = P(z < -0.9) = 0.1841 (not unusual !)

  24. Two-sided Test P-value • P-value is the sum of both tail areas in the two sided test case

  25. Statistical Significance Dfn • Statistically significant means simply that it is not likely to happen just by chance • Significant in the statistical sense does not mean important • Very large samples can make very small differences statistically significant, but not practically important

  26. Statistical Significance – P-value When using a P-value, we compare it with a level of significance, α, decided at the start of the test. • Not significant when α < P • Significant when α ≥ P Fail to Reject H0 Reject H0

  27. Statistical Significance Interpretation Remember the three C’s: Conclusion, connection, context • Conclusion: Either we have evidence to reject H0 in favor of Ha or we fail to reject • Connection: connect your calculated values to your conclusion • Context: Always put it in terms of the problem (don’t use generalized statements)

  28. Statistical Significance Warnings • If you are going to draw a conclusion base on statistical significance, then the significance level α should be stated before the data are produced • Deceptive users of statistics might set an α level after the data have been analyzed to manipulate the conclusion • P-values give a better sense of how strong the evidence against H0 is • This is just as inappropriate as choosing an alternative hypothesis to be one-sided in a particular direction after looking at the data

  29. Summary and Homework • Summary • Significance test assesses evidence provided by data against H0 in favor of Ha • Ha can be two-sided (different, ≠) or one-sided (specific direction, < or >) • Same three conditions as with confidence intervals • Test statistic is usually a standardized value • P-value, the probability of getting a more extreme value given that H0 is true  is small we reject H0 • Homework • 11.3, 11.6 – 11.8, 11.12 – 11.14, 11.19

More Related