1 / 24

PSY 1950 Confidence and Power December, 1 2008

PSY 1950 Confidence and Power December, 1 2008. Requisite Quote. “The picturing of data allows us to be sensitive not only to the multiple hypotheses that we hold, but to the many more we have not yet thought of, regard as unlikely, or think impossible.” Tukey, 1974.

ora-thomas
Download Presentation

PSY 1950 Confidence and Power December, 1 2008

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. PSY 1950 Confidence and Power December, 1 2008

  2. Requisite Quote “The picturing of data allows us to be sensitive not only to the multiple hypotheses that we hold, but to the many more we have not yet thought of, regard as unlikely, or think impossible.” • Tukey, 1974

  3. Point Estimation vs. Interval Estimation • Confidence intervals estimate parameters with intervals instead of simply points

  4. Reliability • Confidence intervals indicate the reliability of an estimate

  5. Confidence Level • Confidence intervals are a function of the desired precision

  6. Confidence Interval • Definition • “For a given proportion p (where p is the confidence level), a confidence interval for a population parameter is an interval that is calculated from a random sample of an underlying population such that, if the sampling was repeated numerous times and the confidence interval recalculated from each sample according to the same method, a proportion p of the confidence intervals would contain the population parameter in question”

  7. http://www.ruf.rice.edu/~lane/stat_sim/conf_interval/

  8. Correct Interpretation • e.g., “If we replicated our experiment, the calculated confidence intervals would contain the true population mean 95% of the time.” • e.g., “The confidence interval represents values for the population parameter for which the difference between the parameter and the observed estimate is not statistically significant at the 5% level."

  9. Incorrect Interpretation • Confidence intervals do NOT reflect the probability that the parameter falls within an estimated range • Wrong: “There is a 95% chance that the actual mean group difference is between 2-3.” • The true mean is fixed, not variable: it either falls inside or outside a particular range • e.g., what is the probability that 2 is between 3-4?

  10. “This is not just quibble” • Imagine CIs constructed from two samples from the same population • 1st sample’s 70% CI: 1-2 • 70% chance that the mean is between 1-2 • 2nd sample’s 70% CI: 2-3 • 30% chance that the mean is less than 2 or greater than 3 • These are incompatible!

  11. Eyeballing Interval Estimates • Question matters • Do you care about estimated values or differences of estimated values? • Values matters • Note actual values of point estimates (and relevant differences) • Note actual values of interval estimates (and relevant differences) • SE, SD, CI? • Context matters • How you interpret CI on individual group/condition means depends on experimental design

  12. Independent Measures • Difference CI can be inferred from mean CIs • Always larger • If group CIs are similar, larger by factor of √2

  13. Dependent Measures • Difference CI cannot be inferred from mean CIs • Depends on correlation • Usually smaller than mean CIs (r is usually positive and large)

  14. Rules of Thumb: 95% CI • So long as nsare at least 10 and one error is not greater than twice the other: • If proportion overlap is .5 or less, t-test is significant at  = .05 • If there is no overlap, t-test is significant at  = .01

  15. Rules of Thumb: SE • So long as nsare at least 10 and one error is not greater than twice the other: • If proportion gap is 1 or greater, t-test is significant at  = .05 • If proportion gap is 2 or greater, t-test is significant at  = .01

  16. CI Calculation

  17. Example

  18. Dependent Measures

  19. Additional Resources

  20. CIs and Replication • Given a sample mean and 95% CI, what is the probability that a repetition of that experiment, with an independent sample of participants, would give a mean that falls within the original CI? • Not .95 • Actually .834 • Why? • Probability that CI “captures” population mean depends on: • Deviation of sampled mean from population mean • Probability that CI “captures” replicant mean depends on: • Deviation of original (sample) mean from population mean • Deviation of replicant (sample) mean from population mean

  21. Power • The probability of correctly rejecting a false null hypothesis • The probability of not making a Type II error • Power depends on • Effect size • Sample size • Alpha Significance = Effect size  Sample size

  22. Prospective Power Analysis • Question: How many subjects do I need to have reasonable odds of getting a significant effect? • Calculation • Determine desired power • Usually .8 • Estimate effect size • Prior research • Theoretical considerations • Solve for n http://wise.cgu.edu/power/power_applet.html

  23. Retrospective Power Analysis • Question: If the true effect size was the one I found, what was the power of my experiment? • Calculation • Use obtained effect size and sample size • Controversial/wrong claim: If you had high retrospective power and nonsignificant results, the null hypothesis is probably true • It is not possible to have high retrospective power and reject the null • With p = .05, power = .5 • With p > .05, power < .5

More Related