Power (Reading Packet Sect III)

1. Power (Reading Packet Sect III) Mon, March 29th

2. Power • The ability of your statistical test to correctly reject the null hypothesis • Remember, the null hypothesis generally states there is no effect or no differences between your sample & pop • Power refers to the power to find any differences that truly exist • Want to maximize power

3. (cont.) • When we fail to reject Ho, we want to be sure it is because there are truly no differences that exist (or no effect) • …Not because our statistical test didn’t have enough power to find a difference that is actually there. • In your 2x2 Decision Table, • power is = 1-probability of a Type 2 error

4. 2x2 Decision Table (Ho: No disease present)

5. What Influences Power? • 4 factors that affect power: • 1) Alpha level: increasing alpha (chance of Type 1 error), increases power • Going from alpha = .01 to alpha = .05, gives you larger chance of finding a difference/effect that is really there. • Consider effect on your critical region of .01 v .05 – region becomes larger by using alpha = .05, more likely to reject Ho

6. Example from last Wed (salary) • Compare ybar = $24,100 w/ y = $28,985 (and y = $23,335, N=100) • For 1-sample z test, z = (ybar - y) / ybar, where ybar = y / sqrt N), or ybar = 23,335/ sqrt (100) = 2333.5 • Z obtained = 24,100 – 28,985 / 2333.5 = -2.09 • Book’s approach  look up z = -2.09 in Appendix (Col C), find its probability = .0183 • If alpha = .05 (1 tail) use .0183, then p < ,so REJECT Ho

7. Ex (cont.) • Another approach is to find the critical z value associated with an alpha level: • When | z obtained | (from formula) > | z critical| (from table)  REJECT Ho  = .05, 1 tail, z critical = 1.645 or –1.645; • = .05 2 tails, z criticals = -1.96 and 1.96; • = .01, 1 tail, z critical = 2.33 or –2.33 • = .01, 2 tails, z criticals = 2.57 and –2.57

8. Ex (cont.) • Using this approach, z obtained = -2.09, z critical ( = .05, 1 tail) = -1.645, since obtained > critical  Reject Ho • Same conclusion as for p < alpha • Notice critical region becomes smaller as alpha level becomes smaller • And as you move from 1 tail to 2 tails • Harder to reject Ho w/ smaller critical region

9. Other Influences on Power • 2) Sample Size – larger N, more power • With larger sample size, more likely a representative sample with less error • 3) 1- v 2-tailed test – 1-tailed test has more power • The critical region for rejecting Ho is larger w/1-tailed test (don’t have to split into 2 tails) • 4) Effect size – larger effect size, more power • Refers to the effect of the manipulation (in an experiment) or the difference betw sample & pop

10. (cont.) • If you have a strong manipulation, will create larger differences among groups or betw sample & population – • easier to see the effect if it’s really there

11. Determining Power • Failure to reject Ho could be due to many things: • There really is no effect / no difference • Your study had low reliability, validity, etc. • Your study didn’t have enough power • How can we determine the amount of power of our study? • Can’t be set by you (as alpha can), but can calculate after the fact or try to predict before the experiment • Can calculate needed sample size for certain level of power (in adv stat class you’ll do this!)

12. Stat v Practical Significance • It’s also possible to have too much power! • With large enough sample sizes, you’ll have so much power that even very small differences & effects will turn out statistically significant • w/N=5,000, a difference between μ=3.4 and μ=3.5 on a 1-7 scale is significant, but is it important? Practical? • Pay attention to whether a statistically signif finding has any practical significance (is it meaningful? Important?)