How Many Discoveries Have Been Lost by Ignoring Modern Statistical Methods?

1. How Many Discoveries Have Been Lost by Ignoring Modern Statistical Methods? Rand R. Wilcox

2. The theme....... • Despite what we learn, standard methods are NOT robust to violations of normality • Heteroscedasticity • Skewness • Outliers => Reduce chances of detecting true differences & obtaining accurate confidence intervals

3. Alternatives to the Mean: • Need an estimator that performs as well as the mean under normal conditions AND is robust to departures from normality • 4 options: • 10%t trimmed mean • 20% trimmed mean • Μm – Mean estimator by some chap called Huber. • Ө.5 – Median estimator by some chaps called Harrell & David

4. Dealing with Outliers: • Sample mean & sample SD are inflated by outliers => masks them • Trimming is not simply “throwing” data away and applying standard methods • This is a bad idea! If you take out extreme values and then continue => use of the wrong SE.

5. Skewness:

6. How much trimming & what to choose? • Rule of thumb = 20% • Trimmed means tend to perform better that M estimators in more situations; M estimators are better with correlation & regression

7. Why can’t we just test normality and then decide? • Because conventional tests are insensitive..... • Only way to determine if modern methods are useful is to use them • Modern methods can be extended to more complex designs as well; including multivariate analyses

8. Correlation: • Pearson’s r is not resistant to outliers; modern methods/alternatives can help e.g. Kendall’s Tau & Spearman’s rho • Percentage Bend correlation: • Population value of assoc is zero under independence (unusual apparently) • Good control over type I error in broad range of situations • Allows flexible choice re: how many outliers can be handled

9. Regression: • OLS = poor choice for researchers; SE can be more than 100 times larger than some modern methods! • Recommends a bootstrap method in conjunction with a robust estimator e.g. S-PLUS function regci • Critics argue that robust regressoin estimators fail to check for curvature of the line – this can be fixed by using a “smoother”

10. An example.....

11. An example (2):

12. Conclusions: • Use of trimmed means and funky modern methods is recommended • Education in psychology should reflect modern advances in stats • Not all problems are solved, but you could be missing something really important due to the vulnerability of standard methods to minor departures from normality.