ANALYSIS OF VARIANCE

ANALYSIS OF VARIANCE Chapter 1

t-test refresher • In chapter 7 we talked about analyses that could be conducted to test whether pairs of means were significantly different. • For example, consider an experiment in which we are testing whether using caffeine improves final marks on an exam. We might have two groups, one group (say 12 subjects) who is given normal coffee while they study, another group (say also 12 subjects) who is given the same amount of decaffeinated coffee. Chapter 1

t-test refresher We could now look at the exam marks for those students and compare the means of the two groups using a “between-subjects” (or independent samples) t-test: Chapter 1

t-test refresher Chapter 1

t-test refresher The critical point of the previous example is the following: The basic logic for testing whether or not two means are different is to compare the size of the differences between the groups (which we assume is due to caffeine), relative to the differences within the groups (which we assume is due to random variation .. or error). Chapter 1

t-test refresher This exact logic underlies virtually all statistical tests, including analysis of variance, an analysis that allows us to compare multiple means simultaneously. Chapter 1

Analysis of Variance (ANOVA) – the why? The purpose of analysis of variance is to let us ask whether means are different when we have more than just two means (or, said another way, when our variable has more than two levels). • In the caffeine study for example, we were interested in only one variable (caffeine) and we examined two levels of that variable, no caffeine versus some caffeine. • Alternately, we might want to test different dosages of caffeine where each dosage would now be considered a “level” of caffeine Chapter 1

Analysis of Variance (ANOVA) – the why? • As you’ll see in PsyC08, as you learn about more complicated ANOVAs (and the experimental designs associated with them) we may even be interested in multiple variables, each of which may have more than two levels. • For example, we might want to simultaneously consider the effect of caffeine (perhaps several different dose levels) and gender (generally just two levels) on test performance. Chapter 1

Analysis of Variance (ANOVA) – the what? The critical question is, is the variance between the groups significantly bigger than the variance within the groups to allow us to conclude that the between group differences are more than just random variation? Chapter 1

Analysis of Variance (ANOVA) – the what? Chapter 1

Analysis of Variance (ANOVA) – the how? The textbook presents the logic in a more verbal/statistical manner, and it can’t hurt to think of this in as manner different ways as possible, so, in that style: Let’s say we were interested in testing three doses of caffeine; none, moderate and high. Chapter 1

Analysis of Variance (ANOVA) – the how? First of all, use of analysis of variance assumes that these groups have (1) data that is approximately normally distributed, (2) approximately equal variances, and (3) that the observations that make up each group are independent. Given the first two assumptions, only the means can be different across the groups - thus, if the variable we are interested in is having an affect on performance, we assume it will do so by affecting the mean performance level. Chapter 1

Analysis of Variance (ANOVA) – the how? Chapter 1

Analysis of Variance (ANOVA) – the how? Mean = 78.83 72.08 67.92 s2 = 71.06 48.99 59.72 s = 8.43 7.00 7.73 Chapter 1

Analysis of Variance (ANOVA) – the how? From this data, we can generate two estimates of the population variance 2. “Error” estimate (σ2e): One estimate we can generate makes no assumptions about the veracity (trueness or falseness) of the null hypothesis. • Specifically, the variance within each group provides an estimate of σ2e. Chapter 1

Analysis of Variance (ANOVA) – the how? Given the assumption of equal variance (all of which provide estimates of 2), our best estimate of 2 would be the mean of the group variances. This estimate of the population variance is sometimes called the mean squared error (MSe) or the mean squared within (MSwithin). Chapter 1

Analysis of Variance (ANOVA) – the how? Treatment estimate (σ2t ): Alternatively, if we assume the null hypothesis is true (i.e., that there is no difference between the groups), then another way to estimate the population variance is to use the variance of the means across the groups. By the central limit theorem, the variance of our sample means equals the population variance divided by n, where n equals the number of subjects in each group. Chapter 1

Analysis of Variance (ANOVA) – the how? Therefore, employing some algebra: This is also called the mean squared treatment (MStreat) or mean squared between (MSbetween). Chapter 1

Analysis of Variance (ANOVA) – the how? OK, so if the null hypothesis really is true and there is no difference between the groups, then these two estimates will be the same: However, if the treatment is having an effect, this will inflate σ2τ as it will not only reflect variance due to random variation, but also variance due to the treatment (or variable). Chapter 1

Analysis of Variance (ANOVA) – the how? The treatment will not affect σ2e, therefore, by comparing these two estimates of the population variance, we can assess whether the treatment is having an effect: Measure of Chance Variance + Treatment Effect Measure of Chance Variance Only Chapter 1

Analysis of Variance (ANOVA) – the how? 1) Calculate a SSerror, SStreat, and SStotal. 2) Calculate a dferror, dftreat and dftotal 3) By dividing each SS by its relevant df, we then arrive at MSerror and MStreat (and MStotal). 4) Then we divide MStreat by MSerror to get our F-ratio, which we then use for hypothesis testing. Chapter 1

Sums of Squares The sum of squares is simply a measure of the sum of the squared deviations of observations from some mean: OK, so rather than directly calculating the MSerror and MStreat (which are actually estimates of the variance within and between groups), we can calculate SSerror and SStreat. Chapter 1

ANOVA Chapter 1

SSerror • To calculate SSerror, we subtract the mean of each condition from each score, square the differences, and add them up, and then add up all the sums of squares Chapter 1

SSerror There is a different way of doing this. First, calculate ΣX2for each group For example, for Group 1, the X2 would equal (722+652+….+882+712) = 71622. Once we have them, we then calculate the sum of squares for each group using the computational formula: Chapter 1

SSerror For example, for Group 1, the math would be: To get SSerror we then sum all the SSerrors. SSerror = SS1+SS2+SS3 = 781.67+538.92+656.92 = 1977.50 Chapter 1

SStreat • To calculate SStreat we subtract the grand mean from each group mean, square the differences, sum them up, and multiply by n. Chapter 1

SStreat • Again, there is a different way of doing this. Basically, all we need are our three means and the squares of those means. We then calculate the sum of the means, and the sum of the squared means: Chapter 1

SStreat Now we can calculate the SS using a formula similar to the one before: Once again, because we are dealing with means and not observations, we need to multiply this number by the n that went into each mean to get the real SStreat SStreat = 12(39.81) = 477.72 Chapter 1

SStotal • The sum of squares total is simply the sum of squares of all of the data points, ignoring the fact that there are separate groups at all. • To calculate it, subtract the grand mean from every score, square the differences, and add them up Chapter 1

SStotal • Surprise, surprise – there is another way of calculating this as well • Here you will need the sum of all the data points, and the sum of all the data points squared. An easy way to get this is to just add up the X and the X2 for the groups: X = X1+X2+X3 = 922+865+815 = 2602 X2 = X21+X22+X23 = 71622+62891+56009 = 190522 Chapter 1

SStotal Then, again using a version of the old SS formula: If all is right in the world, then SStotal should equal SSwithin+SStreat. For us, it does. Chapter 1

df OK, so now we have our three sum of squares, step two is to figure the appropriate degrees of freedom for each. Here’s the formulae: dferror=k(n-1) dftreat=k-1 dftotal=N-1 where k = the number of groups, n = the number of subjects within each group, and N = the total number of subjects. Chapter 1

From SS to MS to F MS estimates for treatment and within are calculated by dividing the appropriate sum of squares by its associated degrees of freedom. We then compute an F-ratio by dividing the MStreat by the MSerror. Finally, we place all these values in a Source Table that clearly shows all the steps leading up to the final F value. Chapter 1

ANOVA source table The source table for our data would look like this: OK, now what? Chapter 1

Hypothesis Testing Now we are finally ready to get back to the notion of hypothesis testing. . .that is, we are not ready to answer the following question: If there is really no effect of caffeine on performance, what is the probability of observing an F-ratio as large as 3.99. More specifically, is that probability less that our chosen level of alpha (e.g., .05). Chapter 1

Sampling distribution of F How do we arrive at the probability of observing some specific F value? Recall our example when we created 3 groups by randomly sampling individuals from the same population and asking them for some piece of data (e.g. age). In this case, the null hypothesis should be true … the means of the three groups should only vary as a result of chance (or error) variation Chapter 1

Sampling distribution of F If we perform an analysis of variance on this data, the F value should be about 1. However, it will not be exactly 1; rather, there will be a distribution with a mean of 1 and some variance around that mean. This distribution is termed the F distribution, and its exact shape varies as a function of dftreat and dferror. The important point here is that for any given degrees of freedom, the function can be mathematically specified, allowing one to perform calculus and, therefore, to find the probabilities of certain values. Chapter 1

Hypothesis Testing All we really want to know is whether the F we have obtained in our analysis is significantly larger than we would expect by chance. That is, we want to know whether it falls within the extreme “high” 5% of the chance distribution. Thus, all we really need to know is the critical F value that “cuts off” the extreme 5% of the distribution. If our obtained F is larger than the critical F, we know it is in the “rejection region” and, therefore, that the probability of obtaining an F that large is less than 5%. Chapter 1

Finishing the example From the table, Fcrit(2,33) = 3.32 Since Fobt (3.99) > Fcrit (3.32) we reject the null hypothesis Mean Fcrit = 1 = 3.32 Chapter 1

Finishing the example • One thing to keep in mind – all an ANOVA (significant) tells you is that there is a difference between the means. You can’t tell where exactly this difference lies just yet. That’s in chapter 12 – and PsyC08 Chapter 1

Violation of Assumptions The textbook discusses this issue in detail and offers a couple of solutions (including some really nasty formulae) for what to do when the variances of the groups are not homogeneous. What I want you to know is the following: 1) If the biggest variance is more than 4 times larger than the smallest variance, you may have a problem. 2) There are things that you can do to calculate an F if the variances are heterogeneous. Chapter 1

The Structural Model • Let’s assume that the average height of all people is 5’7”. Let’s also assume that males tend to be 2” taller than females, on average. • Given this, I can describe anyone’s height using three components: 1) the mean height of all people, 2) the component due to sex, and 3) individual contributions • My height is about 6’0”. I can break this down into: 5’7”+2”+3” Chapter 1

The Structural Model • In more general terms, we can write the model out like this: Chapter 1

Chapter 1

ANALYSIS OF VARIANCE