Randomized Experimental Design

1. Randomized Experimental Design

2. What is an Experiment? • Campbell & Stanley stressed random assignment to experimental treatments. • I stress manipulation of the independent variable. • Quasi-Experiments: C&S’s term for research where • there is a manipulated IV • but not random assignment to groups

3. Random Selection • Refers to the selection of data points from a population into a random sample. • This selection procedure will be random if each possible sample of size N is equally likely to be sampled. • Random samples should be representative of the population. • Our inferential statistics assume random sampling.

4. Note. The tabled values are probabilities. • Y is random, X is not. • In Y, each time a single score is sampled, all scores in the population are equally likely to be sampled.

5. Random Assignment • Refers to the assignment of subjects to treatment conditions. • Allows us to consider the populations (subjects who will get special treatment and those who will not) as equivalent prior to treatment. • The samples will likely differ a little.

6. Two Basic Randomized Designs • Randomized Pretest-Posttest Control Group Design R  O  X  O R  O     O • Randomized Posttest Only Control Group Design R  X  O R     O

7. Noise Reducing Designs • These designs reduce noise (error variance) and thus increase power.

8. Randomized Blocks Designs • Matched pairs, randomized blocks, split-plot. • Repeated measures or within-subjects. • Variance due to the blocking variable is removed from error variance.

9. Analysis of Covariance • Change an noise-producing extraneous variable into a covariate that is included in the statistical model. • Must be able to measure the covariate. • Variance due to covariate is removed from the error variance. • Can have more than one covariate.

10. Factorial ANOVA • Convert a categorical extraneous variable to an ANOVA factor. • Variance due to that factor will be removed from the error term. 2 x 2 Factorial Design R  X11  O R  X12  O R  X21  O R  X22  O

11. Controls threats to internal and external validity as well as the posttest only control group design. But has greater power. And greater cost Need more data More complex analysis R  O  X  O R  O     O R     X  O R        O Other Randomized DesignsSolomon Four Group Design

12. Solomon Four Group DesignANOVA • Arrange all four groups’ posttest scores into a 2 x 2 ANOVA. • Pretested or Not x Experimental Treatment or Not. • Significant Interaction – Testing x Treatment threat to External Validity • Main effect of pretesting. • Main effect of treatment

13. Treatment effect but no testing or Testing x Treatment interaction Pretest means in parentheses

14. Treatment and testing effects but no Testing x Treatment interaction

15. Treatment and testing effects and a Testing x Treatment interaction

16. Solomon Four Group DesignPretest-Posttest Analysis • To gain power, analyze the pretest-posttest portion of the design with • ANCOV, using pretest scores as covariate • Mixed factorial ANOVA • planned comparisons using t • control versus treatment change scores (independent t) • pre versus post for control group (correlated t) • pre versus post for treatment group (correlated t)

17. Randomized Switching- Replications Design R  O  X  O     O R  O     O  X  O • Attempt to control social threats to internal validity. • Both groups get the special effect, one early, one later. • May still be social effects with respect to who gets it first.

18. Switching ReplicationsTemporary Treatment Effect • Group 1 got the treatment first. • Treatment is anxiety-reducing drug • DV = measure of anxiety reported by patient

19. Switching ReplicationsPersistent Treatment Effect • Treatment is psychotherapy • DV = measure of anxiety reported by patient • A third posttest could show the effect does not last indefinitely.

20. Switching ReplicationsContinuing Treatment Effect • Treatment = cognitive psychotherapy • Anxiety continues to decline beyond the first post-treatment observation, as patients get better at employing the cognitive technique.