Controlling extraneous variables

1. Controlling extraneous variables How to recognize ’em, and how to fix ’em

2. Extraneous variables • Extraneous variables are all variables other than the Ivs and the DVs. • Some extraneous variables threaten the internal validity of a study by varying systematically along with the independent variable(s). These co-varying EVs are called confounding, because they confuse, mix up, or confound the explanation.

3. Control of extraneous variables • Potentially confounding variables may be controlled by procedures which ensure that they do not co-vary with the IV. • 1. Keep extraneous variables constant. • 2. Keep the influence of evars constant. • 3. Randomly distribute the influence of evars.

4. Theoretical threats • Campbell and Stanley’s list of categories of extraneous variables: Threats to internal validity: • History: Extraneous events which occur during the course of an experiment. [Not just in repeated measures designs] • Maturation: Changes within the individual which occur during the course of the study, as a function of the passage of time.

5. More theoretical threats • Testing carryover: In repeated measures studies, having been measured once with a test affects how one responds at the posttest. • Instrumentation: Changes in the measurement of the dependent variable during the course of the study, as a function of the passage of time. • Statistical regression: Similar to testing carryover, except that regression is the phenomenon that extreme scores change more from pretest to posttest than do average scores.

6. Still more threats • Selection bias: Non-random assignment of participants to groups • Ultimately, this is an ex post facto design • Interaction of selection bias with other threats • Mortality: Loss of group equivalence through the unequal loss of participants from groups. • Thus, Campbell and Stanley list eight threats to internal validity.

7. Participant and experimenter variables • Participants are active, not passive • Participant demand characteristics • Positive self-presentation motive • Greatest when assessing the participant’s true intentions, beliefs, or feelings • Minimal if attributions are global and external • Behavior systems analysis is necessary

8. Positive self-presentation • Intertreatment interaction: What to do to look good depends on the experimental condition. • Intratreatment interaction: What to do to look good depends on the individual participant, or even on the phase of the experiment, within an experimental condition.

9. Experimenter variables • Motives and knowledge • Experimenter attributes • Biosocial attributes: Age, gender, race, religion • Psychosocial attributes: Anxiety, need for approval, dominance, intelligence • Situational attributes: Experience with research, reaction to participant attributes, laboratory cues • Are these variables biasing?

10. Experimenter expectancies • Effects on researcher: • Data collection • Data recording and analysis • Data interpretation: “Practically any set of data can be interpreted in different ways, depending on the orientation of the person doing the interpreting” (Christensen, 1997, p. 250). Cf. our archival research.

11. Experimenter expectancies • Effects on participants: • Interaction with positive self-presentation motive • Most obvious in animal studies • Means of influencing participants: • Recording errors -- Nonverbal cues • Intentional biases -- “Social influence” • Expectancy effects can exceed treatment effects

12. Sequencing effects • Within-participant sequencing effects: • Carry-over effects • Practice effects: Learning • Between-participant sequencing effects: • Order effects • Counterbalancing