Experimental Research

Experimental Research

Experimental Research • What is experimental research? • Research investigation in which conditions are controlled so that hypotheses can be tested and alternative explanations can be ruled out • Research used to make “cause-and-effect” statements (X causes Y) • X is the independent (or manipulated or causal) variable • Y is the dependent variable

Experimental Research • Evidence of causality (i.e., X causes Y) • Evidence of concomitant variation (X and Y co-vary) • The more of X, the better chance that we will get Y • Time order sequence of variables • If X causes Y, X must occur before Y • Elimination of other possible explanations of why Y occurred • Key is to keep all experimental conditions equal

Experimental Research • Must try to eliminate all possible extraneous influences • Possible extraneous influences • History - event occurring during the course of an experiment (but not really part of the experimental manipulation) that influences the results • Maturation - changes which occur in the experimental unit which occur during the course of an experiment that are due to the passage of time

Experimental Research • Possible extraneous influences (continued) • Testing Effects - changes in experimental unit due to the experiment itself (but not related to the key experimental manipulation • Measurement effect -- prior measurements effect the measure of the dependent variable • Interaction effects -- subject pays greater attention to stimuli than they normally would because its and experiment

Experimental Research • Possible extraneous influences (continued) • Instrument variation -- changes in the measuring instrument cause changes in Y (i.e, interviewer changes the way in which they ask questions) • Selection Bias -- if two groups are compared, this notion suggests that the groups may not have been equivalent to begin with; i.e., a manipulation did not cause Y; group dissimilarities did

Experimental Research • Possible extraneous influences (continued) • Experimental mortality -- respondents in experimental groups were lost during the duration of the experiment • Statistical Regression -- Extreme responses move closer to the midpoint during the course of an experiment; subjects after questionning do not want to appear to be extreme

Experimental Research • Experimental Symbols • X = Experimental Treatment (e.g., ad exposure) • O = Observation • R = Random Assignment of Subjects • Pre - Experimental Designs • One-Shot Case Study • X O • Problems -- Selection Bias, Control Group, Mortality, History, etc.

Experimental Research • Pre-Experimental Designs (Cont’d) • One Group Pre-Test/Post-Test Design • O1 X O2 • Analysis -- O2 - O1 • Problems -- History, Maturation, interactive Testing Effect, etc. • Static Comparison • X O1 O2 • Analysis -- O2 - O1 • Problems -- Selection Bias, Mortality, etc.

Experimental Research • True Experimental Designs • Before/After with Control Group • (R) O1 X O2 • (R) O3 O4 • Analysis (O4 –O3) – (O2 – O1) • Lessens History and Maturation Problems • However, interactive testing effect still an issue • O2 and O4 may a function of O1 and O3

Experimental Research • Experimental Designs (True Experimental Designs) • Six Group - Four Study Design • (R) O1 X O2 • (R) O3 O4 • (R) X O5 • (R) O6 • O2 – O1 compared to O4 – O3 tells if X “worked” (manipulation check) – If yes, go on • If O5 > O6 –we havetrue effects

Experimental Research • True experiments are very complex • Experimental designs that are used are often flawed (or not “True”) • Random assignment helps

Experimental Research • Analyzing Data • Typically uses Analysis of Variance (ANOVA) • Tests (determines if at least one treatment group mean is different from the others) • Ho: Μ1 = M2 = M3 = … = Mk • Ha: M1≠ M2≠ M3≠ … ≠ Mk • Why not use t-tests? • Probability • One comparison at 95% confidence – 95% of rejecting null when it should be rejected • Two comparisons -- .95 x .95 = 90.25% • Three comparisons -- .95 x .95 x .95 = 86% • Like regression • Dependent Variable (interval or ratio scaled) • Independent Variable (nominal or ordinal scaled) • Often group membership

Experimental Research (Responses on 1 = Bad; 7 = Outstanding)

Experimental Research • One-Way ANOVA • Attempts to partition variance • Total Variance (Within Group + Between Group) • Within treatment groups (Observation – Group Mean) • SSw = Σ Σ (x ij – Meani)2 • MSw = SSw / df w • Between treatment groups (Group Mean – Grand Mean) • SSb = Σ nj (Meanj – Meani) 2 • MSb = SSb / dfb • F-statistic • MSb / MSw • With dfb,; df w • Follow-up test • Tukey (most common in MR)

Experimental Research • Additional Issues with ANOVA • Real World • Often used with survey data • Compare means of several groups • DV – is interval (or ratio) scaled • IV -- categorical • Two-Way ANOVA • One dependent variable • Two or more independent variables • Simultaneous effects • MANOVA • Multiple dependent variables • ANCOVA • Investigate effects after “controlling” for another variable (that is interval or ratio scaled)

Experimental Research • Compare means of several groups • EXAMPLE (Survey Data)

Experimental Research • DV = Parent Responsibility to Restrict TV Choices for Kids • F= 4.42; p = .005 • Neglecting (M = 29.5) • Indulgent (M = 31.26) • Authoritarian (M = 30.26) • Authoritative (M = 32.27) • Follow-up Tests • Authoritative > Authoritarian & Neglecting • Indulgent > Neglecting • Authoritarian = Neglecting

Experimental Research • Two-Way ANOVA – Example • Main Effects • Similar to one-way ANOVA effect (for 2 variables) • Interactive Effects • Main Effects look null • Differences observed only when looking at both factors simultaneously

Two-Way ANOVA One dependent variable – Main Effects P N Named Not-Named Experimental Research

Two-Way ANOVA One dependent variable – Interaction Effects N P Named Not-Named Experimental Research

Experimental Research