1 / 20

Research Design and Validity Threats

This resource provides an overview of research design categories, including quantitative and qualitative approaches. It explores various types of research designs and their strengths and weaknesses, as well as threats to internal and external validity. The information presented helps researchers evaluate the quality of research studies and draw informed conclusions.

kpoole
Download Presentation

Research Design and Validity Threats

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Research Design and Validity Threats

  2. Evaluation Designs • Quantitative versus Qualitative • Combination

  3. Categories of Research Designs • Nonexperimental or Pre-experimental designs • One group, little validity control • Quasi-experimental • Experimental and comparison group, but no random assignment or selection • Experimental • Random assignment of experimental and control groups

  4. Terminology • Internal validity • Extent to which an observed outcome can be attributed to a planned intervention • External validity • Extent to which an observed outcome can be attributed to a replicable intervention and generalized to other settings and populations

  5. Internal validity threats • History • An event that occurs during the intervention that could have an impact on the results • Maturation • Bias from biological, natural, or social events that can bias results

  6. Internal validity threats • Testing • Testing might cue a person in to change behavior, regardless of the program • Instrumentation • Bias in data collection instruments

  7. Internal validity threats • Statistical Regression • Bias from selecting a group with unusually high or low scores on something • Selection • Comparison groups are unequal

  8. Internal validity threats • Attrition/subject mortality • Dropouts of subjects; if there is more than one group, then unequal dropouts between groups • Interactive effects • Combinations of the above

  9. Other internal validity issues • Diffusion • Contamination of comparison condition or intervention condition • Demoralization • Subjects upset they are not receiving the other condition

  10. External Validity • Threats • Social desirability • Expectancy effect • Hawthorne effect • Placebo effect • Novelty effect

  11. Research Designs • Key to abbreviations: O = data collection X = treatment/intervention R = random assignment Solid line separating groups – equal groups Dashed line separating groups – unequal groups

  12. Pre-experimental designs • One group, pretest, post-test O X O • Good for pilot testing • Does not control for IV threats • Be certain to use valid and reliable instruments

  13. Pre-experimental designs • One shot case study X O • No control for validity threats, no pretest measures

  14. Quasi-experimental designs • Nonequivalent comparison group O X O - - - - - O O • A comparison group is added, but they are not equal

  15. Quasi-experimental designs • Time series O O O O X O O O O • Several measures to assess if there is a trend • No comparison group

  16. Quasi-experimental designs • Multiple Time series O O O O X O O O O - - - - - - - - - - - - - - - - - - O O O O O O O O • Added a comparison group

  17. Experimental Designs • Pre-test, post-test, control group design R O X O R O O • Randomly assigned to groups

  18. Experimental Designs • post-test only, control group design R X O R O • Randomly assigned to groups

  19. Experimental Designs • Solomon Four Group Design R O X O R O O R X O R O

  20. Why do you care? • Having this knowledge will help you determine the quality of research studies, which will impact your conclusions regarding the results.

More Related