1 / 17

Reliability and Validity

Reliability and Validity. Threats to Internal Validity. Da Lee Caryl, Fall 2006. Internal Validity. Defending against sources of bias arising in research design.

josterberg
Download Presentation

Reliability and Validity

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Reliability and Validity Threats to Internal Validity Da Lee Caryl, Fall 2006

  2. Internal Validity • Defending against sources of bias arising in research design. • When there is lack of internal validity, variables other than the independent(s) being studied may be responsible for part or all of the observed effect on the dependent variable(s).

  3. Suppose: • An experimenter observes an increase in the probability of salivating after presenting the smell of delicious food • Leads to the conclusion that the smell of food produces salivation. • For this conclusion to be internally valid, the experiment must be designed so that conditions other than the food smell are ruled out as potential causes for salivation. • Does the sight of food along with the smell of food cause salivation?

  4. Threats to Internal Validity • Selection • Experimental Mortality • Location • Instrumentation • Testing • History • Maturation • Attitude of Subjects • Regression • Implementation

  5. Selection Participants in a study may have different characteristics and those differences may affect the results. Such as: Age Intelligence Vocabulary Strength Attitude Fluency Maturity Fluency Ethnicity Coordination Speed Socioeconomic status

  6. Experimental Mortality Loss of subjects from the study due to: • Illness • Family relocation • Requirements of other activities • Absent during collection of data or fail to complete tests

  7. Location Place in which data is collected, or an intervention is carried out, may influence the results.

  8. Instrumentation Inconsistent use of the measurement instrument. • Instrument Decay – instrument changes in some way • Data Collector Characteristics – affects the nature of the data they obtain. Such as gender, age, ethnicity, age, language patterns, etc. • Data Collector Bias – unconsciously influence the outcome of the data

  9. Testing Pretests may influence the result of the posttest. A group may perform better on a posttest because the pretest primed them to perform better.

  10. History Occurrence of events that could alter the outcome or the results of the study • Previous – events that occurred prior to the study • Concurrent – happening during the study

  11. Maturation Any changes that occur in the subjects during the course of the study that are not part of the study and that might affect the results of the study; such as changes due to aging and experience.

  12. Attitude of Subjects • Subjects opinion and participation can influence the outcome. • Observing or studying subjects can affect their responses a.k.a. Hawthorne effect. • Subjects receiving experimental treatment may perform better due to “receiving” treatment. • Subjects in the control group may perform more poorly than the treatment group.

  13. Regression Groups that are chosen due to performance characteristics, either high or low, will on the average score closer to the mean on subsequent testing regardless of what transpires during the experiment. Example: A group of students who score low on a mathematics test are given additional help. Six weeks later they are given an exam similar problems on another test, their average score has improved. Is it due to the additional help or other influences?

  14. Implementation Personal bias in favor of one method or another. Preference for the method may account for better performance by the subjects. Individuals assigned to implement different methods, differ in ways related to the outcome.

  15. Methods to Minimize Threats • Standardization of the conditions under which the research study is carried out will help minimize threats from history and instrumentation. • Obtain as much information as possible about the participants in the research study minimizes threats from mortality and selection. • Obtain as much information as possible about the procedural details of the research study, for example, where and when the study occurs, minimizing threats from history and instrumentation. • Choose an appropriate research design which can help control most other threats.

  16. Journal of Agriculture Education Miller, Greg & Pilcher, Carol(2002). Can Selected Learning Strategies Influence the Success of Adult Distance Learners in Agriculture? Journal of Agricultural Education, 43,34-43. Retrieved October 15, 2006 from the World Wide Web: http://pubs.aged.tamu.edu/jae/pdf/vol43/43-02-34.pdf

  17. Bibliography Fraenkel, J.R. & Wallen, N.E. (2006). Internal Validity.How to design and evaluate research in education (6th ed.,pp.168-186). New York: McGraw Hill. Martin, Wendy (1997). Single Group Threats to Internal Validity. Retrieved October 15, 2006 from the World Wide Web: http://www.socialresearchmethods.net/tutorial/Martin/intval1.html. Trochim, William (2006). Internal Validity. Retrieved October 15, 2006 from the World Wide Web: http://www.socialresearchmethods.net/kb/intval.html.

More Related