1 / 31

Evidence-Based Practices: Scientific Procedures to Guide Instruction

Evidence-Based Practices: Scientific Procedures to Guide Instruction. A module for pre-service and in-service professional development MN RTI Center Author: Ann Casey, Ph.D. www.scred.k12.mn.us click on RTI Center. MN RTI Center Training Modules.

amycummings
Download Presentation

Evidence-Based Practices: Scientific Procedures to Guide Instruction

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Evidence-Based Practices: Scientific Procedures to Guide Instruction A module for pre-service and in-service professional development MN RTI Center Author: Ann Casey, Ph.D. www.scred.k12.mn.us click on RTI Center

  2. MN RTI Center Training Modules This module was developed with funding from the MN legislature It is part of a series of modules available from the MN RTI Center for use in preservice and inservice training: 2

  3. Overview • Why do we need to know about evidence based practices? Education is a fad-ridden profession • 3 levels of evidence based practice • Criteria to judge whether a practice is research-based • Evaluating research DRAFT May 27 2009

  4. Education is a fad-ridden profession • We mistake opinion for fact • Something written in a professional journal may be research but often is just opinions of the author • Theories are just that - one has to do the research to prove or disprove a theory DRAFT May 27 2009

  5. What constitutes evidence (research) that a practice is effective? • Ellis & Founts classification system (1994) • Level 1 , Basic Research • Level 2, Test of theory in real classroom • Level 3, Program evaluation or large scale data collection DRAFT May 27 2009

  6. Level 1 - Basic Research • Descriptive studies and qualitative studies are level 1 • Often use correlations to show the relationship between variables under study (e.g relationship between achievement and _______) • Correlational data can not disprove a theory - instead correlations suggest the degree of relationship between variables DRAFT May 27 2009

  7. Level 2 - classroom application • Use your hypothesis (theory) about what might be effective and try it out in a classroom • In the best controlled circumstances, you have students/teachers randomly assigned to the intervention group and to a control group who doesn’t get the intervention - then compare results DRAFT May 27 2009

  8. Level 3 - Large scale trials • Evidence that a particular intervention works not just in one school or with certain types of students but works in a wide variety of settings (replication) • Level 3 research is the kind of data we need to feel confident in the interventions we use with students DRAFT May 27 2009

  9. Fads - jump from Level 1 right into mainstream acceptance • Examples • Theory of Multiple Intelligences • Integrated curriculum • Block scheduling • Piaget • Learning styles DRAFT May 27 2009

  10. Criteria to judge whether a practice is research-based • IRA recommendations • Objective, 2 people come to same conclusion about the data • Valid, data represent tasks we expect children to accomplish • Reliable, data remain unchanged if collected on different day or by different person • Systematic, data collected according to rigorous design of experimentation or observation • Refereed, data that have been approved for publication by a panel of independent reviewers DRAFT May 27 2009

  11. What we need to do • Encourage our teachers to use curriculum and instruction that have evidence to support their use • Discourage teachers from picking and choosing based on their personal preference • Grossen, “To be a profession is to have a professional-knowledge base comprised of shared procedures that work.” DRAFT May 27 2009

  12. Evaluating Research • Theoretical framework. • What was the theoretical basis of the program being considered? Is it related to learning theory and best practices in the field of study [e.g. reading?]
 • Research design. • Did the study have a control and an experimental group? Are there other reasons they may have gotten these findings? DRAFT May 27 2009

  13. Evaluating Research Continued • Research methodology. • What was the treatment for the experimental group? What activities were implemented? How many schools were in the study? In what settings did the research take place? Was the context similar to our school demographics? • Implementation and replication. • Is the treatment program understandable so that it can be replicated in our school? What kinds of resources were used in the study, and do we have the capacity to provide them? DRAFT May 27 2009

  14. Evaluating Research Continued • Evidence of results. • What assessment data were used to measure the treatment? If achievement testing, how comparable is this to our assessment system? Has the hypothesis been adequately tested in order to justify the conclusions? What statistical measurements were used, and do we have the capability to replicate them? DRAFT May 27 2009

  15. What is Quality Evidence? DRAFT May 27 2009

  16. Evaluating Research Continued • Approved research. • To what extent has the research been accepted by a peer-reviewed journal or approved by a panel of independent experts? Is the research considered rigorous, objective, and scientific to meet the SBR criteria? • After completing these processes, the school leaders make a commitment to translate specific, applicable research into practices in their own setting. With these steps, the school has begun to incorporate research into its culture and to increase the likelihood of strengthening teaching and improving learning. DRAFT May 27 2009

  17. What makes a journal article research/evidence based? • It has an introduction to the topic • It has a subheading referred to as the Method or Procedures section - this section tells you exactly everything you need to know about how they implemented the intervention DRAFT May 27 2009

  18. What makes a journal article research/evidence based? Continued • Another subheading entitled Results - where you’ll find the tables, graphs and statistical results • Final subheading entitled Discussion section - this is where the author gets to say what they found out and why they think it turned out the way it did (their interpretations of the study.) DRAFT May 27 2009

  19. Accessing reviews of programs done by other people… What Works clearinghouse http://ies.ed.gov/ncee/wwc/ Best Evidence Encyclopedia http://www.bestevidence.org/ Florida Center for Reading Research review of programs http://www.fcrr.org/FCRRReports/reportslist.htm DRAFT May 27 2009

  20. Be a smart consumer of these reviews of programs DRAFT May 27 2009 • Be sure you dig into the information enough to know what the intervention program is focused on or “good at” • Even if it is a well reviewed program, if it target comprehension and your students need an intervention that targets phonics it wouldn’t be a good choice • Different sites (e.g., what works vs. best evidence vs. fcrr) use different criteria • All of these sites use reasonable criteria, but they are not identical so you can get differing reviews. If that happens you can dig in and figure out why by reading the criteria or reading the reviews in more depth • If a program was not reviewed it doesn’t mean it is bad, just that it was not reviewed!

  21. What else would you do? Look for the research yourself Google Scholar, ERIC, PsychInfo, EBESCO, etc. Use guidelines from Horner et al. 2005 or Gersten et al. 2005 to guide your decisions about whether the research was well done and look for different levels of research (basic, classroom, larger scale) Look at individual effectiveness with your students- collect progress monitoring data to show if it is working or not! DRAFT May 27 2009

  22. Activity: Using program review internet sites 1. Compare and contrast the information (or lack) on what works, best evidence and fcrr sites on at least 2 of the following reading interventions: Read Well, FastForword, Earobics, Read Naturally, PALS (from Vanderbilt) 2. If the intervention was not reviewed or if the reviews were conflicting, go to direct sources (ERIC, EBESCO, etc) and see if you can find any actual research studies on the intervention. 3. Would you recommend the interventions you examined as “evidence based”? why or why not? If so, for what students/skill focus? DRAFT May 27 2009

  23. Articles included with this module • Institute of Education Sciences (ED), Washington, DC. (2003). Identifying and implementing educational practices supported by rigorous evidence: A user friendly guide. Retrieved from ERIC. • International Reading Association, Newark, DE. (2002). What is evidence-based reading instruction? A position statement of the international reading association Retrieved from ERIC database. • Lembke, E. S., & Stormont, M. (2005). Using research-based practices to support students with diverse needs in general education settings. Psychology in the Schools, 42(8), 761-763. Retrieved from: http://dx.doi.org/10.1002/pits.20110. DRAFT May 27 2009

  24. Articles included with this module, Cont’d • Gersten, R., Fuchs, L. S., Compton, D., Coyne, M., Greenwood, C., & Innocenti, M. S. (2005). Quality indicators for group experimental and quasi- experimental research in special education. Exceptional Children, 71(2), 149. For full text: • Horner, R. H., Carr, E. G., Halle, J., McGee, G., Odom, S., & Wolery, M. (2005). The use of single-subject research to identify evidence-based practice in special education. Exceptional Children, 71(2), 165-179. For full text: DRAFT May 27 2009

  25. Articles included with this module • National Inst. for Literacy, Washington, DC., & RMC Research Corp., Arlington, VA. (2005). What is scientifically based research? A guide for teachers. National Institute for Literacy. Retrieved from ERIC. • Stanovich, P. J., & Stanovich, K. E. (2003). Using research and reason in education: How teachers can use scientifically based research to make curricular & instructional decisions. Retrieved from ERIC database. • Try to find and include pdf of the Gersten and the Horner articles. DRAFT May 27 2009

  26. References The Access Center. (2009). K8 access center.org: Electronic references.  Retrieved (what date should I put here?) from http://www.k8accesscenter.org/index.php Ellis, A., & Fouts, J. (1994). Research on school restructuring. Princeton, NJ: Eye on Education. DRAFT May 27 2009 26

  27. Resources • www.rti4success.org • www.rtinetwork.org • http://ies.ed.gov/ncee/wwc/ • http://www.bestevidence.org/ • http://www.fcrr.org/FCRRReports/reportslist.htm • http://www.nectac.org/topics/evbased/evbased.asp DRAFT May 27 2009

  28. Quiz Question: Essay Questions 1. List and explain the 3 levels of research 2. If you were asked to find a “research based” intervention for students with decoding problems, what would you do? And what criteria would you use to determine if the program/intervention was truly research based? DRAFT May 27 2009

  29. Quiz, Cont’d 3. Level 1 basic research is (a) A. Coming up with a hypothesis and doing a lit review B. Descriptive Study C. Research that shows causation 4. Which of the following is not a helpful way to judge whether a practice is research-based A. Research on it has to be published in a refereed journal B. Research on it has to be objective C. The research data must have been collected with rigorous designs of experimentation/observation D. It has to have testimonials discussing whether it was useful to teachers or not

  30. Quiz, Cont’d 5. Level 2 classroom application is not A. Preferably done in a controlled setting B. Testing your hypothesis in an actual classroom C. Done in a lab setting 6. T or F: When looking at internet sites that review different curriculum or intervention programs, If a program is not reviewed you should assume it is not good. 7. Level 3 large scale trials are A. Important for us to feel confident in the intervention. B. Evidence that the intervention works in more than one setting. C. Replication D. All of the above

  31. Note: The MN RTI Center does not endorse any particular product. Examples used are for instructional purposes only. • Special Thanks: • Thank you to Dr. Ann Casey, director of the MN RTI Center, for her leadership • Thank you to Aimee Hochstein, Kristen Bouwman, and Nathan Rowe, Minnesota State University Moorhead graduate students, for editing work, writing quizzes, and enhancing the quality of these training materials

More Related