Spending Public Money Wisely Scaling-Up Educational Interventions

1. Spending Public Money Wisely Scaling-Up Educational Interventions Barbara Schneider John A. Hannah University Distinguished Professor College of Education & Department of Sociology

2. The Question • How confident can teachers, principals, parents, district administrators, and state agencies be regarding the quality of evidence upon which school improvement plans and educational reform policies are based?

3. How confident can we be that reforms that work will be effective for students in different classroom settings?

4. What works well, for whom, under what conditions? • Evidence that informs decisions regarding which innovations can and cannot be expected to be similarly effective in a wide range of school and classroom contexts.

5. Background • Past six years studying these questions using information from 101 studies funded by the National Science Foundation that are designed to improve student learning in mathematics and science, teacher instructional practices, and professional development. The total programmatic costs are 223 million. The interventions are located in 31 states, three of which are in Michigan.

6. Volume 1: Ideas in Principle

7. Volume 1 – Brings together contributions from disciplines that routinely take promising interventions to scale including medicine, business, engineering, computing, & education.

8. Volume 2: Ideas in Practice

9. Volume 2 – Explores the challenges of implementing and assessing educational interventions in varied classroom contexts.

10. What we learned? How to design and conduct studies that provide compelling evidence How to implement a specific intervention with fidelity once it has been successfully tested

11. Definition of Scale Up Scale-up is taking innovations whose efficacy has been established in one context into a new setting and produce similar impacts in larger more diverse populations

12. Evidence Shows that: • Student achievement in science, mathematics and reading can be improved • Teacher training and professional development can be executed effectively and result in improved student performance • Assessment and assessment driven instruction can be successfully implemented • Innovative technologies can be developed and implemented throughout the curriculum

13. Successful Interventions Require Sequential Testing Phases Begin with Proof of Concept

14. I. What is the evidence for a Phase 1 Intervention? • Importance of design • Value of quasi-experimental studies • How large were the effects?

15. II. Implementing A Phase 2 Intervention • Importance of partnerships • Providing incentives • Clear and logical administration • Procedures for discussing treatment with parents, community, and students

16. Monitoring the implementation • Carefully receiving the effects of intervention {statistical versus feeling good} • Measuring outcomes at various times throughout the year • Obtain pre- and post-test results

17. III. Scaling-Up to a Phase 3 Intervention • Estimate probable effects of the intervention for different populations including costs • Repeat monitoring, implementation, and measurement of phase 2