Preliminary Highlights from the Noyce National Program Evaluation May 30, 2013

1. Preliminary Highlights from the NoyceNational Program Evaluation May 30, 2013 Ellen Bobronnikov Cris Price

2. Study Overview • Implementation study examines the Noyce Program across awards from PI, Faculty, Recipients, and K-12 Administrators’ perspectives • Data sources: Web surveys and interviews • Impact study assesses program’s impact on teacher certification and employment in high-need districts • Data sources: Teacher certification and employment data from 5 states and annual “monitoring” data entered by PIs

3. Selected Implementation Study Research Questions • What activities do teacher preparation programs use to prepare and support Noyce recipients? • What are STEM faculty responsibilities for preparing K–12 mathematics and science teachers? • What activities do Noyce recipients engage in?

4. Noyce Has Funded Recipients Across the Country

5. Number of Recipients First Supported Each School Year

6. Results: Activities to Prepare and Support Noyce Recipients • Noyce recipients in teacher preparation, including TFs, enrolled in same range of courses available/required of all students in teacher preparation • Additional activities were available to Noyce recipients • Supports for Noyce recipients who were teaching included mentoring, PD, courses, conference support • Supports for MTFs included leadership training and educational resources • Supports for interns included hands-on experience with K–12 students, courses to introduce them to teaching, and internships in various settings

7. Results: STEM Faculty Involvement with Noyce • After receiving the Noyce award… • Over half of faculty/PI respondents reported increased STEM faculty member engagement in training STEM K–12 teachers • About a quarter of STEM faculty respondents reported changes in their teaching due to the Noyce Program (e.g., focus on active learning, adapting course content to needs of teachers)

8. Results: Recipient Experiences While in Teacher Prep • Most recipients decided to enter K–12 teaching during or after college • Just under half decided to teach in high-need districts at the time of application to Noyce • Almost all completed student teaching in a STEM subject area, and 80% taught in a high need district • Majority of recipients reported that they felt adequately prepared for teaching responsibilities

9. Results: Recipient Experiences While Teaching • Induction supports were primarily received in the first year of teaching, other than support for conferences, which was received throughout • Common leadership roles held by Noyce teachers included mentoring, committee service, departmental leadership, advising student/school organizations • Almost all Noyce teachers reported that they plan to complete their Noyce teaching obligation and continue teaching science/ math in a high-need school district

10. Results: TF Experiences • TFs must be in a Master’s Program and are required to teach in a high-need district for 4 years while receiving a salary supplement • Four-fifths of TFs reported that they had decided to teach some time during or after college; one-third were career changers • Almost all TFs indicated that their teacher prep program used a cohort model and included K-12 teachers as science/math mentors; half of TFs participated in a PLC during their first year teaching

11. Results: MTF Experiences • MTFs are required to teach in a high-need district for 5 years while receiving a salary supplement • MTFs reported involvement in leadership activities prior to Noyce, but generally reported involvement in more leadership activities post-Noyce • The majority of MTFs planned to retain both classroom AND leadership responsibilities; fewer indicated plans to assume primarily leadership roles or to shift to higher education

12. Results: Intern Experiences • One-third of respondents who had applied for the Noyce internship had not considered teaching prior to learning about Noyce • Common internship settings included math/science camps, research labs, schools, or museums • About half of intern respondents indicated their interest in workingas a K–12 teacher increased after participating in the Noyce internship Now, a transition to impact study …

13. Impact Study Research Questions • Does an IHE’s receipt of a Noyce grant affect its production of graduates who are certified by their state to teach STEM content? • Does an IHE’s receipt of a Noyce grant affect its production of certified STEM teachers who take teaching jobs in high-need schools? • What is the impact of Noyce on student achievement?

14. Years to Certification and Teaching • Among allrecipients: • Nearly two-thirds had received their teaching certificate • Among those who had enough time to complete program and earn certifications (2+ years from first receipt of support): • 83% have been certified to teach • Among recipients who have had at least 2 years to find a teaching position after certification: • 90% had taught in high-need districts, in fulfillment of their service requirement

15. Program is being implemented as intended… • Most recipients received STEM certifications and entered teaching in high-need schools (according to monitoring data) • However, this alone does not answer the question: “Would recipients have earned STEM certifications and taught in a high-need schools in the absence of the program?”

16. Preliminary Findings from Teacher Impact Study • Two of the 5 study states had significant positive impact estimates on STEM certification and employment in high-need schools • Impact represents an additional 4-5 teachers per IHE per impact year, about what we would expect from monitoring data • Two study states had impact estimates that were not significantly different than zero • One study state had a large significant negative impact estimate

17. Teacher Impact Study Limitations • All study states except one had small numbers of Noyce IHEs that graduated recipients early enough to be included in state datasets • Year-to-year variation in numbers of recipients who were certified and/or employed within IHEs was often large relative to the expected size of the impact per IHE per year • Impact estimates are NOT from a randomized study • The quasi-experimental comparative short interrupted time series approach that was used to estimate impacts depends on models and assumptions

18. Next Steps • Examine state contexts to better understand the mixed findings in the teacher impact study. • Collect certification and employment data from another large state to include in teacher impact study • Collect an additional year of data from some study states • Conduct student impact analyses in 3 districts • Prepare dissemination report