Leveraging Teacher Evaluation for Improving Teaching and Learning

1. Leveraging Teacher Evaluation for Improving Teaching and Learning • Laura Goe, Ph.D. • Research Scientist, ETS • Sr. Research and Technical Assistance Expert, Center for Great Teachers and Leaders • Georgia Title IIa Conference  March 2013  Savannah, Georgia

2. Laura Goe, Ph.D. • Former teacher in rural & urban schools • Special education (7th & 8th grade, Tunica, MS) • Language arts (7th grade, Memphis, TN) • Graduate of UC Berkeley’s Policy, Organizations, Measurement & Evaluation doctoral program • Research Scientist in the Understanding Teaching Quality Research Group at ETS • Senior Research & Technical Assistance Expert for the federally-funded Center for Great Teachers and Leaders

3. The goal of teacher evaluation

4. To be discussed… • Using teacher evaluation results to inform professional growth opportunities • Characteristics of high-quality professional development • Assessing impact of professional growth • Recruitment, retention, and distribution of effective teachers

5. Accountability vs. professional growth • Effective evaluation for accountabilityrelies on: • Clearly defined and communicated standards for performance • Quality tools for measuring and differentiating performance • Quality training on standards and tools • Evidence to make decisions

6. Accountability vs. Professional Growth • Effective evaluation for professional growth relies on: • Clearly defined and communicated standards for performance • Quality tools for measuring and differentiating performance • Quality training on standards and tools • Evidence to make decisions

7. Validity of measures key to PD aligned with evaluation results • Consider whether human resources and capacity are sufficient to ensure fidelity of implementation • Poor implementation threatens validity of results • Establish a plan to evaluate measures to determine if they can effectively differentiate among teacher performance • Need to identify potential “widget effects” in measures • If measure is not differentiating among teachers, may be faulty training or poor implementation, not the measure itself • Examine correlations among results from measures • Evaluate processes and data each year and make needed adjustments

8. Interpreting results for alignment with teacher professional learning options • Different approach; not looking at “absolute gains” • Requires ability to determine and/or link student outcomes to what likely happened instructionally • Requires ability to “diagnose” instruction and recommend/and or provide appropriate professional growth opportunities • Individual coaching/feedback on instruction • Observing “master teachers” • Group professional development (when several teachers have similar needs)

9. Five components in an aligned teacher evaluation-professional growth system

10. High quality professional growth opportunities • Characteristics of High Quality Professional Development (Archibald, et al, 2011) • Alignment with school goals, state and district standards and assessments, and other professional learning activities including formative teacher evaluation • Focus on core content and modeling of teaching strategies for the content • Inclusion of opportunities for active learning of new teaching strategies • Provision of opportunities for collaboration among teachers • Inclusion of embedded follow-up and continuous feedback

11. Teaching standards • A set of practices teachers should aspire to • A teaching tool in teacher preparation programs • A guiding document with which to align: • Measurement tools and processes for teacher evaluation, such as classroom observations, surveys, portfolios/evidence binders, student outcomes, etc. • Teacher professional growth opportunities, based on evaluation of performance on standards • A tool for coaching and mentoring teachers: • Teachers analyze and reflect on their strengths and challenges and discuss with consulting teachers

12. Quality training for teachers and leaders on teaching standards and evaluation tools • Increases inter-rater reliability • Increases the validity of system • Without valid results, professional development based on results will be unlikely to improve practice • Ensures mutual understanding • Is a form of professional development for those trained • Training, certification, and calibration on instruments may be more important than who the evaluator is

13. Memphis professional development system: An aligned system • Teaching and Learning Academy began April ‘96 • Nationally commended program intended to • “…provide a collegial place for teachers, teacher leaders and administrators to meet, study, and discuss application and implementation of learning…to impact student growth and development” • Practitioners propose and develop courses • Responsive to school/district evaluation results • Offerings must be aligned with NSDC standards • ~300+ On-line and in-person courses, many topics

14. Professional Growth Plan (From GTEP manual) • “The plan includes specific objectives for improvement, activities and a time line for meeting these objectives, criteria for measurement of progress toward meeting the objectives, a record of participation in recommended activities, and a record of performance on specified criteria.” http://www.pelham-city.k12.ga.us/docs/GTEP%20manual.pdf

15. Georgia School Keys Professional Development • “The increased practice of more professional learning in individual schools connected to student data and improvement plans has increased the opportunities for supporting the improved performance of students and staff. This practice has lead to more results-driven professional learning, which is standards-based, job-embedded, and collaborative.” http://www.doe.k12.ga.us/School-Improvement/Documents/FY%202011%20Professional%20Learning%20Annual%20Report%204-3-12.pdf

16. Professional development planning • Tap in-district expertise to provide professional development • Could be teachers, administrators, instructional leaders, etc. (Memphis is a good example) • Find outside supports to fill gaps in internal capacity • For example, partner with a local college • Create mechanism for connecting teachers to PD based on evaluation results (such as Memphis model) • Prepare for structural changes in the school week/calendar

17. Effective teachers • Strong, effective teachers and leaders are the key to improving student outcomes • Effective teachers may be in limited supply • Ways to get effective teachers and leaders: • Hire only highly effective teachers • Remove less effective teachers and leaders and replace them with more effective ones • Provide guidance and support to help less effective teachers become highly effective teachers

18. Teacher turnover • Each year, close to 200,000 teachers leave the profession, with another 200,000 transferring schools. • This teacher turnover costs our public school system about $5 billion annually (Alliance for Excellent Education, 2005).

19. A single “bad” year vs. a trend • Any teacher can have a year where they struggle • The key is to identify whether this is a “difficult” year or a “pattern” of poor performance • Response to a “difficult” year should be mostly supportive with targeted assistance • Response to a pattern should be more intensive with diagnosis of problem areas, improvement plan, time limit for improvement, etc. • Teachers want to be successful!

20. Interpreting results and making professional development recommendations • Analyzes evaluation data to connect with PD opportunities at the teacher, school and district level • Ideally provides guidance at the time of feedback • Can be a trained administrator, evaluator, coach, mentor, etc. (for individual teachers) • Could be a committee (for determining grade, subject, school or district-wide needs)

21. Criteria for Measurement of Progress (from GTEP manual) • “Specific criteria for measurement of progress on the PDP should be described. Measurement of progress involves keeping a record of successful completion of activities and determining whether or not the teacher's performance has improved in the targeted areas. ‘Improvement in targeted areas’ means previously marked areas have been judged to show improvement. The evaluator makes this judgment. For teachers who have received a required plan for specific needs development, progress toward completion of the PDP shall be evaluated during the next evaluation cycle if the individual is employed in the same school system. http://www.pelham-city.k12.ga.us/docs/GTEP%20manual.pdf

22. Are those who leave more effective? • Research suggests that those who leave (whether the school, the district, or the profession) tend to be less effective than those who stay in schools • The exception is first year teachers who leave—they tend to be more effective than those who stay (Hanushek and Rivkin, 2010)

23. Match between teachers & schools • Investigated the contribution of “match between teachers and schools” to student achievement (Jackson, 2010) • Showed that teacher effectiveness is higher after a move to a different school • 25% of what is typically considered to be a teacher effect may actually be a teacher-school (match) effect

24. “Fit” between person and organization • “…characteristics that predict individual effectiveness may not predict organization effectiveness, even if the assumptions about individual predictors are valid” (Harris & Rutledge, 2010, pg 923) • Person-Organization fit matters more when the organization has a “distinctive culture, the career ladder is lengthy, the interaction among organization members is great, and work flexibility is high.” (ibid, pg 924)

25. Impact of incentives on teacher turnover • Evaluators found that the probability of teacher turnover fell as the magnitude of the TEEG bonus award increased, while the probability of teacher turnover increased sharply among teachers receiving no bonus award, or a relatively small award (Springer et al., 2009) • “…a $3,000 award reduced turnover for beginning teachers by roughly 23 percentage points” • “…awards of $3,000 reduced turnover rates to less than a quarter of what the turnover rate was prior to implementation of a TEEG performance pay plan.”

26. Teacher peer effects • Using longitudinal elementary school teacher and student data, we document that students have larger test score gains when their teachers experience improvements in the observable characteristics of their colleagues. Using within-school and within-teacher variation, we further show that a teacher’s students have larger achievement gains in math and reading when she has more effective colleagues (based on estimated value-added from an out-of-sample pre-period). Spillovers are strongest for less-experienced teachers and persist over time, and historical peer quality explains away about twenty percent of the own-teacher effect, results that suggest peer learning. (Jackson & Bruegmann, 2009)

27. Cohort model to improve recruitment and retention of effective teachers • Consider a cohort model of teacher recruitment • Retain cohorts through ensuring collegial interactions such as professional learning communities are a priority • May include a pay incentive for retention at the cohort level (i.e., percentage of cohort that stays in school) • Partner with institutions of higher education to better prepare teachers for teaching at-risk students • Create a feedback loop with IHEs • Field experiences in high-needs schools should be extensive and high quality

28. Documenting distribution patterns of effective & less effective teachers • Many states have the capability to follow teachers’ movements but don’t track them • Develop a database that allows you to track teachers from school to school as well as exiting and reentering the profession

29. Documenting school & district changes in teacher effectiveness • With teacher effectiveness data from evaluations, districts may be better able to document the currentdistribution of effective teachers by the percentage of poor and minority students in the school • The challenge is to document change in the distribution of effectiveness over time and relate changes to specific policies or incentives, such as teacher professional growth • To do this, you need complete and accurate data about teachers’ participation in professional growth activities, mentoring, coaching, participation in professional learning communities, content-specific coursework, etc.

30. Tracking change: An Illustration

31. Questions to ask about your district’s professional development plan • Within your current or proposed teacher evaluation system, do you have a mechanism to ensure that evaluation results are aligned with and being used to guide professional development plans? • Are teachers given the opportunity to reflect on and discuss evaluation results and develop goals for further growth with supervisors, coaches, mentors, colleagues? • Can teachers collaborate with instructional managers to develop a plan for improvement and/or professional growth?

32. Questions to ask about your district’s professional development plan (cont’d.) • What additional supports and requirements are in place for teachers when trends in evaluation scores indicate that they are struggling? • What opportunities are available for teachers with high evaluation scores to pursue additional knowledge, growth, and leadership? • What follow-up mechanisms are in place to ensure that teachers are taking maximum advantage of professional growth opportunities? • How will change in teacher practice be documented?

33. Next steps • Ensure that evaluation systems are valid for identifying effective and less effective teachers • Focus on improving effectiveness of teachers you already have with high-quality, sustained professional development • Develop strategies for supporting and retaining effective and potentially effective teachers • (YOUR GREAT IDEAS HERE)

34. Resources/further reading • Coggshall, J. G., Rasmussen, C., Colton, A., Milton, J., & Jacques, C. (2012). Generating teaching effectiveness: The role of job-embedded professional learning in teacher evaluation. Washington, DC: National Comprehensive Center for Teacher Quality. http://www.tqsource.org/publications/GeneratingTeachingEffectiveness.pdf • Goe, L., Biggers, K., & Croft, A. (2012). Linking teacher evaluation to professional development: Focusing on improving teaching and learning. Washington, DC: National Comprehensive Center for Teacher Quality, http://www.tqsource.org/publications/LinkingTeacherEval.pdf • Imazeki, J., & Goe, L. (2009). The distribution of highly-qualified, experienced teachers: Challenges and opportunities. Washington, DC: National Comprehensive Center for Teacher Quality. http://www.tqsource.org/publications/August2009Brief.pdf • Memphis Teacher Development and Support Individualized Growth Resource Book http://tntp.org/assets/tools/MCS%20TEM-Aligned%20PD%20Resource%20Guide_TSLT%203.12.pdf

35. References Alliance for Excellent Education. (2005). Teacher attrition: A costly loss to the nation and to the states. Washington, DC: Author. http://www.all4ed.org/files/archive/publications/TeacherAttrition.pdf Archibald, S., Coggshall, J., Croft, A., & Goe, L. (2011). High-quality professional development for all teachers: Effectively allocating resources. Washington, DC: National Comprehensive Center for Teacher Quality. http://www.tqsource.org/publications/HighQualityProfessionalDevelopment.pdf Harris, D., & Rutledge, S. A. (2010). Models and predictors of teacher effectiveness: A comparison of research about teaching and other occupations. Teachers College Record, 112(3), 914-960.http://eps.education.wisc.edu/Faculty%20papers/Harris/Manuscripts/IES%20Harris%20Rutledge%20Models%20and%20Predictors%20TCR%20Nonblind.doc Hanushek, E. A., & Rivkin, S. G. (2010). Constrained job matching: Does teacher job search harm disadvantaged urban schools? Cambridge, MA: National Bureau of Economic Research. http://www.urban.org/uploadedpdf/1001395-constrained-job-matching.pdf

36. References (cont’d.) Jackson, C. K. (2010). Match quality, worker productivity, and worker mobility: Direct evidence from teachers. Cambridge, MA: National Bureau of Economic Research. http://www.ssc.wisc.edu/~scholz/Seminar/Jackson.pdf Jackson, C. K., & Bruegmann, E. (2009). Teaching students and teaching each other: The importance of peer learning for teachers. Cambridge, MA: National Bureau of Economic Research. http://digitalcommons.ilr.cornell.edu/cgi/viewcontent.cgi?article=1079&context=workingpapers Springer, M., Lewis, J. L., Podgursky, M. J., Ehlert, M. W., Taylor, L. L., Lopez, O. S., & Peng, A. X. (2009). Governor’s Educator Excellence Grant (GEEG) Program: Year three evaluation report. Nashville, TN: National Center on Performance Incentives. http://ritter.tea.state.tx.us/opge/progeval/TeacherIncentive/GEEG_Y3_0809.pdf

37. Questions?

38. Laura Goe, Ph.D. • 609-619-1648 • lgoe@ets.org • www.lauragoe.com • https://twitter.com/GoeLaura • Center on Great Teachers and Leaders at AIR • 1000 Thomas Jefferson Street NW • Washington, DC 20007-3835 • General Information: 877-322-8700 • www.gtlcenter.org