1 / 36

THE PUZZLING PURSUIT OF TEACHER EFFECTIVENESS: LONGITUDINAL DATA ANALYSIS OF THE IMPACT OF TEACHER CHARACTERISTICS ON S

THE PUZZLING PURSUIT OF TEACHER EFFECTIVENESS: LONGITUDINAL DATA ANALYSIS OF THE IMPACT OF TEACHER CHARACTERISTICS ON STUDENT ACHIEVEMENT Marco Muñoz, Ed.D. Florence Chang, Ph.D. NEI 2007. Overview.

king
Download Presentation

THE PUZZLING PURSUIT OF TEACHER EFFECTIVENESS: LONGITUDINAL DATA ANALYSIS OF THE IMPACT OF TEACHER CHARACTERISTICS ON S

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. THE PUZZLING PURSUIT OF TEACHER EFFECTIVENESS: LONGITUDINAL DATA ANALYSIS OF THE IMPACT OF TEACHER CHARACTERISTICS ON STUDENT ACHIEVEMENT Marco Muñoz, Ed.D. Florence Chang, Ph.D. NEI 2007

  2. Overview • Using multi-level analysis, high school teacher effectiveness was studied by examining the relationship between teacher characteristics (experience, education, race) and reading test scores in a large urban district. • Significance of the topic and methodological approach.

  3. Why Evaluate Teacher Characteristics and Student Growth? • Recognition of the importance of quality teachers is imbedded in the No Child Left Behind Act of 2002 requirement that there be a “highly qualified” teacher in every classroom. • Improving the quality of teacher performance is a viable and important strategy for improving student achievement. • Research call for evidence that certain teacher characteristics positively impact K-12 student outcomes.

  4. Research on Teacher Characteristics and Student Achievement • Improving teacher quality is a concern among educators and policy makers. • Research has consistently shown that teachers are a primary causal driver of student achievement gains (Darling-Hammond & Youngs, 2002), • There are some identifiable characteristics of teachers that are predictive of their success in the classroom (Darling-Hammond & Youngs, 2002; Wayne & Youngs, 2001).

  5. Research Teacher Characteristics Framework • Evidence about teacher characteristics (Wayne & Youngs, 2003) : • Teachers’ college ratings • Test scores (e.g., PRAXIS) • Course taking and degrees • Certification status (e.g., HS math) • Low-income students may have fewer teachers with certain characteristics (Wayne, 2002)

  6. Research Framework • Teacher characteristics and student achievement gains: A review (Wayne & Youngs, 2003) • “In addition, for many potentially salient teacher characteristics—such as experience, race, and study of teaching methods—studies that use convincing research designs simply do not exist or have not been conclusive.” (p. 107).

  7. Research Framework • Teachers, Schools, and Academic Achievement (Rivkin, Hanushek, & Kain, 2001) • Using student achievement at three points in time, the authors attempt to isolate the effects of teachers over time. • Promising methodology!

  8. Research Hypotheses 1) Unconditional Mean Model: There will be some variance between classrooms to be explained. 2) Unconditional Growth Model: Adding time to the model will show significance of time as a Level 1 predictor and explain some Level 1 variance. There will be significant variation in the Level 2 components to be explained. 3) Conditional Growth Model (Experience, Education, and Race): HS English teacher characteristics will be positively associated with the rate of change in Grade 9 reading achievement.

  9. Participants • The data set included grade 9 students and teachers from a large urban district high schools • School year 2005-2006) • 1,487 freshman students • 58 English teachers

  10. Descriptive Statistics LEVEL-1 VARIABLE NAME N MEAN SD MINIMUM MAXIMUM • PAS 4,684 44.42 16.96 0.00 94.00 • TIME 4,684 0.98 0.82 0.00 2.00 • LUNCH 4,684 0.70 0.46 0.00 1.00 LEVEL-2 VARIABLE NAME N MEAN SD MINIMUM MAXIMUM • EDUC 58 0.24 0.43 0.00 1.00 • EXP 58 8.64 8.65 0.00 31.00 • MINORITY 58 0.24 0.43 0.00 1.00

  11. Measurement • The Predictive Assessment Series (PAS) in Reading was used to assess high school students in JCPS • Three waves of data: Fall 2005, Winter 2005, and Spring 2006 • Level I: Time • Level II: Teacher Characteristics (Education, Experience, & Race)

  12. Design & Analytical Approach Multilevel Model for Change • Hierarchical linear modeling (HLM), the state-of-the-art tool used in school effects research • This study used two-level HLM: time-and teacher-level characteristics. • Cross-validation conducted with two-level (aggregated) and three-level HLM model.

  13. Basic Terminology • Level-1 variables: • Typically these are individual-level variables that are nested within groups. • Level-2 variables • Typically these are higher-level, grouping variables. Our growth models have time periods at level-1, and teachers at level-2.

  14. Multilevel Model for Change • The level-1 submodel for individual change —empirical growth trajectories • The level-2 submodels for systematic interindividual differences in change • Fitting the multilevel model for change to data • Interpreting the results of model fitting (fixed effects and variance components)

  15. Multilevel Model for Change This study applied a three-step procedure by using Singer and Willett’s Longitudinal Model (2003): 1. The Unconditional Means Model 2. The Unconditional Growth Model 3. The Conditional Growth Model

  16. Exploring Data

  17. Unconditional Means Model • The 1st model: • Unconditional Means Model • The equations are: Level 1: Yij = β0j + rij Level 2: β0j = 00 + u0j • The Composite Model Yij = 00 + u0j + rij

  18. Unconditional Model: Grade 9 Reading

  19. Results • The ratio of between-classroom variance to the total variance was estimated by the intra-class correlation (ICC) = Between Variance / Total Variance • The result indicated that 14% of the variance in Grade 9 teachers in PAS reading scores was between teachers. • Significant variation existed among classrooms in student achievement growth.

  20. 2nd Model: Unconditional Growth Model • Level 1: Yij = β0j + β1j (TIME) + rij • Level 2: β0j = 00 + u0j β1j = 10 + u1j • Composite Model Yij = 00 + 10 (TIME)+ u0j +u1j (TIME) + rij

  21. Unconditional Growth Model: Grade 9 Reading

  22. Unconditional Growth Model (Cont.)

  23. Explained Variance by Time Predictor • Level I • Significant variance at Level I (86%) • Unconditional Means Model: 257.41 • Unconditional Growth Model: 245.16 • Explained variance (Pseudo R2): 5% of the 86%

  24. Conditional Growth Model:Experience, Education, & Race as Predictors • Level 1: Yij =β0j + β1j (TIME) + rij • Level 2: β0j =00 +01 (EXP/EDUC/RACE)+u0j β1j =10 +11 (EXP/EDUC/RACE)+u1j • Composite Model Yij = 00 +01 (EXP/EDUC/RACE)+u0j + 10 (TIME) + 11 (EXP/EDUC/RACE) (TIME)+u1j (TIME) + rij

  25. Conditional Growth Model: Experience, Education, & Race

  26. Conditional Growth Model: Experience, Education, and Race

  27. Fit Statistics

  28. Explained Variance by Experience, Education, & Race • Level II, significant variance (14%) • Experience explained variance on Rate of Change (Pseudo R2): 0% • Education explained variance on Rate of Change (Pseudo R2): 3% • Race explained variance on Rate of Change (Pseudo R2): 15%

  29. Graph of Model Equation: Experience

  30. Graph of Model Equation: Education

  31. Graph of Model Equation: Race

  32. Conditional Growth ModelResults • Based on an applied longitudinal data analysis approach, teacher-level predictors—years of experience, education, and race—were not significantly related, on average, with grade 9 reading growth. • An aggregated two-level model and a three-level model were run and the results were cross-validated.

  33. Discussion • Emerging body of studies that examines the relationship between student achievement gains and the characteristics of the teachers. • New methodological approach (longitudinal multilevel model) • The study did not confirm that students learn more from teachers with certain characteristics. • Teacher quality is complex and multi-faceted (see Early et al., 2007 for early childhood study)

  34. Limitations • Strictly exploratory study of one content area (reading) in one high school grade (9th) in one district in Kentucky (JCPS) • Instrumentation—not an accountability test (teacher assignment, low stakes) and testing window (i.e., spring) issues • Definition of teacher education – did not consider rigor of pre-service program • Did not take into account process-oriented teacher characteristics (i.e., instructional method)

  35. Implications for Future Studies • H.R. hiring (performance vs. credentials) and compensation (tenure) policies • Teacher quality and the role of PD • Unfortunately, some of the variation in teacher quality is unseen: the art of teaching • Teachers differ greatly in their effectiveness, but teachers with different qualifications differ only a little…

  36. Contact Info • Marco Munoz, marco.munoz@jefferson.kyschools.us • Florence Chang, florence.chang@jefferson.kyschools.us Mailing Address: 3332 Newburg Road Dept. of Accountability, Research, and Planning VanHoose Education Center Louisville, KY 40218

More Related