Beyond test scores: the role of primary schools in improving multiple child outcomes

1. Beyond test scores: the role of primary schools in improving multiple child outcomes Claire Crawford and Anna Vignoles Institute of Education, University of London

2. Motivation • League tables measure school performance in terms of academic achievement e.g. test scores • But Every Child Matters agenda suggests schools should have a wider remit than just increasing educational attainment • e.g. improving health, socio-emotional skills, etc

3. Research Questions • Are schools at the top of the league tables also the best at improving child welfare? Or is there a trade-off? • What methodological approaches are most appropriate when measuring multiple outcomes from education?

4. Previous research • Small (but positive) impact of schools on educational attainment once variation in pupil characteristics is taken into account • e.g. Goldstein and Sammons 1997; Reynolds et al. 1996; Leckie, 2008; Hanushek 1997 • Limited work on role of schools in promoting wider outcomes, such as well being • e.g Gibbons and Silva, 2009; Feinstein & Gutman 2008 • But different disciplines have used different methods and comparisons of results is difficult • e.g. educationalists relied on multi level models/ economists tend to use fixed effect models

5. Outline of today’s talk • Use simple value-added model of educational attainment to compare RE and FE models • Are assumptions underlying RE/multi level model satisfied? • Use results to inform later model development

6. Outline of today’s talk • Document development of cognitive and non-cognitive skills in primary school • Can we think of these relationships as causal? • What are our next steps? • How to consider joint determination of cognitive and non-cognitive skills

7. RE vs. FE • Start with a model of the following form: Yist = α + βYist-1 + γXist-1 + δs + εist-1 • Where: • Y: Key Stage maths points • X: vector of pupil characteristics (including family composition, parents’ education, parents’ socio-economic status, etc) • RE model assumes: E[Xist-1|δs] = 0 • i.e. pupil characteristics are unrelated to the school effects • If E[Xist-1|δs] ≠ 0, RE model produces biased estimates • Rely on FE model instead • Use Hausman test to judge whether condition has been met • Future work will explore other tests as well

8. Determinants of KS2 maths score • Hausman test rejects RE model

9. Modelling cognitive and non-cognitive skills • Focus on value-added models between ages 7 and 11: KSist = α + β1KSist-1 + β2SDQist-1 + γXist-1 + δs + εist-1 SDQist = α + β1KSist-1 + β2SDQist-1 + γXist-1 + δs + εist-1 • KS: Key Stage test results • Standardised average of reading, writing, maths at age 7 • Standardised average of English, maths, science at age 11 • SDQ: Strengths & Difficulties score • Teacher reports • 5 questions on 5 topics: pro-social, hyperactivity, emotional symptoms, conduct problems, peer problems • Invert pro-social and combine to create total “difficulties” score • Standardise total “difficulties” score and invert so SDQ measures “strengths” rather than “difficulties”

10. Model controls • X is a vector of pupil characteristics, including: • Month of birth • Ethnicity • Family composition (including twin status, number of older and younger siblings, whether parents are married or cohabiting) • Parents’ education • Parents’ socio-economic status • Birthweight and whether child was breastfed • Whether parents often read to the child at age 3 • δ represent school fixed effects

11. Model estimation • Estimate using seemingly unrelated regression analysis • Takes account of the fact that the error terms across the two equations may be correlated • Can extend to consider more outcomes • e.g. may additionally want to consider health?

12. Development of cognitive and non-cognitive skills

13. Causal effects? • Instrument KS1 attainment using month of birth to identify causal effects of KS1 on KS2 and SDQ scores

14. Summary • Tested RE model with rich set of controls • Rejected RE model, so focus on FE models instead • Investigated development of cognitive and non-cognitive skills between ages 7 and 11 • Found strong statistical relationships • Inclusion of school fixed effects marginally increases the relationship between SDQ scores at 7 and 11 • Impact of attainment at age 7 on KS2 APS appears causal; but impact of attainment at age 7 on SDQ score at age 11 is not

15. Next steps • Further robustness checks surrounding rejection of RE model, using other models and data • Use other statistical software to run seemingly unrelated IV regressions • Consider first difference models of KS and SDQ scores • Instrument change in SDQ score with death of a parent/close family member