Improving school and college accountability at 16-19

1. Improving school and college accountability at 16-19 Abigail Shaw, Performance Tables Development Unit, Department for Education

2. Aims • To provide a brief overview of the 16-19 accountability reforms. • To explain how we identified data requirements and utilised additional data sources. • To share the benefits of collaborative working with a range of stakeholders across the development cycle. • To reflect on lessons learned when making significant changes to a large and established process.

3. Overview • The school and college accountability system is being reformed to set higher expectations, with the aim to make the system fairer, more ambitious and transparent. • This new system will be based on five headline measures to be published in post-16 performance tables from 2016 • We are using additional data sources… • … so we can align the allocation of results with the institution receiving funding for the student, ensuring we are holding the right providers to account • …and develop measures that provide a rounded view of provider performance

4. Background In 2013/14, DfE set out proposals for improving 16-19 accountability… • To introduce more rigorous minimum standards • To publish clearer and more comprehensive performance information about schools and colleges, including new headline measures to provide a richer overview of provider performance. The 16-19 education landscape is complex… • Students can choose to study a range of qualifications… • at a range of levels… • at a range of institution types • There is also a lot more movement during 16-18 study compared with earlier key stages and a significant minority of students attending more than one institution.

5. Highest study aim of 16/17 year olds in Full Time Education and Apprenticeships, end 2015, by institution type (inc relative size) State School 6th forms General Further Education Sixth form colleges 146,100; 13.1% of FT participants 387,700; 34.7% of FT participants 411,700; 36.9% of FT participants Independent schools Apprenticeships Special schools HE Institutions 11,400; 1.0% of FT participants 6,200; 0.6% of FT participants 82,400; 7.4% of FT participants 71,700; 6.4% of FT participants Sizes are approximate based on area in relation to number of participants

6. Reformed measures Traditionally school performance tables have been based on exam data, however some of the new measures require information that cannot be obtained from exam data. This can be illustrated through the new retention measure.

7. The new retention measure

8. Development cycle There have been challenges to solve across all of these stages. The next few slides will pick up some of these in more detail

9. Identifying and evaluating data sources • Student-level data available from school census (school) / individualised learner record (ILR) (colleges) • Requirements – aim type, defining a core aim, start and end dates, completion status • Timeliness – data not available until January/February meaning retention measure will not be available when performance tables go live.

10. Which students should be included • Decision made to align with funding methodology where possible. Ensures a joined up message across the department. • Funding data sourced separately from the Education Funding Agency. • Ongoing conversations with funding policy colleagues as changes to funding methodology are considered.

11. Identifying the core aim • Different rules for school census and ILR data and for academic and vocational qualification types. • Align with funding policy e.g. for an A level student to be retained they need only complete 1 A level. • Needed to develop our own rules to deal with situations where a student has no core aim defined, or where multiple core aims are identified.

12. Retention measure as part of a suite of headline measures • There are changes to the methodology used to allocate students to institutions (deals with situations where students are registered at multiple institutions) • All students will be allocated on an annual basis (at present students are allocated to one institution for the whole of their 16-18 study) • Students will be assigned to the provider which is directly funded by the EFA in each year of study and where the student has enrolled to take their main programme of study • But key aim of retention measure is to identify and penalise providers where students leave before the end of their study even if they then move to a different institution. • Needed an exception to normal allocation rules to allow more than one provider per year for the retention measure.

13. Building and testing systems with contractors • Introducing brand new data sources and new indicators to an already complex process. • Decision made to ‘pilot’ the system by producing ‘shadow’ measures using 2015 data. • Invaluable as enabled system to be built, tested and refined in advance. Also gave us an extra year of data to enable some comparisons to be made between the old and new measures and between 2015 and 2016 data. • Shadow data was shared with schools and colleges to give them early sight of the new measures.

14. Shadow data analysis - retention

15. Shadow data analysis - retention • Academic and A-level cohorts – half of providers had a 95-99% retention rate – 20% achieved a rate of 100% • Tech Level and Applied General – wider distribution

16. 16-19 accountability – where are we now? • Provisional statistical first release to be published in October • Performance tables to be published in January 2017 (excluding retention and completion and attainment) • Performance tables update to be published in spring 2017. • Development work has already moved on to prepare for next year’s tables where more big changes are coming!

17. Key lessons learned when introducing significant change • Ensure sufficient time is available • Identify all relevant stakeholders early on. Engagement should be ongoing throughout the whole process. • Pilots are invaluable

18. Any questions?