Top-to-Bottom Ranking & Priority/Focus/Reward Designations

1. Top-to-Bottom Ranking &Priority/Focus/RewardDesignations Understanding the

2. Why the TTB Ranking? • Move from metric as designation only (“stick”) to leveraging the metric as a diagnostic tool for schools • Resist urge for “more data” until we understand the metrics available; avoids “dying in data” • Focus of this presentation: • Overview of calculations • How to interpret results (for schools and districts)

3. How Designations are Related

4. The Top-to-Bottom List • Statewide ranking of ALL schools that meet ranking criteria • Bottom 5% = Priority (PLA) Schools • 10% schools with largest achievement gaps = Focus Schools • Also used for Reward School status: • Use top 5% from overall ranking = Reward Schools • Use top 5% improving schools = Reward Schools • Use Beating the Odds Schools = Reward Schools

5. Top to Bottom(TTB) Ranking • Three main components by subject: • Achievement • Improvement in achievement over time (top 5% here become Reward Schools) • Achievement gap top 30% vs. bottom 30% of students (bottom 10% here become Focus Schools) • Each component tells schools something about their overall performance and can be used for diagnostics

6. TTB Ranking • Graduation rates are included in the statewide Top-to-Bottom Ranking. • Schools with a graduation rate have it included in the following two ways: • Graduation Rate • Improvement in graduation rate over time

7. Who Receives a Ranking? • Schools with 30+ full academic year (FAY) students over the last two years in at least two state-tested content areas; school must be OPEN at time of list generation • Application  Some schools do not receive a ranking if they: • Have too few FAY students • Only have one year of data

8. Tested Grades & Subjects • Reading and Mathematics: Grades 3-8 and 11 • In grades 3-8, testing every year allows us to figure out student performance level change (our current “growth” metric) in reading and math • Students can either significantly improve, improve, maintain, decline or significantly decline • Writing: Grades 4 & 7 • Science: Grades 5 & 8 • Social Studies: Grades 6 & 9

9. What About Reconfigured Schools? • A school must change by four or more grades in order to get a new code • Example: A K-2 building becoming a K-6 building. • New codes NOT granted when a school is reopened as a charter, for example • If not, the school retains the old code and continues to have data “point” at it from all students for whom that code is their feeder school • There is no “phase reset” like there was in AYP • If school population changed by 51%, could request a phase reset—still got AYP calculations, but sanctions delayed • Under Priority/Focus interventions, would simply have a customized intervention.

10. How Is the Top to Bottom Ranking Calculated • For grade 3-8 reading and mathematics Two-Year Average Standardized Student Scale (Z) Score School Achievement Z-Score 1/2 Two-Year Average Performance Level Change Index School Performance Level Change Z-Score Content Index Z-score School Content Area Index 1/4 Two-Year Average Bottom 30% - Top 30% Z-Score Gap School Achievement Gap Z-Score 1/4

11. How do we get Standardized Scale Scores for Each Student? • Step #1: Take each student’s score on the test they took and compare that score to the statewide average for students who took that same test in the same grade and year • This creates a student-level z-score for each student in each content area • Compare • MEAP to MEAP • MEAP-Access to MEAP-Access • MME to MME • MI-Access • Participation to Participation • Supported Independence to Supported Independence • Functional Independence to Functional Independence

12. What do we do with those Standardized Scores? • Step #2: Once each student has a z-score for each content area (based on the test they took), we take all of the students in a each school, and rank order the students within the school. • Z-scores will have come from different tests, and compare students to statewide average for that grade, test, and subject • But they can now be combined for the school • Step #3: Add up all z-scores and take the average. This is now the average standardized student scale score. • Step #4: Define the top and bottom 30% subgroups, based on that rank ordering.

14. Average Z-score (average standardized student scale score): 0.28 (sum all z-scores, divide by 14)

15. Top 30% Bottom 30%

16. How Is the Top to Bottom Ranking Calculated? • For grade 3-8 reading and mathematics Two-Year Average Standardized Student Scale (Z) Score School Achievement Z-Score 1/2 Two-Year Average Performance Level Change Index School Performance Level Change Z-Score Content Index Z-score School Content Area Index 1/4 Two-Year Average Bottom 30% - Top 30% Z-Score Gap School Achievement Gap Z-Score 1/4

17. What is Important to Show Schools? • For grade 3-8 reading and mathematics Two-Year Average Standardized Student Scale (Z) Score School Achievement Z-Score 1/2 Two-Year Average Performance Level Change Index School Performance Level Change Z-Score Content Index Z-score School Content Area Index 1/4 Step #1: Achievement How well did the school do in that subject? Positive number = better than average Near zero = average Negative number = worse than average Two-Year Average Bottom 30% - Top 30% Z-Score Gap School Achievement Gap Z-Score 1/4

18. What is Important to Show Schools? Step #2: Improvement Is the school improving in that subject? Positive number = greater rate of improvement than average Near zero = average improvement Negative = slower rate of improvement than average; can also mean they are declining • For grade 3-8 reading and mathematics Two-Year Average Standardized Student Scale (Z) Score School Achievement Z-Score 1/2 Two-Year Average Performance Level Change Index School Performance Level Change Z-Score Content Index Z-score School Content Area Index 1/4 Two-Year Average Bottom 30% - Top 30% Z-Score Gap School Achievement Gap Z-Score 1/4

19. What is Important to Show Schools? Raw value is also meaningful: Positive number: More students improving than declining Negative number: More students declining than improving • For grade 3-8 reading and mathematics Two-Year Average Standardized Student Scale (Z) Score School Achievement Z-Score 1/2 Two-Year Average Performance Level Change Index School Performance Level Change Z-Score Content Index Z-score School Content Area Index 1/4 Two-Year Average Bottom 30% - Top 30% Z-Score Gap School Achievement Gap Z-Score 1/4

20. What is Important to Show Schools? Step #3: Achievement Gap Is the gap in that subject between top 30% and bottom 30%: (positive number) = smaller gap than average (negative number) = larger gap than average (near zero) = average gap • For grade 3-8 reading and mathematics Two-Year Average Standardized Student Scale (Z) Score School Achievement Z-Score 1/2 Two-Year Average Performance Level Change Index School Performance Level Change Z-Score Content Index Z-score School Content Area Index 1/4 Two-Year Average Bottom 30% - Top 30% Z-Score Gap School Achievement Gap Z-Score 1/4

21. Once you have Looked at each Component, Discuss: • What’s the overall pattern? • Low achievement? • Declining achievement? • Large gaps? • Where are the actionable areas? • Which subjects need the most attention? • Is everyone doing poorly (small gap, low achievement) or are some students doing well and others falling behind (decent achievement, but large gap)

22. What is Important to Show Schools? • For grade 3-8 reading and mathematics Two-Year Average Standardized Student Scale (Z) Score School Achievement Z-Score 1/2 Reward Schools (for improvement) Two-Year Average Performance Level Change Index School Performance Level Change Z-Score Content Index Z-score School Content Area Index 1/4 Focus Schools Two-Year Average Bottom 30% - Top 30% Z-Score Gap School Achievement Gap Z-Score 1/4

23. Weighted Performance Level Change (PLC) • A weighted composite of individual student performance level change is used to calculate improvement in grades 3-8 reading and mathematics • Rewards large improvements more heavily, rewards maintenance of proficiency if a student was already proficient

24. How is the Top-to Bottom Ranking Calculated? • For science, social studies, writing, and grade 11 all tested subjects Two-Year Average Standardized Student Scale (Z) Score School Achievement Z-Score 1/2 Four-Year Achievement Trend Slope School Performance Achievement Trend Z-Score School Content Area Index Content Index Z-score 1/4 Two-Year Average Bottom 30% - Top 30% Z-Score Gap School Achievement Gap Z-Score 1/4

25. How is the Top-to-Bottom Ranking Calculated? • For graduation rate Two-Year Average Graduation Rate School Graduation Rate Z-Score 2/3 Four-Year Graduation Rate Trend Slope School Graduation Rate Trend Z-Score School Graduation Rate Index Grad Index Z-score 1/3

26. How is the Top-to-Bottom Ranking Calculated? • Calculating a four-year slope (e.g., graduation rate) • Plot the school’s graduation rate for the last four years • Plot a linear regression line through the points • Calculate the slope of the line (gives the school’s annual improvement rate)

27. How is the Top-to-Bottom Ranking Calculated? • Calculating a four-year slope (e.g., graduation rate) • Plot the school’s graduation rate for the last four years • Plot a linear regression line through the points • Calculate the slope of the line (gives the school’s annual improvement rate)

28. How is the Top-to-Bottom Ranking Calculated? • Calculating a four-year slope (e.g., graduation rate) • Plot the school’s graduation rate for the last four years • Plot a linear regression line through the points • Calculate the slope of the line (gives the school’s annual improvement rate) Slope = 2.3%

29. How is the Top-to-Bottom Ranking Calculated? • Calc an overall rank for a school with a grad rate School Mathematics Std Index 18% School Reading Std Index 18% Overall Standardized School Index Overall School Percentile Rank School Science Std Index 18% School Social Studies Std Index 18% School Writing Std Index 18% School Graduation Rate Std Index 10%

30. How is the Top-to-Bottom Ranking Calculated? • Calculating an overall ranking for a school without a graduation rate School Mathematics Std Index 20% Look at each subject index. Help schools understand which subjects are strong/weak for them. Positive number: better than average Negative number: below average Near zero: near average School Reading Std Index 20% Overall School Standardized Index Overall School Percentile Rank School Science Std Index 20% School Social Studies Std Index 20% School Writing Std Index 20%

31. How is the Top-to-Bottom Ranking Calculated? • Calculating an overall ranking for a school without a graduation rate and without a writing score School Mathematics Index 25% School Reading Index 25% Overall School Standardized Index Overall School Percentile Rank School Science Index 25% School Social Studies Index 25%

32. Which years of data are in the ranking?

33. For elementary and middle schools • Michigan tests in the fall. • These fall tests reflect the learning of students in the previous school year. SY 2010-2011 SY 2011-2012 SY 2012-2013 Fall 2011 Testing Fall 2009 Testing Fall 2010 Testing Fall 2012 Testing

34. For High Schools • Michigan tests in the spring • The spring test (MME and MI-Access) measures what students have learned from grades 9, 10 and grade 11 prior to the MME testing.

35. What do the 2012 Rankings reflect? • For elementary/middle schools: • MEAP and MI-Access tests from fall 2011 • For high schools: • MME and MI-Access tests from spring 2012

36. Resources Available • List of all schools and their ranking • Individual school look-up to see school-specific results • Overview presentation with voice over • FAQ • Business rules by which the rankings were calculated • Complete data file and validation file You can access these resources at www.mi.gov/ttb You can also request individual assistance by calling the Evaluation, Research and Accountability unit at 517-373-1342, or emailing mde-accountability@michigan.gov

37. Priority School Status • Schools in the bottom 5% of the Top-to-Bottom Ranking • MDE ensures that the number of schools identified as Priority Schools includes >=5% of the state’s Title I schools. • MDE will also add schools with a grad rate of <60% for 3 consecutive years, and any Tier I or Tier II school using SIG funds to implement a turnaround model.

38. How does a school EXIT Priority School status? • For a school to exit priority school status, it has to receive a Green, Lime, Yellow or Orange on the Accountability Scorecard at the close of its third year in the Priority School intervention system. • A school must either meet aggressive proficiency targets (toward 85% of students proficient by 2022), or must have demonstrated significant improvement. • The proficiency and/or improvement gains must be demonstrated all nine traditional ESEA subgroups as well as in the new “bottom 30%” subgroup.

39. How does a school EXIT Priority School status? • This means that a priority school who achieves a Green, Lime, Yellow or Orange on the Accountability Scorecard and exits Priority School status has: • Met all interim measurements of progress for priority schools (approved plan, leading and lagging indicators). • Met proficiency and/or improvement targets on average as a school. • Increased the proficiency rate of all traditional subgroups. • Increased the proficiency rate of their very lowest performing students.

40. Focus School Status • 10% of Schools with the largest achievement gaps in scale score between the top 30% of students and bottom 30% of students within a school from the Top-to-Bottom Ranking. • Focus schools may have high average performance overall, but have a significantly large gap, suggesting struggles addressing low achieving students

41. The Purpose ofIdentifying Focus Schools • Critical component to closing achievement gaps within schools and statewide. • Highlight where changes in teaching and learning practices need to be undertaken to respond to the learning needs of low-achieving students. • These changes are difficult; both accountability and support need to be differentiated.

42. Focus Schools: Achievement Gaps • Top-to-Bottom list includes a component that examines the gap in achievement scores between top 30% and bottom 30% of students within a school • Gaps are standardized between all students using a common assessment within a school, and then averaged for the school • Gaps are calculated for all subject areas and combined to form a composite gap

43. How are Focus Schools Identified? • Average gap is then standardized and ranked among all schools • The 10% of schools with the largest achievement gaps are identified as Focus Schools

44. Focus Schools: Common Concerns • Are Focus Schools only high-achieving schools? • Are Focus Schools only high socioeconomic status schools? • Is the bottom 30% subgroup in Focus Schools actually high performing? • Are schools more likely to be Focus Schools if they have [fill in the blank group] kids?

45. Focus Schools are NOTJust High Achieving...

46. Focus Schools are NOT ExclusivelyHigh (or Low) Socioeconomic Status...

47. Bottom 30% Students are NOT High Achieving ... Non-Focus Schools Focus Schools Across all subject areas and grade levels, the bottom 30% subgroup consistently had average achievement z-score below zero, and most of them are between -0.5 and -1.5. (This example: E/MS Reading) Top 30% Bottom 30%

48. Bottom 30 are Not Dominated by any 1 Subgroup… • Focus schools have higher concentrations of subgroups in bottom 30% than other schools

49. Focus Designation vs. AYP • Achievement gap between top 30% and bottom 30% of students within a school. • This approach targets ACHIEVEMENT gaps and THEN asks the demographic question. • Methodology detects differences in achievement within subgroups; between subgroups; or with small populations. • Limited by the size of groups and demographic status only. • Methodology detects differences in achievement within a subgroup as a whole or as an overall student population

50. Reward School Status Identification as a Reward School results from achieving one or more of the following distinctions: • Being in the top 5% of the Top-to-Bottom Ranking • Being in the top 5% of improving schools from the improvement metric in the Top-to-Bottom ranking • Being a school identified as “Beating the Odds” (BTO).