The Big Picture

1. RE-THINKING HOWSCHOOLS IMPROVE:A Team Dialogue Model for Data-Based Instructional Decision Making

2. The Big Picture • In today’s session, we are going to: • Re-think our understanding of how schools improve -- moving from the dysfunction of the “old model” to the requirements for what a “new model” might look like. • Focus on a “new model” for improving performance that enables content, vertical, or departmental teams to use data more effectively for classroom instructional improvement and increased student learning.

3. “Every organization is perfectly designed to get the results it achieves.” --W. Edwards Deming

4. What are data? Data are observations, facts, or numbers which, when collected, organized and analyzed, become information and, when used productively in context, become knowledge.

5. The DRIP Syndrome DATA RICH INFORMATION POOR

6. Being Data Rich Your school may suffer from DATA OVERLOAD You may need ways to organize and use all the data you have.

7. Sources of Student Achievement Data • External assessment data • Benchmark or course-wide assessment data • Individual teacher assessment data --Supovitz and Klein (2003)

8. Data-driven schools and school districts use data for two major purposes: • Accountability (to prove) • Instructional decision making (to improve)

9. The Hierarchy of Data for Accountability Purposes External (State & National) Assessments System Benchmark Assessments Common School or Course Assessments Classroom Assessments of Student Work

10. The Hierarchy of Data for Instructional Decision Making Classroom Assessments of Student Work Common School or Course Assessments System Benchmark Assessments External (State & National Assessments)

11. Think about how long you have been engaged in the school improvement process. Has the school gotten better each year? Has the performance of each student improved as a result of each year he/she spends in the school? If your answer to one or both questions is no, what will it take to change it to yes?

12. Think about your School Improvement Plan and the data on which it is primarily based. Is it . . . • State assessment data? (data to prove) • OR • Classroom assessments of student work? (data to improve)

13. Then ask yourself this question: Do you have a school improvement plan? Or a school accountability plan? A SIP ? Or a SAP?

14. WHY IS THE OLD MODEL OF SCHOOL IMPROVEMENT NOT WORKING ANY MORE?

15. WHY IS THE OLD MODEL OF SCHOOL IMPROVEMENT NOT WORKING ANY MORE? You may wish to stop the presentation at this time to discuss your faculty’s views on this question.

16. Why? Wrong Data • We have been using the wrong data. State test data are: • Way too general • Instructionally insensitive -- not designed for instructional improvement

17. Why? Wrong Time • The data come at the wrong time. State test data are: • Out of date when they arrive • For students we may no longer have The results of the changes that are implemented will not be known for a year.

18. Why? Wrong Team • The SIT, a full department, or a Data Committee are the wrong groups to do the analysis. • Membership is too diverse (often including parents) • Meets too infrequently • Not connected to immediate classroom needs

19. Why? Wrong Plan • The initiatives that are put in place are: • Too global to address the diversity of students • Aimed at performance increases of groups on average • Looking for the “silver bullet” that will have a schoolwide impact

20. We need a new model. • Uses “real time” data • Sessions build on each other • Addresses individual students’ needs • Results in instructional improvements that will actually occur at a high level of quality • Can be re-directed frequently • Has meaning for teachers and is seen by teachers as a worthwhile use of their time

21. We need a new model. • Uses “real time” data • Sessions build on each other • Addresses individual students’ needs • Results in instructional improvements that will actually occur at a high level of quality • Can be re-directed frequently • Has meaning for teachers and is seen by teachers as a worthwhile use of their time You may wish to stop the presentation at this time and discuss the extent to which these positive characteristics are present in the data analysis process your school is currently using.

22. What should that new model look like? “School improvement is most surely and thoroughly achieved when teachers engage in frequent, continuous, and increasingly concrete and precise talk about teaching practice . . . adequate to the complexities of teaching, [and] capable of distinguishing one practice and its virtue from another.” --Judith Warren Little

23. Fundamental Concepts of Collaborative Learning Communities • Teachers establish a common, concise set of essential curricular standards and teach to them on a roughly common schedule. • Teachers meet regularly as a team for purposes of talking in “. . . concrete and precise terms” about instruction with a concentration on “thoughtful, explicit examination of practices and their consequences.” • Teachers make frequent use of common assessments. Continued on next slide

24. “These elements, so rarely emphasized in school . . . improvement plans, deserve our attention more than anything else we do in the name of school improvement.” --Mike Schmoker (2006)

25. Components of THE NEW MODEL THE CLASSROOM-FOCUSED IMPROVEMENT PROCESS (CFIP): A Team Data Dialogue Protocol

26. The new process needs to be built on: 1. Dialogue 2. Protocols 3. Triangulation of Data (use of multiple data sources)

27. Our Goal in the Data Dialogues: Frequent, continuous, and increasingly concrete and precise dialogue by school teams, informed by data

28. What are the right teams to conduct data dialogues? • Teams that share common standards and assessments • Grade-level teams • Content teams • Vertical teams

29. When is the right time to conduct data dialogues? • At a minimum, devote at least one hour to data dialogues every two weeks. • According to several studies, schools that realized the greatest results from a shift to a data culture scheduled data dialogues at least once a week.

30. Frequency of Data Dialogues • Source: Stanford University, Stanford Research Institute, Education Week, January 24, 2004

31. What are the right data to use in the data dialogues? • Triangulate three types of data: • External Assessment Data • Course-wide Benchmark Assessment Data • Classroom Assessment Data • --Supovitz & Klein (2003)

32. What is the right plan where the results of the data dialogues should be used? • Conclusions are specific to students in the class. • Conclusions are used to plan upcoming daily instruction. • The plans are implemented.

33. What is the right way to use the results of the data dialogues? • Conclusions are used to identify enrichments and interventions for the students in the class. • Conclusions are used to plan upcoming daily instruction.

34. What Is a Data Protocol? A protocol consists of guidelines for dialogue – which everyone understands and has agreed to – that permit a certain kind of conversation to occur, often a kind of conversation which people are not in the habit of having. Protocols build the skills and culture necessary for collaborative work. Protocols often allow groups to build trust by doing substantive work together.

35. Using a Data Protocol Protocols can help us to navigate difficult and uncomfortable conversations by: • Making it safe to ask challenging questions • Making the most of scarce time • Providing an opportunity for all to be involved • Resulting in an analysis that will lead to positive action

36. Using a Data Protocol • The point is not to do the protocol well, but to have team dialogue that is: • In-depth • Insightful • Concrete • Precise

37. Six Easy CFIP Steps 1. Understand the data source. 2. Begin with a question. 3. Look for class-wide patterns. 4. Act on the class patterns. 5. Address individual students’ needs. 6. Improve instruction in the next lesson.

38. CFIP Step 1: Understand the datasource. • Build ASSESSMENT LITERACY with questions like these: • What assessment is being described in this data report? What were the characteristics of the assessment? • Who participated in the assessment? Who did not? Why? • Why was the assessment given? When? • What do the terms in the data report mean? • Be sure you have clear and complete answers to these questions before you proceed any further.

39. CFIP Step 2: Identify the questions that can be answered by the data. • All data analyses should be designed to answer a question. • Unless there is an important question to answer, there is no need for a data analysis.

40. CFIP Step 3A: Look for class-wide patterns in a single data source. • What do you see over and over again in the data? • What are the strengths of the class? What knowledge and skills do the students have? • What are the weaknesses of the class? What knowledge and skills do the students lack?

41. CFIP Step 3B: Identify patterns of class strengths and weaknesses from multiple data sources. • TRIANGULATION • In what ways are the results similar among data sources? For example, how do benchmark test results compare with ongoing classroom assessment data? • In what ways do the results among data sources differ? • Why might these differences occur?

42. Power When Multiple Types of Data Are Used • Reduces the anxiety and the mistakes of relying on a single measure as the only definition of student success • Provides more frequent evidence on which to act • Develops and sustains a culture of inquiry in the school based on data

43. CFIP Step 4: Act on the class-wide patterns. • What instructional factors might have contributed to the class-wide patterns? • What will we do to address patterns of class needs? • How and when will we reassess to determine progress?

44. CFIP Step 5: Drill down to individual students. Identify and implement needed enrichments and interventions. • What are the implications for enrichments and interventions from what you learn from the data? • Which students need enrichments and interventions? • What should enrichments and interventions focus on?

45. CFIP Step 6: Reflect on the reasons for student performance -- What in our teaching might be preventing all students from being successful? To what extent did we implement research-based instructional practices as we: • Planned instruction? • Introduced instruction? • Taught the unit? • Brought closure to instruction? • Assessed formatively?

46. CFIP Step 6: Reflect on the reasons for student performance. Identify and implement instructional changes in the next unit. • How will we change instruction in our next unit? • Content focus • Pacing • Teaching methods • Assignments

47. CFIP Step 6: Reflect on the reasons for student performance. Identify and implement instructional changes in the next unit. • When will we review the data again to determine the success of the enrichments, interventions, and instructional changes? • What do the data not tell us? • What questions about student achievement do we still need to answer? • How will we attempt to answer these questions?

48. The Next Steps • Unless teams emerge from the data analysis process with a clear plan of action for their classroom, they have wasted their time. • Implement the plan of interventions, enrichments, and changes in instruction. • Collect the next set of data.

49. Six Easy CFIP Steps 1.Understand the data source. 2. Begin with a question. 3. Look for class-wide patterns. 4. Act on the class patterns. 5. Address individual students’ needs. 6. Improve instruction in the next lesson. You may wish to stop the presentation at this time and review the six steps again, keeping in mind that teams will study and practice using the steps prior to their implementation of CFIP.

50. Caveats about CFIP • It is a paradigm shift from the traditional lesson planning format. • It is not easy, especially at first. • Follow the steps faithfully until they become second nature. • Expect mistakes and imprecision in the data. • The results are worth the effort.