Using Multiple Sources of Data to Measure Success

1. Using Multiple Sources of Datato Measure Success December 15, 2009 School Improvement Webinar Series www.acteonline.org/multimedia.aspx

2. Your Moderator, Host and Presenter Diana Rogers • Regional Coordinator • HSTW NE Ohio Region Catherine Imperatore • Electronic Media Manager • ACTE Mike Ross • HSTW/MMGW School Improvement Coach • HSTW SW Ohio Region

3. Sound Check • Can everyone hear me? • If not, please check the volume on your computer

4. Questions To ask about the content type a question in the Q&A panel and send to All Panelists. Questions will be addressed at the end of the presentation For technical problems or any other questions, type in the Chat panel and send to the Host.

5. Replay or Register for Webinars Replay/register www.acteonline.org/multimedia.aspx Invite your colleaguesto register Complete webinar survey Graduate credit available

6. Why do we need multiple sources? • Single sources of data don’t provide us with the complete picture! • Reliance on a single data source is incomplete! • We need multiple sources of data to more accurately identify causes of the problem and to find more appropriate solutions.

7. Why do we sometimes have “tunnel vision” when looking at data? • Emphasis on paper and pencil testing for accountability • “Reaching the standard,” “making the cut-off score,” etc. may blur our focus • Absence of a data culture…and resources and time for creating one!

8. Data are critical for all parts of the improvement process – from initial needs assessments to monitoring progress and evaluating outcomes/results

9. Data Smart Information Rich Don’t Know Don’t Care “Sure we use data… I think.” Data Rich Information Poor A Data Continuum – Where on this continuum is your school?

10. Delving into Data • School culture is everything! • Data are accessible and usable. • Data are viewed as an invaluable resource for improvement. • Data serve as a basis for inquiry, reflective dialogue, problem-solving, and decision-making. • Discrepant data provide the “teachable moment” in the school improvement process.

11. Bernhardt’s Suggested Use of Data • Replace hunches with facts concerning what changes are needed • Identify the root causes of problems so we can then solve the problems • Assess needs to target our services on important issues • Know if goals are being accomplished • Determine if we are “walking our talk” From the work of Victoria L. Bernhardt, Data Analysis for Comprehensive School Improvement, 1998, Eye on Education.

12. Quantitative versus Qualitative • Is one form of data better than the other? • What is the purpose for using the data? • Both forms can be effectively utilized!

13. Categories of Data “Measures of student learning help us understand how students are performing and what students know as a result of instruction.” Student Learning From the work of Victoria L. Bernhardt, Data Analysis for Comprehensive School Improvement, 1998, Eye on Education.

14. Categories of Data School Processes “…programs, practices, and instructional strategies…that produce school and classroom results.” From the work of Victoria L. Bernhardt, Data Analysis for Comprehensive School Improvement, 1998, Eye on Education.

15. Categories of Data Perceptions “A particular view, judgment, or appraisal formed in the Mind about a particular matter...a belief stronger than impression And less strong than positive knowledge.” From the work of Victoria L. Bernhardt, Data Analysis for Comprehensive School Improvement, 1998, Eye on Education.

16. Categories of Data “Statistical characteristics of human populations…builds the context of the school…for which change is planned and takes place.” Demographics From the work of Victoria L. Bernhardt, Data Analysis for Comprehensive School Improvement, 1998, Eye on Education.

17. Categorizing Data • D – Demographic • P – Perception • SP – School Process • SL – Student Learning From the work of Victoria L. Bernhardt, Data Analysis for Comprehensive School Improvement, 1998, Eye on Education.

18. What category? • An additional hour of extra help is available after school this year. • Only 78% of boys scored at the proficient level in reading as compared to 92% of girls. • For two years, “making inferences using non-fiction passages” has been our lowest performing area.

19. Poll Activity • How would you categorize the following data: • D – Demographic, P – Perception, SP – School Process, SL – Student Learning • Our math scores have declined during the past three years. • Students classified as low socioeconomic status score less well on constructed response test items. • Last week, problem-based learning was observed in about 35% of the classrooms. • None of the students in the focus group discussion mentioned project work as a “quality” learning experience. • Over 90% of parents said they were satisfied with the new grading scale.

20. How may more than one category interact for a better data analysis? • How do students who regularly use the writing lab perform on their senior projects compared to those who don’t? • Does the reading performance of males increase with the number of hours intervention they have experienced? • Do the parents of underperforming students have confidence in the school’s enhanced intervention programs?

21. Perceptions School Processes Student Achievement Demographics Categories of Data - Interactions From the work of Victoria L. Bernhardt, Data Analysis for Comprehensive School Improvement, 1998, Eye on Education.

22. How Data Can Be Used • To guide improvement efforts! • Provide students with feedback on their performance • Gain common understanding of what quality performance is and how close we are to achieving it • Measure program effectiveness • To understand if what we are doing is making a difference

23. More Ways Data Can Be Used • Make sure students don’t “fall through the cracks” • Know which programs are getting the results we want • To get to the real causes of problems • To guide curriculum development • Promote accountability • Meet state and federal requirements • (all from V. Bernhardt) From the work of Victoria L. Bernhardt, Data Analysis for Comprehensive School Improvement, 1998, Eye on Education.

24. An “Overload” of Data??? • OGT score rosters…frequency distributions…extra-help options and sign-in sheets…quarterly failure reports… IPDP… ICP… IRN…disaggregated OGT percent passing…subscale analysis…test item analysis…Student Assessment…Teacher Survey…Annual Progress Report…Ohio School report card… student grades… scheduling demand… similar district comparison… course enrollment…curriculum mapping …attendance by date…attendance by day of week… tardies… discipline…free and reduced lunch…disability profile…categorical report…IEP achievement...parent income level…drop-out rate…graduation rate…completer status…national certification …health records…mobility…families on public assistance…books read…credit deficiencies…teacher licensure data…CCIP…program evaluation reports… student survey … parent/community survey… open house attendance…parent conference attendance…student participation in extra-curricular activities…safe school survey…equity survey… harassment complaints… ACT…SAT… Board of Regents remediation reports… suspensions… expulsions…gifted and talented…advanced placement…post-graduate follow-up survey…percent students entering post-secondary training…alternative program…reasons for drop-out…level of technology

25. State Assessments Teacher Assessments Course Failure (ninth-grade) ACT/SAT Results Attendance Rates Graduation Rates Certification Exam Results Post-Secondary Readiness Assessing Readiness Practice Sources of Data for Measuring Performance and Practices

26. Sources of Data for Measuring Performance and Practices • Instructional Review • Staff Experience Chart • Remedial Studies Reports • Follow-up studies • Drop-out exit reports • Master Schedule • Focus Group Interviews • Graduate Feedback • Assessing Practice

27. Compiling Data Sets • Collect related data appropriate to a particular goal, objective or strategy • Organize data for review • Develop thoughtful questions for making meaning • Draw conclusions

28. Example of Data Sets • Overriding Goal: To close achievement gaps, meet state and federal accountability requirements…to maintain high expectations and extra help. • What data sets are appropriate for addressing this goal?

30. Elements of Data Sets • Organize data for review • How should we best organize the data?

32. Elements of Data Sets • Overriding Goal: To close achievement gaps, meet state and federal accountability requirements…to maintain high expectations and extra help. • What questions will help us to make the data meaningful?

33. Elements of Data Sets • Overriding Goal: To close achievement gaps, meet state and federal accountability requirements…to maintain high expectations and extra help. • What conclusions may be draw from analyzing our data?

34. …on being the best professional “Over everything else, all the tools they have in their toolboxes, the difference between ineffective and effective teachers is that effective teachers reflect on their teaching in meaningful ways.” “Ineffective teachers may work hard, but they don’t move along the continuum of self-reflection from unaware, to consciousness, to action, to refinement.” - Pete Hall ASCD 2005 • Outstanding Young Educator Awardee

35. Data Helps Us to Reflect • Effective teachers consciously use data as a source of guidance and reflection. • The use of multiple sources of data provides a more accurate and complete picture of performance and effectiveness. • Make time for using data and reflecting on your practice.

36. Recommended Resources Book: Victoria L. Bernhardt, Data Analysis for Comprehensive School Improvement, 1998, Eye on Education. Website: Common Core of Data, National Center for Educational Statistics http://nces.ed.gov/ccd/

37. Questions To ask about the content type a question in the Q&A panel and send to All Panelists. Questions will be addressed at this time Or an email response will be sent to you after the webinar.

38. Question Do you have examples of data sets used by schools to measure student success?

39. Question What professional development is available to assist school teams in learning more about using multiple sources of data?

40. More Q & A Questions and responses

41. Contact Information If you have questions or would like to learn more about using multiple sources of data to measure success, please contact: Mike Ross, School Improvement Consultant michaelross@embarqmail.com

42. Next Webinar in the Series Developing a School-wide Literacy Plan Paulette Dewey, HSTW/MMGW Coach January 19, 2010 from 11:30 – 12:30 ET

43. Thank you for participating! Reminders… Register for future webinars or to view archived webinars at www.acteonline.org/multimedia.aspx or www.hstwohioregions.org After leaving today’s webinar… Please complete the webinar survey. If you are interested in graduate credit, remember to print a copy of the survey. Please click the X to exit the webinar. Have a great day!