Progress Monitoring in an RTI Model

1. April, 2011 Progress Monitoring in an RTI Model

2. The ultimate goal of assessment is to identify problems with instruction and to lead to instructional modifications. A good share of present-day assessment activities consist of little more than meddling…We must use assessment data to improve instruction…The only way to determine the effectiveness of instruction is to collect data. Ysseldyke and Algozzine (1995) Basic Goal of Assessment Curriculum-Based Measurement: Introduction

3. Objectives • Learn levels of assessment & associated purposes • Increase knowledge and skills in curriculum-based measurement (CBM) • Apply knowledge of CBM to diagnostic assessment practices • Learn can’t do/won’t do assessment • Learn survey level assessment

4. Objectives (continued) • Learn goal setting formula • Apply graphing techniques • Analyze data

5. A + B = pC

6. Define the Problem Defining Problem/Directly Measuring Behavior Problem Analysis Validating Problem I.D. Variables that Contribute to Problem Develop Plan Evaluate Response to Intervention (RtI) Implement Plan Implement As Intended Progress Monitor Modify as Necessary Problem Solving Process DATA

7. Response to Intervention • RtI is the practice of (1) providing high-quality instruction/intervention matched to student needs and (2) using learning rate over time and level of performance to (3) make important educational decisions (Batsche, et al., 2005) • Problem-solving is the process that is used to develop effective instruction/interventions.

8. Anita Archer • Relentless in education • Ability and outcome

9. Formative vs. Summative • “When the cook tastes the soup, that’s formative. When the guests taste the soup, that’s summative.” - Robert Stake

10. Benchmarking & Progress MonitoringWhat is the difference? • Benchmark: aid in identification of students at-risk; administered 3 times a year (fall, winter, spring) • Progress Monitoring: used to track individual students’ learning, plan instruction, and provide feedback to students; administered weekly or biweekly

11. Why CBM?

13. Benchmarking & Progress MonitoringWhat is the difference? • Benchmark: aid in identification of students at-risk; administered 3 times a year (fall, winter, spring) • Progress Monitoring: used to track individual students’ learning, plan instruction, and provide feedback to students; administered weekly or biweekly

14. CBM – What is It? • Formative assessment • Measure of student performance over time • An analysis of specific skill on an individual student • Tool • Identifying struggling students • Set goals • Align instruction with desired outcomes • Provides diagnostic information • Progress monitor • IEP development

15. What is the Goal of CBM? • Goal is two-fold: • Monitor student progress • Inform instruction / teacher practice

16. Why Should I Do It? • Measure of class-wide performance • An alternative to other assessment procedures– often replaces costly, time-consuming, disruptive practices • Quick & Easy • Establishes reliability & validity • Direct low-inference measures • Can be easily summarized & presented • Parents, students, colleagues

17. B D I P

18. B Benchmarking - Norms Diagnostic Assessment D - Can’t do/Won’t Do - Survey Level Intervention Selection I - Standard Treatment (Tier II) - Who, where, frequency, duration, materials Progress Monitoring & Analysis P - Data collection - Data Management - Next Steps

19. Assessment and MTSS Tier 3 Tier 2 Measurement Precision Measurement Frequency Problem Analysis Tier 1 Adapted from Burns & Riley-Tillman (2010)

20. Assessment and MTSS • Tier III – Identify discrepancy for individual. Identify causal variable. Implement individual intervention. • Tier II – Identify discrepancy for individual. Identify category of problem. Assign small group solution. • Tier I – Identify discrepancy between expectation and performance for class or individual. Tier 3 Tier 2 Measurement Precision Measurement Frequency Problem Analysis Tier 1 Adapted from Burns & Riley-Tillman (2010)

21. How Does It Fit Together? Step 1 Step 2 Step 3 Step 4 Additional Diagnostic Assessment Instruction/Intervention Results/Monitoring All students at a grade level Individual Instruction Individual Diagnostic Weekly Intensive 5% Group Diagnostic Small Group Differentiated by Skill Universal Screening Targeted 15% 2x month Fall Winter Spring Continue with Core Instruction Benchmarks Grades Classroom Assessments Utah CRT None Universal 80%

22. B - Benchmarking • What - An indicator used to identify the expected understandings and skills needed for content standards by grade level - consider norms • When - typically 3x/year - predetermined intervals (e.g., a mid-year benchmark). • Who - all students • How - timed leveled probes

23. What is Reading CBM? • One-minute probe (e.g., DIBELS, 6-min. Solution) • Administered individually • Provide intervention and progress monitor at instructional level • Different measures • Oral Reading Fluency (ORF) • Maze (Comprehension) • Early-reading (Initial Sound, Phoneme Segmentation, Nonsense Word, Letter Naming Fluency) • (See Chapters 3 & 4 in ABCs of CBM)

24. Select appropriate material for probe Place probe in front of and facing the student Keep copy for the examiner (on clipboard) Provide directions Start timer Have student perform task for allotted time (1 minute for reading tasks) Score probe Display data on graph/chart Video Clips . . . . Examples Reading CBM - How?

26. Scoring Reading Probe • Oral Reading Fluency: • Mark as correct • # of words read correctly in one minute • Mark as incorrect: • Misread words • Omissions • Hesitations - words read by assessor (read after 3 seconds) • Reversals – two or more words not read in order • (see page 146 in ABCs of CBM)

27. What is Math CBM? • Math CBM can be broken into three areas: • Early numeracy • Computation • Concepts and Applications • Focus will be on administering, scoring, and using CBM for Computation

28. Math CBM-How? • Math CBM is conducted having students answer computational problems for two minutes • Count correct digits – NOT correct problems • Use standardized procedures • Have necessary materials • Different but equivalent math sheets • Directions for administration • Writing utensils • Stopwatch • Quiet environment • Scoring rules & procedures

29. Select appropriate material for probe Place probe in front of and facing the student Can be administered individually or as an entire class Provide directions Start timer Have student perform task for allotted time (2 minutes for math tasks) Score probe Math CBM-How? (cont) Curriculum-Based Measurement: Introduction

30. Administration Practices • First time administration • Have three equivalent math sheets • Recommend doing it in one session • Median score of three samples used for baseline (first data point on graph) • Every time • Standardized procedures (see pg 156) • Count correct digits • Record on graph • Use for decision making

31. Scoring Procedures • Correct answers . . . • Correct answers gets credit for longest method used • Correct digits for unfinished problem • Correct digits for reversed or rotated digits • Correct digits for placeholder – any symbol! • Errors • Mark with slash (/)

32. Expected Growth Rates & Norms • Benchmarks - Table 7.1 (p. 109) • Norms – Table 7.3 (p. 111) • Growth Rates – Table 7.1 (p. 109) • greater progress is possible • If student doesn’t make adequate progress, it doesn’t mean she lacks the ability to learn math – it means instruction needs to be changed!

33. Math CBM Sheets • Math often has a specific scope & sequence that can vary from state to state, school to school, or curriculum to curriculum • Educators often create math sheets linked to the state’s core – progress monitoring aligned with outcome measure! • Math sheets should have different problems but equivalent difficulty • Can be purchased or created

34. Math Sheets (cont) • Items on sheet not in order presented in curriculum • Similar item types are grouped diagonally – assists when looking for patterns in student responses (see page101) • Consider mixed vs. single skill • Single-skill sheet helpful for short-term planning – used to gain some diagnostic information & assist in decision making

35. CBM – Types & Their Purpose (see page 15) • General Outcome Measures (GOMs) – used to sample performance across several goals by using “capstone” tasks that are complex • Skills-based Measures (SBMs) – used to screen, progress monitor, & do survey-level assessment where “capstones” not available • Mastery Measures (MM) – used on parts of curriculum that contain discrete & easily identified sets of items

36. Diagnostic - Can’t Do/Won’t Do • Purpose • Determine motivation vs. skill deficit • Technique • Administer same probe – add incentive • Timing - Soon after benchmark/screener • Decision Rules • >=15% increase=motivation (Witt & Beck, 1999) • <15% skill deficit • Consider both

38. Diagnostic Survey-Level Assessment • Purposes • To determine the appropriate instructional placement level for the student. • The highest level of materials that the student can be expected to benefit from instruction. • To provide baseline data, or a starting point, for progress monitoring • In order to monitor progress toward a future goal, you need to know how the student is currently performing.

39. 1. Start with grade level passages/worksheets (probes) 2. Administer 3 separate probes (at same level of difficulty) using standard CBM procedures. 3. Calculate the median score (i.e. the middle). 4. Is the student’s score within instructional range? Yes - This is the student’s instructional level. No - If above level (too easy), administer 3 probes at next level of difficulty. No - If below level (too hard), administer 3 probes at previous level of difficulty. Survey Level Assessment-Reading

40. Reading CBM – Tasks by Grade • Kindergarten – Letter sound fluency (LSF) • Grade 1 – Oral reading fluency (ORF) and/or word identification fluency (WIF) • Grade 2 – Oral reading fluency (ORF) • Grade 3 – Oral reading fluency (ORF) • Grade 4+ - Oral reading fluency (ORF) & mazes • See Chapters 3 & 4

41. Reading CBM – Norms & Growth(see pages 47 & 49) • Norms • Compare student’s score to the performance of others in her grade or at her instructional level • Data collected on thousands of students – numbers are very similar • Growth Rates • Provide an indication of the average number of words per week we would expect students to improve • Not necessarily new words - students reading same words at a faster rate each week

42. 1. Start with grade level math worksheets (probes) 2. Administer 3 separate probes (at same level of difficulty) using standard CBM procedures. 3. Calculate the median score (i.e. the middle). 4. Is the student’s score within instructional range? Yes - This is the student’s instructional level. No - If above level (too easy), administer 3 probes at next level of difficulty. No - If below level (too hard), administer 3 probes at previous level of difficulty. Survey Level Assessment-Math

43. Math CBM – Tasks by Grade • Grade 1 – Addition, subtraction • Grade 2 – Addition, subtraction • Grade 3 – Addition, subtraction, multiplication, division • Grade 4+ - Multiplication, division • All Grades – Mixed-math skills • Secondary – consider common assessments based on state or district core

44. What if Student Data Doesn’t Reflect Adequate Growth? • It is our obligation to fix the problem! • Build up prerequisite skills • Increase length of daily lesson • Alter way we respond when error is made • We do NOT lower expectations! • “Learning is a result of instruction, so when the rate of learning is inadequate, it doesn’t always mean there is something wrong with the student. It does mean the instruction needs to be changed to better meet the student’s needs.” (p. 47)

45. How Often Should Data Be Collected? • Three considerations: • 1. Purpose – screening vs. benchmarking • 2. Importance of task – learning to read vs. learning Roman numerals • 3. Significance of problem – student’s difficulty increases need effective instruction need more frequent monitoring

46. I - Intervention

47. P - Progress Monitoring

48. How to Set & Graph Goals • 1. End of Year Benchmarks • 2. Norms - Levels of performance • 3. Rate of progress – goal setting • (# of weeks x growth rate) + median baseline = goal • Students with greatest deficits need steepest slopes – more intense & effective interventions

49. CBM – Easy To Display • One of the main benefits of CBM is the data are easily displayed in graphs & charts • Standard Graph for CBM line graph • Vertical axis = skill being measured • Horizontal axis = time or sessions

50. Graphing – What’s Included • Baseline • Y Axis Label • X Axis Label • Aim Line - Goal • Data Points • Intervention Line • Intervention Line Label • Change Line • Change Line Label • Level of material being used • Student demographics