Innovations in the Assessment of reading fluency Elizabeth Meisinger, Ph.D. Katherine Price, M.S. Tera Traylor, B.A.

1. Innovations in the Assessment of reading fluencyElizabeth Meisinger, Ph.D.Katherine Price, M.S.Tera Traylor, B.A. University of Memphis

2. What is Reading Fluency? • The ability to read quickly, accurately, and with appropriate expression (NRP, 2000). • Typically focus on large blocks of text , but you can define more narrowly at letter, word, or phrase level (Kuhn & Stahl, 2004). • Oral versus silent reading

3. Why is reading fluency important ? • Supports comprehension in early elementary school. • Automaticity theory (LaBerge & Samuels, 1974) • By fourth grade children shift from learning to readtoreading for the purpose of learning content area knowledge (Chall, 1996).

4. Relevance to School Psychology • Response to Intervention (RTI) • CBM Oral reading fluency probes • Conceptualized as a general indicator of reading competency (Fuchs et al., 2001). • Specific Learning Disability in Reading Fluency • Subtype added in reauthorization of Individuals with Disabilities Education Improvement Act (IDEIA, 2004).

5. Overview • Brief review of oral reading fluency measures • Review understudied aspects of fluency • Prosody • Miscues • Silent reading • Psychometric properties • Utility in practice of school psychology

6. Assessments of Oral Reading Fluency • Criterion-referenced tests • CBM (Localized, DIBELS, AIMSweb) • Informal Reading Inventories (ex. QRI-4) • Norm-referenced tests • Gray Oral Reading Test, 4th Ed. (GORT-4) • Text level • Test of Word Reading Efficiency (TOWRE) • Word level • Kaufman Test of Educational Achievement, 2nd Ed., (KTEA-2) • Word level • Wechsler Individual Achievement Test, 3rd Ed. (WIAT-III) • Text and prosody

7. Prosody • Appropriate intonation or expressiveness coupled with phrasing that allows for the maintenance of meaning (Miller &Schwanenflugel, 2006, 2008). • Ex. Jake’s going to the dance with Jenifer. • Often viewed by teachers as important for reading comprehension.

8. Prosodic Measures: Rating Scales • National Assessment of Educational Progress (NAEP) oral Reading Fluency Scale (Pinnell et al., 1995; Daane et al., 2005) • 4 point scale • primarily word-by-word with occasional 2- or 3-word phrases; • primarily 2-word phrases with some 3- or 4-word phrases; • Primarily 3- and 4-word phrase groups with little expression; • Expressive reading in larger, meaningful phrases with few regressions, repetitions, or deviations. • Inter-rater agreement 81% (exact) to 100% (adjacent) (Daane et al., 2005)

9. Rating Scales cont. • Multidimensional Fluency Scale (Raskinski et al., 2009; Zutell&Raskinski, 1991) • 3 subscales, 1-4 rating • Phrasing and expression • Smoothness and accuracy • Pacing • Inter-rater agreement 86%-99% • Test-retest reliability .90 • NAEP & MDFS correlate moderately with silent reading comprehension

10. Spectrographic Analysis(Kuhn, Schwanenflugel, &Meisinger, in press) • Digital recordings of readings • Comprehensive speech software programs • Allows for specific analysis of a variety of prosodic features: • Pitch, stress, duration, & pausing

11. Prosody’ s Role in Reading Development(Kuhn, Schwanenflugel, &Meisinger, in press) • Prosody is related to the development of fluent reading (fast, accurate text reading). • The relation between prosody and comprehension is less clear. • Does comprehension lead to prosody? • Does prosody support comprehension?? • Is the relation reciprocal???

12. Future Implications for Practice (Kuhn, Schwanenflugel, &Meisinger, in press) • Compare spectrographic measures and rating scales to determine reliable, valid assessments for classroom use. • Integrate prosody into assessments of reading fluency. • Psychoeducational reports • Progress monitoring • Address prosody in reading fluency interventions (if prosody contributes to comprehension).

13. Oral reading accuracy • Word reading accuracy is important component of reading fluency. • Readers who have difficulty pronouncing individual words experiencing difficulties deriving meaning from text (Harris & Hodges, 1995). • Fluent readers translate words quickly and easily, freeing attention for comprehension (LaBerge& Samuels, 1974). • Difficulties with word recognition is most pervasive cause of reading disability (Perfetti, 1985; Stanovich, 1986; Vellutino, Fletcher, Snowling, & Scanlon, 2004).

14. Miscue Analysis • The Reading Wars and Kenneth Goodman • Miscues are oral reading errors: a point in reading where the observed response does not equal the actual response (Goodman, 1973). • Errors provide valuable information about how the reader is engaging with the text. • Examines reading errors in context (whole passage) • Text: She put down her foot. • Child reads: “She put down her food.”

15. Miscue Analysis • Assumes that children use “ 3 cueing system” to help them construct meaning from the text: semantic, syntactic, and graphophonic cues • Goal of analyzing miscues is to determine if reader is using all 3 cues to help them identify words • Errors coded using a taxonomy • Reading Miscue Inventory (Goodman, Watson, & Burke, 2005) • Qualitative information can be collected and compared across time.

16. Miscue Analysis • Types of miscues commonly recorded and coded • Omissions • Insertions • Self-corrections • Repetitions • Substitutions • Grapho-phonically similar/dissimilar • Semantically similar/dissimilar • Syntactically similar/dissimilar

17. Excerpted from Reading Miscue Inventory (Goodman, Watson, & Burke, 2005)

18. Classroom Implications • Teachers exposed to miscue analysis in education programs • Some IRIs and tests of reading ability contain miscue analysis procedures (e.g., Qualitative Reading Inventory-4&GORT-4). • Running records use simplified miscue analysis procedures. • Instruction can be tailored to pattern of miscues • Reading Recovery

19. Research • NAEP Special Study of Oral Reading-- 4th grade students (Daane et al., 2005) • Students who committed fewest errors demonstrated better comprehension scores (NAEP reading assessment). • Oral reading errors, regardless of their effect on text meaning, were negatively related to comprehension. • A positive relationship exists between the proportion of errors self-corrected and comprehension score.

20. Research • Substitutions are the most common types of miscues, followed by omissions, then insertions (Goodman, 1976; D’Angelo&Mahlios, 1983). • The percentage of miscues acceptable within the context of the text increases as a function of grade level and reading ability (Christie, 1981). • Omissions and substitutions of phonologically similar words are best predictors of comprehension (Laing, 2002).

21. Research • Insertions and omissions cause few syntactic and semantic distortions, and do not interfere with comprehension (D’Angelo&Mahlios, 1983). • Specific miscue types are not significantly correlated with reading comprehension (Englert&Semmel, 1981).

22. Some Considerations • No empirical support exists for 3 cueing system • Procedure is complicated and time-consuming • Analysis is subjective: coding may vary depending on passage characteristics and judge • Little is known about how teachers actually use miscue analysis in the classroom • Suggestions for modified use (McKenna & Picard, 2006)

23. Future Research • More information needed regarding types of miscues related to comprehension in early grades • Does miscue analysis provide enough bang for the buck? • Teacher use of miscue analysis in the classroom

24. Implications for Practice • Qualitative writing samples may be used to supplement reading assessment • However, little empirical support for how specific miscues relate to comprehension • Familiarity with teacher assessment practices can facilitate better communication

25. Why silent reading fluency? • Oral reading fluency is often conceptualized as a proxy for general reading competence due to its close relation to comprehension in early elementary school. • i.e., can screen for general reading skill quickly with ORF probes • This relation decreases across development (Vellutino et al., 2004). • Further, children typically transition to primarily silent reading by fifth grade, suggesting that it may become important to utilize silent reading assessments in older children.

26. Silent Reading Assessment • Range of traditional and emerging assessment options: • paper-and-pencil • group administered sentence reading • computerized tasks • Merits and detractions of each type of assessment • Brief overview of areas in silent reading that currently lack strong empirical examination.

27. Paper-and-pencil Assessment • Most common and familiar • Typically involves the reader silently reading from a passage and marking or circling the last word read when the administrator calls time (often after 1 minute) (Fuchs et al., 2001) • Advantages • Familiar and ecologically valid • Inexpensive and easy • Group or individual administration • Flexible, brief • Can be curriculum-based • Disadvantages • Self-report--Is the kid actually reading? • Dearth of information

28. Paper-and-pencil Assessment: Psychometrics • Relation to Comprehension: • .38 (questions tied to passages) (Fuchs, et al., 2001) • .47 (ITBS reading comp) (Fuchs, et al., 2001) • .48 (GMRT) • .42 (Maze task) • Relation to Other Measures of Silent Reading: • Self-paced reading: .61 • Underlining: .58 • Obtained Reliability: • .86 (alternate form)

29. Test of Silent Reading Efficiency and Comprehension (TOSREC) • Recently released and available through Pro-Ed • (though it’s on backorder) • Assesses silent reading of connected text for comprehension • Student reads a series of sentences and circles the answers to comprehension questions following each sentence; the test is administered in 3 minutes • Advantages • Commercially available • Multiple forms • Nationally normed for Fall, Winter, and Spring or progress monitoring • Group or individually administered • Ecologically valid • Comprehension measure • Brief • Disadvantages • Comprehension is directly tied to the score • The test contains sentences instead of longer passages

30. Computerized Measures • Self-pacedreading methods • Based on the assumption that a participant will read at a rate that matches the comprehension process • Analysis of reading rate will reveal information about comprehension and the reading process • Three main categories: • Eye-tracking • Window Methods (not talking about today) • Underlining

31. Eye-Tracking • Text is exposed on a screen and the reader’s eye movements are tracked by computer software • Movements can be sampled up to 1000 times per second • Allows for an in-depth analysis of eye fixations • Where, How long (gaze length), Regressions, how many times the reader fixates on an individual word, etc. • No comparable psychometrics information available • Advantages • Large amount of in-depth information • Intra-word analyses • More analyses available—i.e., silent reading prosody, • Flexible (can be brief) • Disadvantages • Expensive to set up • Some set ups may be difficult to use with children especially in a school setting • Individually administered

32. Underlining • Combines elements of paper-and-pencil and eye-tracking • Text is read from the screen of a tablet PC, and the reader is asked to underline the text “on-line” with his or her reading • Underline each word as they read it, pause their underlining if they pause in their reading, go back and re-underline any regressions • Specialized software is used to track the location of the mouse on the screen • Allows for a deeper analysis than do paper-and-pencil assessments, without some of the disadvantages of eye-tracking • Advantages: • Less expensive than eye-tracking • Ecologically valid • Can be group administered • Greater amount of information • Brief, flexible • Disadvantages: • No intra-word information • No normative information • More expensive than paper-and-pencil methods

33. Example of an Adult with Regression

34. Example of 4th Grade Student’s Passage

35. Underlining:Psychometrics • Relation to comprehension: • 4th: GMRT: .41 Maze task: .53 • 6th: GMRT: .47 Maze task: .37 • Relation to other measures of silent reading: • 4th: SPR: .49 Paper-and-Pencil: .59 • 6th: SPR: .59 Paper-and-Pencil: .83 • Obtained Reliability: • .96 (alternate form)

37. Applications in Practice • Assessment of silent reading in older grades as opposed to oral reading • May make more sense after 4thgrade due todecreased ties to comprehension in older grades • Could be used in lieu of ORF as a brief measure of general reading competence for progress monitoring or screening • More choices available beyond paper-and-pencil assessment depending on what information you are seeking out of your assessment

38. Areas for Future Research • When does the shift occur? • How is this shift moderated by reading ability? • How do traditional and emerging silent reading measures compare to eye-tracking measures? • Seen in the literature as the gold standard for information about reading processes • How does the theoretical model of silent reading development differ from those of oral reading development?

39. Questions? Comments? bmsinger@memphis.edu