1 / 11

A Causal Rasch Model for Understanding Comprehension in the Context of Reader-Text-Task

A Causal Rasch Model for Understanding Comprehension in the Context of Reader-Text-Task. AERA/NCME April 13-17, 2012 Vancouver, Canada A. Jackson Stenner Donald S. Burdick Mark H. Stone.

aria
Download Presentation

A Causal Rasch Model for Understanding Comprehension in the Context of Reader-Text-Task

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Causal Rasch Model for Understanding Comprehension in the Context of Reader-Text-Task • AERA/NCME April 13-17, 2012 • Vancouver, Canada • A. Jackson Stenner • Donald S. Burdick • Mark H. Stone

  2. Reading is a process in which information from the text and the knowledge possessed by the reader act together to produce meaning as measured by a particular task. Anderson, R.C., Hiebert, E.H., Scott, J.A., & Wilkinson, I.A.G. (1985) Becoming a nation ofreaders: The report of the Commission on Reading Urbana, IL: University of Illinois

  3. Data Structure for Testing the Lexile Theory Texts (Native Task) a b c d e f g h i j k l m … N 1 2 3 4 5 6 7 8 9 10 11 12 13 … N Reader-text-task transaction produces an outcome which can be viewed as reading comprehension and/or text comprehensibleness usefully presented as a percent. Readers Reader Ability Measures Empirical Text Complexity Measures New Task Types

  4. A Causal Rasch Model Conceptual Reader Ability - Text Complexity Task Difficulty = Comprehension - Statistical e (RA – TC - TD) i Raw Score = 1 + e (RA – TCi - TD) i RA = Reading Ability TC = Text Complexity TD = Task Difficulty

  5. The Measurement Trade-off Property 0 -300L 300L 200L 200L 1700L 1700L Text Complexity Dial Reader Ability Dial Task Difficulty Dial 72% Comprehension Display

  6. Imagine a world where assessment items are generated and scored in real-time.

  7. Theoretical versus Empirical Text Complexity for 719 Articles* Mean Theoretical = 884.4L (356.2) Mean Empirical = 884.4L (355.0) Reliability = 0.997 SEM = 12.8L r = 0.968 r” = 0.969 R2” = 0.938 RMSE” = 89.6L * Inclusion criteria: 50 encounters and 1,000 items

  8. Artifactual Sources of Variance in Empirical Text complexity Measures • Random measurement error • Sampling error • Range restriction • Systematic error in empirical complexity measures • Wrong function form (not linear) • Variation in empirical text complexities across estimation algorithms We have estimated that the first three of these artifactual sources of variance account for no more than 4% of the total variance in the system – leaving 2% still unexplained. Sources 4-6 may account for this remaining 2%.

  9. Student 1528 7th GradeMaleHispanicPaid Lunch May 2007 – April 2011 347 Encounters138,695 Words3,342 Items983 Minutes Text Demands forCollege and Career 1600 1400 1200 1000 May 2016(12th Grade)

  10. Conclusions • There is only a very small proportion of variation in empirical text complexity left unexplained. • None of the hypotheses about genre (expository vs. narrative), coherence, cohesion, grade dependence, gender dependence, second language dependence have been supported. • It is possible that the small amount of unexplained variance is due to artifacts in the estimation of empirical text complexity.

  11. Contact Info: A. Jackson Stenner Chairman & CEO, MetaMetrics University of North Carolina, Chapel Hill jstenner@Lexile.com

More Related