Sampling Issues and Accessible Assessments

1. Sampling Issuesand Accessible Assessments Christopher Johnstone National Center on Educational Outcomes University of Minnesota

2. There is a population of students who may otherwise be succeeding in reading activities in schools, but may not be succeeding on large-scale assessments. This population probably includes students with learning disabilities, but may also include non-disabled students We still are developing our definitions of “accessible.” “Big Picture Issues”

3. Students who receive services for specific learning disabilities are a heterogeneous group. Specific reading disabilities may be related to different aspects of the reading process (e.g., decoding, comprehension, etc.) Reading issues may be change with grade level of students. LD-specific issues

4. Teacher questionnaire for grades 4 and 8 regular and special education teachers. To be distributed in regular schools, but over-sampling may occur in schools that have high populations of students with learning disabilities, speech language impairments, hearing impairments, and mental retardation. Survey may be used as a mechanism for recruiting students for future research. Research: Sampling Issues

5. Most items use Likert-scales for teacher ratings. Every item has space for teachers to comment if they wish. Questionnaire begins by asking a teacher to think of a student whose test score is likely to differ from otherwise demonstrated reading ability and to estimate how large that difference might be. Questionnaire Design

6. Items ask if difficulty on assessments may be a result of: Temporary sickness Vision or motor difficulties Perceived Causes of Challenges: Situational Factors

7. Test anxiety Motivation Environmental Distractions Distracters within Test (e.g., extraneous graphics, too many multiple choice options*) *the subject of another PARA study General Testing Problems

8. Lack of understanding about phonemic principles Lack of generalization with phonemic principles Poor sight word vocabulary Poor fluency Word-level Challenges

9. Poor vocabulary Background knowledge deficiencies Lack of knowledge about expository text structure Passage length (passages too long or not broken up into manageable chunks) Lack of personal interest in text Needs to use strategies not allowed on large-scale tests (e.g., highlighting) Comprehension Challenges

10. Independent Reading Persistence Effective use of information Under-AssessedAttributes

11. Pilot survey was reviewed by 21 experts in the fields of reading, assessment, and special education in January, 2006. Survey will be distributed and analyzed in Spring, 2006 if NOT used as a sampling tool for students. Survey may be used as a sampling tool for students for experimental research. If so, survey will be distributed in Fall, 2006. Research Plan

12. Results will be used to inform general principles of accessible assessment (to be developed collaboratively between PARA and DARA). Results will also be used to inform the development of testlets or sample items to be used in experimental research. Research Results