Training for ELPA and MI-Access Scoring and Administration

1. Training for ELPA and MI-Access Scoring and Administration 2008 ASSESSMENT & ACCOUNTABILITY CONFERENCES

2. To help coordinators who train assessment administrators better understand scoring procedures for MI-Access and the ELPA • To provide information on resources available to aid understanding • And, ultimately, to… • increase the reliability and validity of assessment results through better understanding, and • ensure that students’ efforts are accurately scored, reported, and counted PURPOSE

3. MI-Access Assessments • Participation — for students who have, or function as if they have, severe cognitive impairment • Supported Independence — for students who have, or function as if they have, moderate cognitive impairment • Functional Independence — for students who have, or function as if they have, mild cognitive impairment Three Assessments

4. Content Areas Assessed • The MI-Access assessments cover three content areas: • English language arts (ELA), • mathematics, and • science • The assessments reflect Michigan’s GLCEs, HSCEs, and/or Benchmarks, but they have been extended — or reduced in depth, breadth, and complexity — so they are appropriate for the student populations being assessed

5. MI-Access Participation & Supported Independence Assessments • The Participation and Supported Independence (P/SI) assessments use TWO item formats: • Activity-based observation — items are presented to students during familiar classroom activities or routines, which provide a performance context for assessing specific EGLCEs, EHSCEs, and/or EBs • Selected response — students are read an item stem, or question, and asked to select the correct response

6. Activity-Based Observation Item • ACTIVITY: The student will correctly indicate the difference between whole and part when engaged in a familiar food preparation activity, such as assembling ingredients for a snack. For example, the student could be shown a whole cup of milk and part of a cup of milk and then be asked, “Which one is a whole cup?” • SCORING FOCUS: Differentiating between a whole object and part of an object Sample Participation Mathematics Item (Grades 6-8)

7. Selected-Response Item Sample Participation Science Item (Grade 8) • P items have 2 picture answer choices • SI items have 3 picture answer choices

8. Picture Card Presentation Styles • Both picture cards must be presented at the same time in one order, then presented again with the positions reversed • ELA activity-based observation items that use words paired with pictures also have specific presentation styles (see Appendix C in the manual) Participation Selected-Response Item with 2 Answer Choices Picture 1 Picture 2 Picture 2 Picture 1 Show first Reverse and show again

9. P/SI Assessment Administrators • Administered by TWO people—a Primary Assessment Administrator (PAA) and a Shadow Assessment Administrator (SAA) • The PAA and SAA work together prior to the assessment to determine the student’s anticipated response mode. • The PAA and SAA observe the student being assessed and simultaneously and independently score the student using a standardized scoring rubric (regardless of item format)

10. P/SI Scoring Rubrics • The scoring rubrics… • are based on the student responding correctly, and • take into consideration the amount of assistance required to engage the student in the item • The scoring rubrics contain both… • score points, and • condition codes (which add meaning to zeroes)

11. P/SI Scoring Rubrics

12. P/SI Scoring Rubrics Online Learning Program • www.mi-access.info • Explains the P/SI rubrics in detail • Shows AAs using them to score students on sample items • Is being updated (still says “pilot”)

13. MI-Access Coordinator & Assessment Administrator Manual • The manual includes… • an entire section explaining how the P/SI assessments are designed and should be administered, • flow charts showing how to apply the scoring rubrics, and • score point and condition code definitions and examples

14. P/SI Item Analyses • Compares how AAs scored released items • Can be used at district/building level to discuss issues of consistency • Produced only when 10 or more students in the same grade take the same test

15. Functional Independence • For Functional Independence, a scoring rubric is used only for ELA Expressing Ideas • Students are asked to respond to an open-ended prompt: • Tell about what you like to do outside when the weather is cold. Be sure to include details and examples in your response. • Students can write, draw, or use a combination to express their response • Students can also dictate their response if their disability prevents them from writing or drawing

16. ELA Expressing Ideas Rubric • Student responses are evaluated by external hand-scorers using a 4-point rubric that measures • topic focus, • organization, and • use of language and visual conventions • 4 is the highest score a student can receive • The rubric includes condition codes and comment codes

17. ELA Expressing Ideas Rubric • Condition codes add meaning to a score point of zero • A = off topic • B = illegible • C = written in a language other than English • D = blank/refused to respond • Comment codes elaborate on each score point • Lacks clear focus on the prompt topic • Shows limited development with insufficient details and/or examples • Lacks clear organization of ideas and/or arrangement of figures • Contains errors in language/visual conventions that interfere with understanding

18. FI ELA Item Analyses • Shows how many students received each score point or condition code • Shows how many student received each comment code

19. MI-Access P/SI Scoring Rubrics Online Learning Program • www.mi-access.infounder the “Online Learning” tab • 2008/2009 MI-Access Coordinator and Assessment Administrator Manual • www.mi.gov/mi-access under “Resources” • Sample Assessment Booklets and Released Item Booklets • www.mi-access.infounder the “Functional Independence” and “Participation and Supported Independence” tabs RESOURCES

20. Extended GLCEs, HSCEs, and Benchmarks • www.mi.gov/mi-access under the “Participation and Supported Independence” and “Functional Independence” sections • Assessment Plans • www.mi.gov/mi-access under the “Participation and Supported Independence,” “Functional Independence,” and “IEP Team Information” sections • IEP Team Tools • www.mi.gov/mi-access under the IEP Team Information” section RESOURCES

21. MI-Access Linda Headley lheadley@cablespeed.com (517)339-3940 Deb Rakas debrakas@comcast.net (517)699-3017 CONTACTS

22. ELPA: Speaking Scoring • Item formats at all levels: • Sentence Repeat • Short Conversation • Guided Discussion • Storytelling • Story Retell

23. ELPA: Speaking Scoring Sentence Repeat • Student hears and reads sentence before repeating it • One point maximum score • Standard focus is on S.6Demonstrate comprehensible pronunciation and intonation for clarity in oral communication

24. ELPA: Speaking Scoring Sentence Repeat • Proctor listens for fluency and smoothness of speech, proper pronunciation, and appropriate intonation

25. ELPA: Speaking Scoring Short Conversation • Student hears and reads prompt and responds to question(s) • Questions are as open-ended as possible, but still guide student toward focused answer

26. ELPA: Speaking Scoring Short Conversation • Two points maximum score • Standard focus is on S.2Engage in conversations for personal expression and enjoyment

27. ELPA: Speaking Scoring Short Conversation • Proctor listens for fluency and accuracy, but with an emphasis on the student’s ability to use appropriate vocabulary to express personal opinions, ideas, or points of view

28. ELPA: Speaking Scoring Guided Discussion • Item format is a two-part set of questions with short prompt to lead into discussion • Student hears and reads prompt and responds to first question

29. ELPA: Speaking Scoring Guided Discussion • Student then hears and reads follow-up question and responds • Two points maximum score for each prompt; total four points maximum • Proctor scores each portion individually

30. ELPA: Speaking Scoring Guided Discussion • Standard focus is on S.4Use English to interact in the classroom • Benchmark S.4.2.fParticipate in guided discussions

31. ELPA: Speaking Scoring Guided Discussion • Proctor listens for conversational flow and exchange of information; also for the student’s ability to focus the discussion on specific details or examples after the second prompt

32. ELPA: Speaking Scoring Story Telling • Student sees sequence of three graphics and creates oral story or narrative about pictures • Sequence of graphics is marked First, Next, and Last

33. ELPA: Speaking Scoring Story Telling • Four points maximum score • Standard focus is on S.7Present information, concepts, and ideas to an audience of listeners on a variety of topics

34. ELPA: Speaking Scoring Story Telling • Proctor listens for complex sentence structures, varied and precise vocabulary, and logical presentation of ideas with appropriate transitions

35. ELPA: Speaking Scoring Story Retell • Student hears and reads narrative or content-related text • Student responds by telling back to the proctor as much as possible, with allowance for added details

36. ELPA: Speaking Scoring Story Retell • Student bases response on comprehensible English input appropriate to grade level • Four points maximum score

37. ELPA: Speaking Scoring Story Retell • Standard focus is on S.7Present information, concepts, and ideas to an audience of listeners on a variety of topics

38. ELPA: Speaking Scoring Story Retell • Proctor listens for authentic speech patterns of student, including use of grammatical constructions, descriptive vocabulary, and transitional phrases to retell recognizable story with fluency of speech

39. ELPA: Speaking Scoring Story Retell • Speaking DVD available for training purposes • Contact OEAA for copies of DVD

40. ELPA Phil Chase Department Specialist, Assessment of English Language Learners chasep2@michigan.gov (517) 335-3967 CONTACTS