How can we support students but not interfere with the construct ?

1. Raggedy Ann or Student Performance: Construct Relevance in Alternate Achievement StandardsAssessments

2. How can we support students but not interfere with the construct ? • Does the assessment truly measure that which it intended to measure? • Content Standards in • Reading • Math • Science • How can we support students maximizing the assessment population but not interfere with the construct?

3. Where it gets messy….. • AA-AAAS necessarily require the teacher to • Directly observe student performance or recall observation using ongoing (formative assessment) techniques • Performance tasks • Level of prompt required to make a response • Select the correct response from an array of responses • Portfolios • Compile on-going data collection sheets based either based on non-instructional probes or on level of prompt required to make a response or a combination of both • EVEN Construct items that measure the intended construct…

4. Historically, in Effective Practices in Special Education • The purpose of assessment was • Formative (instructional improvement) • Instruction and assessment in severe disabilities which follows • Techniques from Applied Behavior Analysis • Relies on heavily on • Observational techniques • Task analysis and chains of behaviors • Collection of performance data during instruction • Identifying “controlling prompts” which ensure the intended behavior will occur under the specified stimulus conditions

5. Ongoing Instruction or formative assessments • Must correspond to the skills being taught and the unique characteristics of the learner • (Deno & Mirikin, 1977) • Directly observe student performance using ongoing continuous direct assessment techniques • Non instructional “probes/ tests” • Systematic instructional procedures that record the level of assistance required for the student to make a response • Or a combination of these approaches (Wrestling & Fox, 2004; Alberto & Troutman, 2002; Browder, 1991; Brown & Snell, 1993; Wolery et. al, Sailor & Guess, 1983; White & Haring, 1980)

6. Prompt Definition • Prompt – teacher behaviors that increase the probability of a correct response – used in the acquisition of behaviors/targets and faded as learning improves. • ( Wolery et al. 1992; Wrestling & Fox, 2004)

7. Support Definition • Support – modifications in the presentation of an instructional target, materials, response modes • referring to instruction and levels of participation in general curriculum, inclusive education. ( Weymer, Lattin, & Agran 1992)

8. Probe Definition • Probe – an opportunity for the student to perform the skill given only the target stimulus with NO OTHER input from the teacher. • (Wolery, et al. 1992; Wrestling & Fox, 2004)

9. Scaffold Definition • Instructional scaffoldingis the provision of sufficient support to promote learning when concepts and skills are being first introduced to students. These supports may include the following: • Resources • A compelling task • Templates and guides • These supports are gradually removed as students develop autonomous learning strategies, thus promoting their own cognitive, affective and psychomotor learning skills and knowledge . • The teacher’s level and type of support change over time from directive, to suggestion, to encouragement, to observation. • (Bruner; Vytgosky; Cazden & others)

10. The Question of the Day! How do we “control” for irrelevant variance in the design of assessment of students with significant cognitive disabilities who need supports in order to provide an independent, discriminatory response?

12. Prompt Confusion • The task direction is not a prompt –”Find one more” is NOT a verbal prompt; it’s the cue to respond. Another example, “Read the word” • Support to get student ready to respond is not a prompt – Allowing student to select one answer gets Kyle ready to respond. Another example- placing students hands above all answers so can slide hand to correct card is NOT a physical prompt. Wakeman adapted from Browder (2008) Prompting: Impact on Inferences about Student Learning

14. Student Achievement vs. Something else • Student achievement • – Select picture for main idea • Full credit‐ eye gaze, point • Find main idea across stories • Solve problems- eye gaze to correct answers for TA • “Something Else” • Select picture with model prompt- point where I point • Student works with a peer to select picture • Student did not select picture but could select not my best work Wakeman adapted from Browder (2008)

15. Worksheet • What is the targeted skill? • Is it to trace numbers or add or subtract decimals? • Could make very little inference about student learning for tracing numbers given. • Better way to assess AND support learning using a scribe? • One idea: Steps in a TA to solve problem and have student eye gaze at or select from written choices correct answer for each step. Teacher or student could place correct answers in the equation. Wakeman

18. Teaching vs. Testing • This teacher may have previously assessed the student without prompts on this task and is now teaching the student. • Assessment= unprompted independent responses • Teaching= prompts (changes in definitions of terms to support student understanding; review of individual items; provides correct answers when student is wrong) Wakeman

19. Scaffolding • Interesting/confusing when considering scaffolds in assessment- does the scaffolded support change the construct? • Behaviorist would consider a verbal prompt hierarchy • Where do we get apples? • – Let’s read the paragraph again (rereads paragraph) • – Let me read some key sentences (rereads sentences • with correct answer) • – Listen to this sentence‐ “The apples came from an orchard of well established trees.” Wakeman adapted from Browder (2008)

22. Consideration for Partial Credit? • When reducing the number of choices, it can become an issue of independent response vs. an accurate response. • What is the priority within the state scoring system? • Inferences about student learning in assessment- with only two options, guessing plays a significant role. How does that play out in scoring of the assessment? • Good teaching practice regarding distractors (relevancy – “What did they ride in the story- a bus or a goldfish?” vs. “a bus or a car?”) may be based upon student need. Wakeman

24. Response Prompts • Excellent example of Least to Most prompts • Valuable in teaching to support students • Could demonstrate learning for some students (growth) in an assessment if they demonstrate a change over time in the level of prompt needed across different content • (not master Romeo and Juliet and then right back to needing physical prompts for Macbeth but show change over time and content needing fewer physical prompts for each story) Wakeman

26. Lack of Appropriate Response Options or Support = Bias in the Assessment • When an item does not include the appropriate support (picture symbols) as this one does not, the assessment is then biased against the student (no way to participate in the item). • By defining an active response the student CAN make; minimize the barrier of response mode Wakeman adapted from Browder (2008)

28. Construct Relevance Using the AA-AAAS Test Administration Manual to Communicate the Appropriate Use of Prompts

29. Test Administration Handbook • Definitions of Prompts and Prompt Hierarchies • Use of prompts in task/item/artifact development • Recording use of prompts • Scoring rubrics and prompts • Instructional use of prompts

30. Different prompt hierarchies • Verbal-Gesture-model-partial physical-full physical • Gesture-verbal-visual-partial physical-full physical • Auditory-visual-physical • Verbal-gesture-physical

31. Different definitions for prompts • Verbal prompt – This level of prompt requires the teacher to give a specific verbal direction in addition to the task direction. (indirect/direct) • Verbal: Auditory Statement (e.g., more than repeat prompt) • Auditory - Any sound or verbal direction given to help the student initiate or participate in the completion of the skill in an activity.

32. Different Definitions of Prompts • Visual - Any picture, gesture, or modeling given to help the student initiate or participate in the completion of the skill in an activity. • Gesture prompt – This level of prompt requires the teacher to move his/her finger, hand, arm, or make a facial expression that communicates to the student specific information (e.g., teacher taps scanner switch button). • Model prompt – This level of prompt requires the teacher to demonstrate the correct response for the student, and the student imitates the teacher’s model (e.g., the teacher demonstrates how to push the switch and then asks the student to repeat).

33. Code of Ethics IT IS A BREACH OF PROFESSIONAL ETHICS FOR SCHOOL PERSONNEL TO PROVIDE VERBAL AND NON-VERBAL CLUES OF ANSWERS, TEACH ITEMS ON THE TEST, SHARE WRITING PROMPTS, COACH, HINT, OR IN ANY WAY INFLUENCE A STUDENT’S PERFORMANCE DURING THE TESTING SITUATION. A BREACH OF ETHICS MAY RESULT IN INVALIDATION OF TEST RESULTS AND LEA OR MSDE DISCIPLINARY ACTION.

34. Prompts in Task/Item/Artifact Development Various levels of direction • Data Charts. The template for the Full Physical Prompt data chart includes an area to list the various assistive technology devices a student may use to explore the full range of assistive technologies to support demonstration of skills. • Prompts may not be used in the baseline • Use of Prompts No prompts are allowed. Be careful not to use physical or verbal prompts to encourage student to make a choice (for example, gesturing, looking, pointing)

35. Recording of Prompt Use During Assessment • Record on data charts • Report on student work • State after videotaping or audio taping of student performance

36. Prompts: Scoring Rubric Prompts That Prevent Grading for Accuracy The teacher must grade the item as incorrect; the prompt has made it impossible todetermine student performance. Full Physical Prompt Model of Exactly the Same Task Telling Student the Answer

37. Scoring Level ofIndependence • The student seldom requires cues or prompts when demonstrating skills based on the documented AGLIs. (80-100%Independence) • The student requires limited cues or prompts to demonstrate skills based on the documented AGLIs.(60-79%Independence) • The student requires extensive cues or prompts to demonstrate skills based on the documented AGLIs. (30-59% Independence)

38. Scoring • The number of steps not requiring prompts per number of steps or items in the assessment task determines the percentage of independence. • 80% or greater is attained, and the prompt level and number do not exceed prompt level and number of prompts indicated in the mastery objective.

39. I have noticed some significant changes with our students (who represent those with the greatest challenges of all who are tested). One struggle in working with students with SCD is that when they develop skills, these are so often “prompt dependent” skills. Students display these behaviors only once others in their environment elicit them. Despite professional inservice training to minimize over prompting, staff often seems to persist with giving many prompts. This is one of the areas I have worked on in training but the testing process seems to have made the most significant difference in helping staff be more mindful of prompts they are giving as well as providing more wait time for students to respond without prompts. This certainly moves them toward greater independence and that is a wonderful and positive thing! — Submitted by Roslyn Canosa, Principal; Baltimore City Public Schools

40. Advice for Minimizing Irrelevant Variance in AA-AAAS • Use procedures that support independent responses. • If prompted responses are accepted, score appropriately. • Provide structure (e.g. scripts, protocols) for the administration of “items”. • Specify appropriate and inappropriate “supports”, “scaffolds”, “prompts” in the administration manual. • Provide items or provide structure/ protocol for item development specifying the construct. • Caution is warranted when using systematic least to most prompting systems as they may • Increase the number of item presentations for students who may require higher levels of prompts • Prompts are aimed at producing a correct response