SMARTER Balanced Assessment

1. SMARTER Balanced Assessment Overview and Sample Items Huron ISD, September, 2013

2. SMARTER Balanced Assessment (SBAC) Beginning 2014-2015 http://www.smarterbalanced.org/

3. Pascal (Pat) D. Forgione, Jr., Ph.D. Center for K-12 Assessment and Performance Management at ETS A Step Change in K–12 Testing, August 2013

4. SBAC Summative Assessment Performance Tasks • One ELA and one math task per year • Integrate knowledge and skills • Computer-delivered • Scored within two weeks Computer-Adaptive • Given during the final weeks of the school year • Multiple item types scored by computer • Re-take option locally determined Pascal (Pat) D. Forgione, Jr., Ph.D. Center for K-12 Assessment and Performance Management at ETS

5. Pascal (Pat) D. Forgione, Jr., Ph.D. Center for K-12 Assessment and Performance Management at ETS

6. Pascal (Pat) D. Forgione, Jr., Ph.D. Center for K-12 Assessment and Performance Management at ETS

7. Pascal (Pat) D. Forgione, Jr., Ph.D. Center for K-12 Assessment and Performance Management at ETS

8. Pascal (Pat) D. Forgione, Jr., Ph.D. Center for K-12 Assessment and Performance Management at ETS

9. Pascal (Pat) D. Forgione, Jr., Ph.D. Center for K-12 Assessment and Performance Management at ETS

10. Expected Test Construction www.sheeo.org, State Higher Education Executive Officers Susan Gendron

11. Michigan’s online initiatives • Pilot in 2006 • Pilot in 2011 (English Language Proficiency) • Pilot in 2012 ( Dynamic Learning Maps Alternate Assessment Consortium for 1% of population) • Pilots leading up to operational adoption of SMARTER/Balanced Assessment Consortium products in 2014/15 “All challenges will be resolved by 2014-15” ~MDE

12. Smarter Balanced Item and Performance Task Development Huron Intermediate School District

13. Smarter Balanced and Evidence-Centered Design Items and Performance Tasks Smarter Balanced Item and Task Specifications Smarter Balanced Content Specifications Common Core State Standards

14. SBAC Assessment Item Types • Selected Response • Constructed Response • Technology-Enabled • Technology-Enhanced • Performance Tasks

15. Key Concepts • Evidence • Universal Design • Accessibility • Sensitivity • Bias

16. Key Concepts • Evidence • Information that students provide through their responses about their knowledge, skills, and abilities • Universal Design • Designing items and tasks so that they function as intended for as many students as possible • Accessibility • Additional information or presenting items and tasks in a different way in order to meet the specific needs of some students • Sensitivity • Content contained in an item that may be distracting or upsetting for some students • Bias • Use of names, topics, or contexts that may be unfamiliar to a sub-group of students

17. Key Concepts • Evidence • Information that students provide through their responses about their knowledge, skills, and abilities • Universal Design • Designing items and tasks so that they function as intended for as many students as possible • Accessibility • Additional information or presenting items and tasks in a different way in order to meet the specific needs of some students • Sensitivity • Content contained in an item that may be distracting or upsetting for some students • Bias • Use of names, topics, or contexts that may be unfamiliar to a sub-group of students

18. Key Concepts • Evidence • Information that students provide through their responses about their knowledge, skills, and abilities • Universal Design • Designing items and tasks so that they function as intended for as many students as possible • Accessibility • Additional information or presenting items and tasks in a different way in order to meet the specific needs of some students • Sensitivity • Content contained in an item that may be distracting or upsetting for some students • Bias • Use of names, topics, or contexts that may be unfamiliar to a sub-group of students

19. Key Concepts • Evidence • Information that students provide through their responses about their knowledge, skills, and abilities • Universal Design • Designing items and tasks so that they function as intended for as many students as possible • Accessibility • Additional information or presenting items and tasks in a different way in order to meet the specific needs of some students • Sensitivity • Content contained in an item that may be distracting or upsetting for some students • Bias • Use of names, topics, or contexts that may be unfamiliar to a sub-group of students

20. Key Concepts • Evidence • Information that students provide through their responses about their knowledge, skills, and abilities • Universal Design • Designing items and tasks so that they function as intended for as many students as possible • Accessibility • Additional information or presenting items and tasks in a different way in order to meet the specific needs of some students • Sensitivity • Content contained in an item that may be distracting or upsetting for some students • Bias • Use of names, topics, or contexts that may be unfamiliar to a sub-group of students

21. General Guidelines for Developing Mathematics and ELA Selected Response and Constructed Response Items • Aligned to the CCSS, some items may include concepts detailed in the Standards for lower grades • Each item should be written to assess a primary claim • Secondary content claims are also possible

22. General Guidelines for Developing ELA and Math Selected Response and Constructed Response Items • Clearly stated to ensure that students understand the task • Clearly elicit the desired evidence of a student’s knowledge, skills and ability • Appropriate grade-level difficulty, cognitive complexity, and reading level • Depth of Knowledge considered

23. Essential Requirements of ELA and Mathematics Selected Response and Constructed Response Items • Some constructed responses require students to support their reasoning • Plausible distractors represent common mistakes • ELA - appropriate content and contexts • Mathematics – must be accurate and more complex items may include scaffolding • For Mathematics Grades 3–5, items do not require a calculator

24. SBAC Assessment Item Types • Selected Response • Extended Constructed Response • Technology-Enabled • Technology-Enhanced • Performance Tasks

25. Selected ResponseSingle Response – Multiple Choice Many experts will tell you that television is bad for you. Yet this is an exaggeration. Many television programs today are specifically geared towards improving physical fitness, making people smarter, or teaching them important things about the world. The days of limited programming with little interaction are gone. Public television and other stations have shows about science, history, and technical topics. Which sentence should be added to the paragraph to state the author’s main claim? A. Watching television makes a person healthy. B. Watching television can be a sign of intelligence. C. Television can be a positive influence on people. D. Television has more varied programs than ever before.

26. Selected ResponseMultiple Correct Options Which of the following statements is a property of a rectangle? Select all that apply. ☐ Contains three sides ☐ Contains four sides ☐ Contains eight sides ☐ Contains two sets of parallel lines ☐ Contains at least one interior angle that is acute ☐ Contains at least one interior angle that is obtuse ☐ All interior angles are right angles ☐ All sides have the same length ☐ All sides are of different length

27. Benefits and Limitationsof Selected Response Items • Benefits • Answered quickly • Assess a broad range of content in one test • Inexpensive and objectively scored • Results collected quickly • Limitations • Limited ability to reveal a student’s reasoning process • Difficult to assess higher-order thinking skills

28. Formats and Components ofMathematics Selected Response Items • Traditional Selected Response Item • Key and Distractor Analysis Which number is both a factor of 100 and a multiple of 5? A. 4 B. 40 C. 50 D. 500 Did not consider criteria of “multiple of 5” Did not consider criteria of “factor of 100” Correct Multiplied 100 and 5 Which number is both a factor of 100 and a multiple of 5? RATIONALE A. 4 B. 40 C. 50 D. 500 STEM Statement ofthe question KEY DISTRACTOR OPTIONS: Possible answers the students must select from

29. Non-Traditional Math Selected Response Item STIMULUS STEM A multiplication problem is shown below. 17 × 12 Which model(s) below could represent the solution to this problem? Select all that apply. 17 × 12 Which model(s) below could represent the solution to this problem? Select all that apply. A. B. C. (1×1)+(1×7)+(1×2)+(2×7) D. E. F. (17×2)+(17×1)

30. Non-Traditional Math Selected Response Item Key and DistractorAnalysis: Does not understand how to modelmultiplication of two two-digitnumbers using area models. Correct Did not account for the valuesof the digits in the tens places. Correct Did not understand that the 1 represents 10 in the multiplication problem Showed multiplication of 17 and (1 + 2) instead of 17 and 12 Responses to this item will receive 0–2 points, based on the following: 2 points: B, D 1 point: Either B or D 0 points: Any other combination of selections.

31. Non-Traditional Math Selected Response Item For numbers 1a – 1d, state whether or not each figure has ⅖ of its whole shaded. STEM 1a. 1b. MULTIPLE PARTS OPTIONS 1c. 1d.

32. Non-Traditional Math Selected Response Item Scoring Rubric: Responses to this item will receive 0–2 points, based upon the following: • 2 points: YNYN The student has a solid understanding of ⅖ as well as the equivalent form of ⅖. • 1 point: YNNN, YYNN, YYYN The student has only a basic understanding of ⅖. Either the student doesn‘t recognize an equivalent fraction for ⅖ or doesn‘t understand that all 5 parts must be equal-sized in figure 1b. • 0 points: YYYY, YNNY, NNNN, NNYY, NYYN, NYNN, NYYY, NYNN, NNNN, NYNY, NNYN, NNNY. The student demonstrates inconsistent understanding of ⅖ or answers “Y” to figure 1d, clearly showing a misunderstanding of what ⅖ means. Figure 1d is considered a “disqualifier “and an answer of “Y” to this part of the item would cancel out any other correct responses as “guesses” on the part of the student.

33. Non-Traditional Math Selected Response Item Scoring Rule: Students who properly match the four shapes to their name will receive two points. Students who make two or three correct matches will receive partial credit of one point All other connections will receive a score of 0. Match each shape below to its name.

34. Examples of Poorly Written Items The table below shows the weights of three vehicles. Which list shows the vehicles in order from lightest to heaviest? ☐ car, motorcycle, truck ☐ motorcycle, car, truck ☐ truck, car, motorcycle ☐ truck, motorcycle, car The table below shows the number of apples three students picked. Which list shows the number of apples picked in order from greatest to the least? ☐ 95, 107, 121 ☐ 95, 121, 107 ☐ 121, 107, 95 ☐ 121, 95, 107

35. Examples of Poorly Written Items Look at the rectangle below. What is the area, in square feet, of the rectangle? ☐ 3 ☐15 ☐18 ☐63 3 feet 6 feet

36. SBAC Assessment Item Types • Selected Response • Constructed Response • Technology-Enabled • Technology-Enhanced • Performance Tasks

37. Constructed Response The table below shows the number of students in each third-grade class at Lincoln School. There are 105 fourth-grade students at Lincoln School. How many more fourth-grade students than third-grade students are at Lincoln School? Show or explain how you found your answer.

38. Constructed ResponseExtended Response Pen 1: Length: (feet, square feet) Width: (feet, square feet) Area: (feet, square feet) Pen 2: Length: (feet, square feet) Width: (feet, square feet) Area: (feet, square feet) • Ms. McCrary wants to make a rabbit pen in a section of her lawn. Her plan for the rabbit pen includes the following: • It will be in the shape of a rectangle. • It will take 24 feet of fence material to make. • Each side will be longer than 1 foot. • The length and width will measure whole feet. Pen 3: Length: (feet, square feet) Width: (feet, square feet) Area: (feet, square feet) Part A Draw 3 different rectangles that can each represent Ms. McCrary’s rabbit pen. Be sure to use all 24 feet of fence material for each pen. Use the grid below. Click the places where you want the corners of your rectangle to be. Draw one rectangle at a time. If you make a mistake, click on your rectangle to delete it. Continue as many times as necessary. • Part B • Ms. McCrary wants her rabbit to have more than 60 square feet of ground area inside the pen. She finds that if she uses the side of her house as one of the sides of the rabbit pen, she can make the rabbit pen larger. • Draw another rectangular rabbit pen. • Use all 24 feet of fencing for 3 sides of the pen. • Use one side of the house for the other side of the pen. • Make sure the ground area inside the pen is greater than 60 square feet. • Use the grid below. Click the places where you want the corners of your rectangle to be. If you make a mistake, click on your rectangle to delete it. Use your keyboard to type the length and width of each rabbit pen you draw. Then type the area of each rabbit pen. Be sure to select the correct unit for each answer. [Students will input length, width, and area for each rabbit pen. Students will choose unit from drop down menu.] Use your keyboard to type the length and width of each rabbit pen you draw. Then type the area of each rabbit pen. Be sure to select the correct unit for each answer. Length: (feet, square feet) Width: (feet, square feet) Area: (feet, square feet)

39. Purpose of Constructed Response Items • Constructed Response Items • Address assessment targets and claims that are of greater complexity • Require more analytical thinking and reasoning

40. Administration ofConstructed Response Items • Administered during the computer-adaptive component • Scored using artificial intelligence • Most constructed response items take between 1 and 5 minutes to complete • Some more complex items may take up to 10 minutes to complete

41. Qualities of a Rubric • Focus on the essence of the primary claim and sometimes secondary claim • Address the requirements of the specific assessment targets • Distinguish between different levels of understanding and/or performance • Contain relevant information, details, and numbers that support different levels of competency related to the item or task

42. Components of a ELA Constructed Response Item The Shepherd’s Boy and the Wolf A Shepherd's Boy was tending his flock near a village, and thought it would be great fun to trick the villagers by pretending that a Wolf was attacking the sheep: so he shouted out, "Wolf! Wolf!" and when the people came running up he laughed at them because they believed him. He did this more than once, and every time the villagers found they had been tricked, for there was no Wolf at all. At last a Wolf really did come, and the Boy cried, "Wolf! Wolf!" as loud as he could: but the people were so used to hearing him call that they took no notice of his cries for help. And so no one came to help the boy, and the Wolf attacked the sheep. STIMULUS The Shepherd’s Boy and the Wolf A Shepherd's Boy was tending his flock near a village, and thought it would be great fun to trick the villagers by pretending that a Wolf was attacking the sheep: so he shouted out, "Wolf! Wolf!" and when the people came running up he laughed at them because they believed him. He did this more than once, and every time the villagers found they had been tricked, for there was no Wolf at all. At last a Wolf really did come, and the Boy cried, "Wolf! Wolf!" as loud as he could: but the people were so used to hearing him call that they took no notice of his cries for help. And so no one came to help the boy, and the Wolf attacked the sheep. In a few sentences, explain what lesson the reader can learn from the shepherd’s boy. Use details from the story to support your response. STEM In a few sentences, explain what lesson the reader can learn from the shepherd’s boy. Use details from the story to support your response.

43. Components of ELA Constructed Response Item Scoring Notes Response may include but is not limited to: The shepherd’s boy learned that he shouldn’t call wolf unless there is really a wolf. The shepherd’s boy learned that he should only ask for help if he needs it or else he wouldn’t get help when he really needs it. “The people were so used to hearing him call that they took no notice of his cries.” The shepherd’s boy learned not to have fun by tricking people because the people learn not to trust you. Score Point 2 Sample: The lesson learned from this story is do not cry for help when nothing is wrong. The shepherd’s boy pretends that a big wolf is attacking his sheep and yells, “Wolf! Wolf!” The people in the village run out to help him because they believe he needs help. After he tricks the villagers more than once, they realize he is just pretending. Score Point 1 Sample: The lesson learned from this story is do not cry for help when nothing is wrong. The shepherd’s boy cries wolf when there is no wolf and the people come to help him. Score Point 0 Sample: Readers learn a good lesson about how to cry wolf.

44. Components of a MathematicsConstructed Response Item A teacher asked her students to use estimation to decide if the sum of the problem below is closer to 4,000 or 5,000. 496 + 1,404 + 2,605 + 489 = One student replied that she thinks the sum is closer to 4,000. She used the estimation shown below to support her reasoning. Is the student’s reasoning correct? In the space below, use numbers and words to explain why or why not. If the student’s reasoning is not correct, explain how she should have estimated. STIMULI STEM

45. Components of a MathematicsConstructed Response Item Sample Top-Score Response: The student’s reasoning is incorrect. She was rounding to the thousands place. She had 2 numbers that were less than 500, and she decided to round these numbers to 0. This is like saying these numbers were not in the problem at all. She needs to account for these two numbers. Together, they have a sum that is very close to 1,000. I think adding 1,000 + 1,000 + 3,000 is a better strategy. This means the sum is closer to 5,000 than to 4,000. Scoring Rubric: Responses to this item will receive 0–2 points, based on the following: 2 points: Student has thorough understanding of how to estimate and how improper estimation can lead to flawed reasoning. Student states that the student in the scenario used reasoning that is incorrect and provides reasoning that shows a better estimation strategy. 1 point: Student has partial understanding of how to estimate and how improper estimation can lead to flawed reasoning. Student states that the student in the scenario used reasoning that is incorrect, but alternate estimation strategy is also flawed. 0 points: Student has little or no understanding of how to estimate and how improper estimation can lead to flawed reasoning. Student states that the student in the scenario used reasoning that is correct. SCORING RUBRIC TOP-SCORE

46. Examples of Poorly Written Items Mercedes received 32 pieces of candy on Halloween. She ate ¼ of the candy. How many pieces of candy did Mercedes have left? Show or explain how you found your answer.

47. SBAC Assessment Item Types • Selected Response • Constructed Response • Technology-Enabled • Technology-Enhanced • Performance Tasks

48. Technology-Enabled Selected or Constructed Responses that include Multimedia Brianna is running for class president. She needs to give a speech to the 4th grade class. Listen to the draft of her speech and then answer the questions that follow. (Test-takers listen to an audio version of the following speech.) “Hi, My name is Brianna. I am running for class president, and I hope you will vote for me. You know many of my friends said they would. I am involved in many activities, including track and theater. If I am elected, I will hold several fundraisers so that all students in the 4th grade can go on a trip at the end of the year. Also, we can donate a portion of the money to a charity of our choice. If you want a class president who will work hard for you and listen to your needs, please vote for me next week!” This speech needs to be revised before the student presents it. Which sentence should be omitted to improve the speech. A. I am running for class president, and I hope you will vote for me. B. You know many of my friends said they would. C. If I am elected, I will hold several fundraisers so that all students in the 4th grade can go on a trip at the end of the year. D. If you want a class president who will work hard for you and listen to your needs, please vote for me next week!”

49. Example of Technology-Enabled Item Gregory is installing tile on a rectangular floor. • He is using congruent square tiles that each have a side length of ½ foot • The area of the floor is 22 square feet. • The width of the floor is 4 feet. Use the grid and the tile below to model the floor. What is the length, in feet, of the floor?

50. Technology-Enabled Items • Digital Media • Video • Animation • Sound • Interactive Tools • Response Types • Selected Response • Constructed Response Example: Listen to President Kennedy’s 1961 inaugural address and then write an essay analyzing metaphors used regarding foreign policy. Example: View video and write a summary explaining steps in a process. Example: Tangrams, Calculator