Smarter Balanced Assessment Basics

1. Smarter Balanced Assessment Basics Vernon G. Gettone, Ph.D. Instruction and Professional Development (IPD) Staff

2. Table Think, Table Talk! At your table, pick a partner and introduce yourself. Tell one another something you already know about Smarter Balanced Assessment. Then share a question you have about the assessment.

3. SBAC Consortium of States *California is a Governing State 24 States representing 39% of K-12 students 22 Governing 1 Advisory 1 Affiliate Washington State is fiscal agent WestEd provides project management services

4. English Language Arts and Mathematics, Grades 3–8 and High School BEGINNING OF YEAR END OF YEAR INTERIM ASSESSMENT INTERIM ASSESSMENT Computer Adaptive Assessment and Performance Tasks • PERFORMANCE • TASKS • Reading • Writing • Math END OF YEAR ADAPTIVE ASSESSMENT Last 12 weeks of year* Digital Library DIGITAL CLEARINGHOUSE of formative tools, processes and exemplars; released items and tasks; model curriculum units; educator training; professional development tools and resources; scorer training modules; and teacher collaboration tools. Computer Adaptive Assessment and Performance Tasks Scope, sequence, number, and timing of interim assessments locally determined Re-take option Optional Interim assessment system— Summative assessment for accountability * Time windows may be adjusted based on results from the research agenda and final implementation decisions. 4 Source: http://www.ets.org

5. Smarter Balanced Digital Library • The formative assessment component of the Smarter Balanced system of assessments • Will contain formative assessment strategies and professional learning and instructional resources for educators • Access will be provided to all California LEAs at no cost. • The first stage of the Digital Library is scheduled to be available in April 2014.

6. Smarter Balanced Digital Library (cont.) • All resources must meet certain quality criteria to be included in the Digital Library: • Incorporates high-quality formative assessment practices • Reflects learner differences and supports personalized learning • Demonstrates utility, engagement, and user-friendliness • Approximately 150 California educators are helping to review and contribute proposed resources as part of the State Network of Educators (SNE).

7. Smarter Balanced Digital Library (cont.) • Will contain Web-based professional learning and instructional modules on topics such as: • the Common Core State Standards • assessment literacy • understanding the Smarter Balanced content specifications • formative assessment process within the context of the Smarter Balanced assessment system

8. Improving Teaching & Learning Summative: College and career readiness assessments for accountability Teachers and schools have information and tools they need to improve teaching and learning All students leave high school college and career ready Common Core State Standards specify K-12 expectations for college and career readiness Formative resources: Supporting classroom-based assessments to improve instruction Interim: Flexible and open assessments, used for actionable feedback 8

10. Bloom’s Taxonomy Updated Creating Evaluating Analyzing 2010 Common Core Standards Applying Understanding 1997 Standards Remembering

11. Bloom’s Taxonomy Updated Creating Generating new ideas, products, or ways of viewing things Designing, constructing, planning, producing, inventing. Evaluating Justifying a decision or course of action Checking, hypothesizing, critiquing, experimenting, judging Analyzing Breaking information into parts to explore understandings and relationships Comparing, organizing, deconstructing, interrogating, finding Applying Using information in another familiar situation Implementing, carrying out, using, executing Understanding Explaining ideas or concepts Interpreting, summarizing, paraphrasing, classifying, explaining Remembering Recalling information Recognizing, listing, describing, retrieving, naming, finding

12. Cognitive Learning • CCSS require high-level cognitive demand • Asking students to demonstrate deeper conceptual understanding through the application of content knowledge and skills to new situations and sustained tasks • Applies Webb’s Depth of Knowledge (DOK) to Bloom’s Cognitive Process Dimensions • Bloom: What type of thinkingis needed to complete a task? • Webb: How deeplydo you have to understand the content to successfully interact with it? How complex or abstract is the content?

14. Webb’s Depth of Knowledge (DOK) Levels • DOK 1: Recall & Reproduction • Recall of a fact, term, principle, concept; perform a routine procedure, locate details • DOK 2: Basic Application of Skills/Concepts • Use of information, two or more steps with decision points along the way, explain relationships • DOK 3: Strategic Thinking • Requires reasoning or developing a plan or sequence of steps, requires decision-making or justification • DOK 4: Extended Thinking • An investigation or application to real world; requires time to research, problem solve, and process multiple conditions; could require synthesis of information across multiple sources and/or disciplines

15. ELA DOK Levels

16. MATH DOK Levels

17. Bloom to Webb’s DOK Levels

18. ELA Claims Overall Claim for Grades 3–8“Students can demonstrate progress toward college and career readiness in English language arts and literacy.” Overall Claim for Grade 11“Students can demonstrate college and career readiness in English language arts and literacy.”

19. Math Claims Overall Claim for Grades 3–8“Students can demonstrate progress toward college and career readiness in mathematics.” Overall Claim for Grade 11“Students can demonstrate college and career readiness in mathematics.”

20. Claims 2, 3, & 4: Relevant Verbs Claim 2 Claim 3 Claim 4

21. Accessibility & Accommodations CDE website with details: www.cde.ca.gov/sp/se/cc/sbacaa.asp 21

22. Accessibility & Accommodations 22

23. Supports for English language learners

24. Assembly Bill 484 • Assembly Bill (AB) 484, chaptered into law October 2, 2013, established the Measurement of Academic Performance and Progress (MAPP). Through regulation, the name has been changed to the California Assessment of Student Performance and Progress (CAASPP). • CAASPP replaces the Standardized Testing and Reporting (STAR) Program. • Among the requirements set forth in AB 484 is the requirement that local educational agencies (LEAs) participate in the Smarter Balanced Field Test.

25. 2014 Smarter Balanced Field Test Windows The Smarter Balanced Field Test will be administered March 18 through June 6, 2014. Each school has been assigned a six-week window within this time frameand may test anytime within that assigned window. On November 20, 2013, Educational Testing Service (ETS) released testing window assignments for schools and notified LEA testing coordinators. Test window assignments are available at http://www.startest.org/FT-windows2014.html.

26. Field Test Details • Students in grades 3–8, grade 11, and a small sample of students in grades 9 and 10 will participate in the field test. • The field test is estimated to take approximately 3.5 hours, although it is untimed. • Students exempt from participation in the field test: • Students who take the California Alternate Performance Assessment (CAPA). • For ELA only, English learners who have attended school in the United States for less than 12 months • Those unable to take test on the computer

27. CDE Field Test Flash • A nine-minute video: Overview of the Smarter Balanced Field Test is now available at http://www.youtube.com/watch?v=IrG3S-TDcJo. Purpose: Provide an overview of the spring 2014 Smarter Balanced Field Test. • Audiences for the video: Teachers, administrators, school board members, and other educators responsible for test administration. • Resources for use with students and parents are referenced in the video.

28. CDE Field Test Flash •  A table that provides information about Smarter Balanced question types is now available at http://www.cde.ca.gov/ta/tg/sa/question-types.asp. • The table lists various question response types (e.g., multiple choice, drag and drop, matching tables) and the technology skill students will use to respond to each question type. • The table provides examples of specific questions on the Smarter Balanced Practice Tests and Training Tests that are associated with each question type. • Teachers can refer to this table to focus on specific question types and technology skills as they use the Practice and Training Tests, available at http://sbac.portal.airast.org/ca/practice-test-ca/, to prepare students for the Field Test.

29. CDE Field Test Flash A table that describes how students will access embedded resources for the Smarter Balanced assessments is now available at http://www.cde.ca.gov/ta/tg/sa/access-embedded.asp. Embedded resources include such functions as strikethrough, highlighter, American Sign Language videos, and text-to-speech. Students will use various types of devices (e.g., desktops, laptops, and tablets) to take the Field Test This table describes how students will use a mouse, keyboard, touch screen, or trackpad to open the context menu on each type of device to access the embedded resources

30. CDE Field Test Flash If you would like to receive Flash updates via e-mail notification, subscribe to the California Assessment of Student Performance and Progress (CAASPP) listserv by sending a blank message to subscribe-caaspp@mlist.cde.ca.gov

31. Performance Assessment <script src='https://www.teachingchannel.org/videos/growth-mindset-eed/embed?format=js' type='text/javascript'></script>

32. Field Test Resources Preparation: • Sample Test Items and Performance Tasks http://www.smarterbalanced.org/sample-items-and-performance-tasks/ • Practice Test http://www.smarterbalanced.org/practice-test/ • New Training Test: https://sbacpt.tds.airast.org/student/

33. Comparison of Training Test and Practice Test

34. Five Item Types • Selected Response • Constructed Response • Extended Response • Performance Tasks • Technology-Enhanced

35. Benefits and Limitationsof Selected Response Items Benefits • Answered quickly • Assess a large range of content on one test • Inexpensive to score • Results collected quickly Limitations • Limited ability to reveal a student’s reasoning process • Difficult to assess higher-order thinking skills

36. Stimulus Text: Birds Make Good Pets There are many reasons why people keep birds as pets. Canaries sing beautiful songs. Parakeets will sit on your shoulder. Parrots can talk to you. Birds fly outdoors. Pet birds can be fun. ELA Grade 3 Selected Response Item Item Stem: A student is revising this paragraph and needs to take out information that does not support why birds make good pets. Which of the following sentences does not support why birds make good pets? Options: A. “Canaries sing beautiful songs.” B. “Parakeets will sit on your shoulder.” C. “Parrots can talk to you.” D. “Birds fly outdoors.” Distractor Analysis: A. Incorrect: This sentence gives a reason why someone might want a bird as a pet. B. Incorrect: This sentence gives a reason why someone might want a bird as a pet. C. Incorrect: This sentence gives a reason why someone might want a bird as a pet. D. Correct: This sentence states a fact about

37. Math Grade 3 Selected Response Item Below are two rectangles that are joined together. 1a. Yes No Yes No Yes No Yes No 1b. For numbers 1a–1d, choose Yes or No to indicate whether joining each rectangle to the existing two rectangles would total exactly 99 square feet. 1c. 1d.

38. Stimulus Text: Read the paragraph and complete the task that follows it. As my family drove home last evening, the sun was going down. We were treated to a beautiful sunset! All around over our heads, the entire sky was pretty. In a few minutes the amazing show was over. The sun disappeared completely, and the brightly colored sky faded to dark gray as the night began. ELA Grade 4 Selected Response Item Item Stem: Revise the paragraph by choosing the phrase with the best descriptive detail to replace was pretty. Options: A. had a whole lot of bright colors mixed together B. shone because it was almost time for darkness C. glowed with astonishing shades of pink and gold D. looked different than it usually does during the day Distractor Analysis: A. This option makes grammatical and semantic sense as replacement for the phrase, but it contains only basic descriptive details. B. This option makes grammatical and semantic sense as replacement for the phrase, but gives a reason for the bright colors and not a precise description thereof. C. KEY: This phrase contains descriptive details about the beauty of the sky with a precise verb, an adjective of degree, and specific colors. D. This option makes grammatical and semantic sense as replacement for the phrase, but provides a comparison to the daytime color and not a precise description.

39. Math Grade 4 Selected Response Item Which number is both a factor of 100 and a multiple of 5? 4 40 50 500 A B C D Key and Distractor Analysis: A Did not consider criteria of “multiple of 5” B Did not consider criteria of “factor of 100” C Correct D Multiplied 100 and 5

40. ELA Grade 5 Selected Response Item Stimulus Text: Read this text and then answer the question. Animals on the Move A good way to learn about animals is to track them from space. Scientists pick individual animals and fit them with lightweight, comfortable radio transmitters. Signals from the transmitters are received by special instruments on certain satellites as they pass overhead. These satellites are operated by the National Oceanic and Atmospheric Administration (NOAA). The polar orbits of the satellites let them see nearly every part of Earth as it rotates below and receive signals from thousands of migrating animals. After the satellite gets the signal from the animal's transmitter, it relays the information to a ground station. The ground station then sends the information to NASA's Goddard Space Flight Center in Maryland. Goddard then sends the information about the animal to the scientists, wherever they may be. Tracking migrating animals using satellites may help us figure out how to make their journeys as safe as possible and help them survive. A flock of geese flies gracefully overhead. You wish you could see the world as they see it. You wish you could fly and be as free as they are. You wonder where they are going in such a hurry! Well, don't envy them too much, because they may be on a very long, tiring journey. Many geese and other birds migrate thousands of miles every year. Some travel over 7,000 miles one way! Some may travel up to 1000 miles without even a rest stop, crossing the Gulf of Mexico or the Sahara Desert. These birds must follow their food supply and they must return to certain locations to breed. They migrate to survive! Besides birds, some other long-distance travelers are fish, sea turtles, bears, caribou, whales, and porpoises. Some of these kinds of animals are shrinking in population. Some are in danger of disappearing forever. Scientists want to know what is happening to them and why. As part of the answer, they want to know where the animals go, how they get there, and how long they stay. Item Stem: Read this sentence in Paragraph 3. Scientists pick individual animals and fit them with lightweight, comfortable radio transmitters. Which set of words has the same meanings as the underlined words? Options: A. select, equip B. claim , connect C. examine, link D. determine , tame

41. Math Grade 5 Selected Response Item Key: 1a. No 1b. Yes 1c. Yes 1d. No

42. ELA Grade 6 Selected Response Item Stimulus Text: Read the following passage and then answer the question. Have you seen pictures of Uncle Sam? He is a skinny man with a long white beard and a top hat. He usually wears the colors of the American flag: red, white, and blue. He has been a symbol of the United States for about two hundred years. Some people think that there was actually a real Uncle Sam, named Samuel Wilson. He lived in Troy, New York, in the early 1800s, he ran a business that delivered meat to the United States Army. The meat was delivered in barrels stamped “U.S.” The “U.S.” stamp was to show that it belonged to the United States government. The people delivering the meat knew that it came from Sam Wilson. They joked to the soldiers that the “U.S.” on the meat barrels stood for “Uncle Sam.” The joke spread. Soon, all over the country, “Uncle Sam” became another way of saying “United States.” Item Stem: Select the correct way to revise the highlighted sentence. Options: A. He lived in Troy, New York, in the early 1800s. There he had ran a business that delivered meat to the United States Army. B. He lived in Troy, New York, in the early 1800s he ran a business that delivers meat to the United States Army. C. He lived in Troy, New York in the early 1800s, and he ran a business that delivered meat to the United States Army. D. In the early 1800s, Sam Wilson lived in Troy, New York, and ran a business that delivered meat to the United States Army.

43. Math Grade 6 Selected Response Item

44. Stimulus Text: Read the passage and complete the task that follows it. Orangutans When they are hungry, they can use sticks to open their favorite fruit. These great apes can make at least 13 different kinds of noises to send messages to other orangutans; they can also “talk” to each other by making signs with their hands. Orangutans can easily learn to use zippers and to open latches. From watching humans, some have even learned to wash clothes and paddle canoes! ELA Grade 7 Selected Response Item Item Stem: This paragraph about orangutans is missing a thesis statement. Select the most appropriate thesis statement to begin the paragraph. Options: A. I am going to tell you about orangutans and their skills. B. Orangutans are very intelligent animals. C. Orangutans are the monkeys most like humans. D. Orangutans like to do many interesting things.

45. Math Grade 7 Selected Response Item

46. Stimulus Text: Below is a text about Native Americans. Read the text and answer the question that follows. Native Americans Archaeologists suggest that people arrived in several groups or tribes to America, from at least 15,000 years ago. The first Americans came from Asia and followed herds of grazing animals across a land bridge formed during the Ice Age. When the Earth began to warm, this land bridge disappeared and became the Bering Strait. The people journeyed on foot slowly southward into North America through a harsh landscape. They were excellent hunters and speared huge animals such as woolly mammoths and long-horned bison. ELA Grade 8 Selected Response Item Item Stem: Which statement from the text shows how Native Americans survived in North America? Options: A. “people arrived in several groups” B. “the Earth began to warm” C. “The people journeyed on foot slowly” D. “They were excellent hunters”

47. Math Grade 8 Selected Response Item

48. Purpose of Constructed Response Items • ConstructedResponse Items • Address assessment targets and claims that are of greater complexity • Require more analytical thinking and reasoning

49. Administration ofConstructed Response Items Administered during the computer-adaptive component Scored using artificial intelligence Most constructed response items take between 1 and 5 minutes to complete Some more complex items may take up to 10 minutes to complete

50. Stimulus Text: Read the paragraph and complete the task that follows it. Children should choose their own bedtime. There are things to do, and most have homework. Some people need more sleep, but children like talking to friends. The time to go to bed should be children’s decision when they are tired they go to bed earlier. There are activities to go to, so children learn to be responsible. ELA Grade 3 Constructed Response Item Item Prompt: Rewrite the paragraph by organizing it correctly and adding ideas that support the opinion that is given. Sample Responses: