Writing Constructed Response Assessment Items:

Writing Constructed Response Assessment Items: One Element of a Comprehensive Assessment System

What are Constructed Response Items? • Open-ended Questions • Require several sentences or brief paragraph • Require higher level thinking (than simple recall) and the application of students’ knowledge • Making Comparisons • Identifying Patterns • Evaluating Points of View • Making Generalizations • Synthesizing Information • Allow for the examination of Student Thinking • Scored using a Rubric that provides varying degrees of Credit Wisconsin Dept. of Public Instruction, May 2007

Why Use Constructed Response Items? “The notion that learning comes about by accretion of little bits is outmoded learning theory. Current models of learning … contend that learners gain understanding when they construct their own knowledge and develop … interconnections among facts and concepts …” (Shepard, 1989. quoted in Michigan Curriculum Framework)

Constructed Response as part of a Coordinated Assessment System: “An assessment system is an ongoing coordinated process of collecting information for the purpose of continuously improving student learning.” Michigan Curriculum Framework

Elements of a Coordinated Assessment System: Fixed Response: Simple items that assess factual information / discrete skills. Typically there is a single best answer. Constructed Response: Short answers, explanations, essays or diagrams that involve analysis or evaluation. Require judgment-based scoring (i.e. rubrics). Oral Questioning in Class: Could be fixed or constructed response. Teacher Observation: Typically involve a simple checklist. Performance: Speeches, experiments, debates, etc. Require judgment-based scoring. Project: Designing and/or building useful things. May be done individually or collectively. Portfolio: A collection of student work over time – written work, artistic creations, project reports, etc. Standardized Monitoring: e.g. MEAP/MME, ACT, SAT, NWEA, etc. Adapted from Michigan Department of Education, Science Education Guidebook; and Wiggins, G. & McTighe, J (1998). Understanding by Design. Association for Supervision and Curriculum Development (ASCD).

WHEN to use Constructed Response: Consider the VERB of the Content Expectation: • B2.5i Relate cell parts/organelles to their function. • P2.1A Calculate the average speed of an object using the change of position and elapsed time. • C5.5A Predict if the bonding between two atoms of different elements will be primarily ionic or covalent. • E5.3e Determine the approximate age of a sample, when given the half-life of a radioactive substance along with the ratio of daughter to parent substances present in the sample. Can be assessed with FIXED RESPONSE Items

WHEN to use Constructed Response: Consider the VERB of the Content Expectation: • B1.1A Generate new questions that can be investigated in the laboratory or field. • P1.1g Use empirical evidence to explain and critique the reasoning used to draw a scientific conclusion or explanation. • C5.8B Draw isomers for simple hydrocarbons. • E5.4C Analyze the empirical relationship between the emissions of carbon dioxide, atmospheric carbon dioxide levels and the average global temperature over the past 150 years. Best assessed with CONSTRUCTED RESPONSE Items

WHEN to use Constructed Response: Consider the VERB of the Content Expectation: • B1.1C Conduct scientific investigations using appropriate tools and techniques. • P1.2C Develop an understanding of a scientific concept by accessing information from multiple sources. Evaluate the scientific accuracy and significance of the information. Best assessed with a PERFORMANCE exercise • E1.2D Evaluate scientific explanations in a peer review process or discussion format. Best assessed through TEACHER OBSERVATION

HOW to use Constructed Response Items: Standards for Assessment Tasks: 1. Organization of Information —The task asks students to organize, synthesize, interpret, explain, or evaluate complex information in addressing a concept, problem or issue. 2. Consideration of Alternatives —The task asks students to consider alternative solutions, strategies, perspectives, or points of view in addressing a concept, problem or issue. 3. Disciplinary Content —The task asks students to show understanding and/or use ideas, theories, or perspectives considered central to an academic or professional discipline. 4. Disciplinary Process —The task asks students to use methods of inquiry, research, or communication characteristic of an academic or professional discipline. 5. Elaborated Written Communication —The task asks students to elaborate on their understanding, explanations, or conclusions through extended writing. 6. Problem Connected to the World Beyond the Classroom —The task asks students to address a concept, problem or issue that is similar to the one that they have encountered or are likely to encounter in life beyond the classroom. 7. Audience Beyond the School —The task asks students to communicate their knowledge, present a product or performance, or take some action for an audience beyond the teacher, classroom, and school building. Michigan Curriculum Framework

HOW to Develop Constructed Response Items:The Specifics • Set the Context • Specify the knowledge to be brought to bear • Specify the Reasoning • Use specific verbs e.g. analyze, cite, describe… • Point the Way • Inform students of the criteria that will be applied to evaluate their responses • Develop the Scoring Rubric • Clear articulation of the appropriate evaluation criteria by which to judge the quality of student responses. Adapted from Classroom Assessment for Student Learning, Richard Stiggins, et. al. 2006

Examples: 5th Grade Science E.ST.05.22 Explain moon phases as they relate to the position of the moon in its orbit around the Earth, resulting in the amount of observable reflected light. Sample Item: “Sometimes the Moon looks like a full circle, sometimes it looks like a half circle, and sometimes it looks like a crescent (Set the Context). Explain why (Specify the Reasoning) the Moon appears to be different shapes at different times. You may use labeled drawings in your explanation (Point the Way).” Released NAEP Item (http://nces.ed.gov)

Scoring Rubrics: Two Types Generic Rubrics “…define what content understanding looks like, in general, for any body of knowledge.” “3” The response is clear, focused, and accurate. “2” The response is clear and somewhat focused, but not compelling. “1” The response misses the point, contains inaccurate information, or otherwise demonstrates lack of mastery of the material. Classroom Assessment for Student Learning Richard Stiggins, et. al. 2006

Generic Rubrics: Another Example: Sophisticated: an unusually thorough, elegant, and inventive account; fully supported, verified, and justified; deep and broad: goes well beyond the information given. In-Depth: an atypical and revealing account, going beyond what is obvious or what was explicitly taught; makes subtle connections; well supported by argument and evidence; novel thinking displayed. Developed: reflects some in-depth and personalized ideas; going beyond the given – there is supported theory here, but insufficient or inadequate evidence and argument. Intuitive: an incomplete account but with apt and insightful ideas; extends and deepens some of what was learned; some “reading between the lines”; account has limited support/argument/data or sweeping generalizations. There is a theory, but one with limited testing and evidence. Naïve: a superficial account; more descriptive than analytical or creative; a fragmentary or sketchy account of facts/ideas or glib generalizations; a black-and-white account; less a theory than an unexamined hunch or borrowed idea. Understanding by Design Grant Wiggins & Jay McTight, 1998

Scoring Rubrics: Exercise-Specific “Points are awarded when specific information appears in students’ responses.” Classroom Assessment for Student Learning Richard Stiggins, et. al. 2006

Examples: 5th Grade Science Sometimes the Moon looks like a full circle, sometimes it looks like a half circle, and sometimes it looks like a crescent. Explain why the Moon appears to be different shapes at different times. You may use labeled drawings in your explanation. __________________________________________ __________________________________________ __________________________________________ ALL examples taken from NAEP Released Items http://nces.ed.gov/nationsreportcard

Exercise-specific Scoring Rubric Complete (3) Student explanation includes all the points given below. Student can provide a drawing correctly illustrating the phases of the moon. • The Moon is visible because it reflects (or is illuminated by) sunlight. • The Moon revolves around the Earth. • The portion of the illuminated half of the Moon that is visible from Earth changes, thus making the Moon appear to change shape. Partial (1-2) Student explains 1 or 2 aspects of the causes of the phases of the Moon without major misconceptions. Unsatisfactory/Incorrect (0) Student does not correctly explain any aspect of the phases of the Moon, or explains aspects but includes major misconceptions.

Sample Student Response: Complete (3/3)

Sample Student Response: Partial (1/3)

Student Response: Unsatisfactory (0/3)

Examples: 7th Grade Science S.IP.07.15 Construct charts and graphs from data and observations. A student took a sample of water from a pond and examined it under a microscope. She identified several species of protozoans, including two species of Paramecium that are known to eat the same food. The student decided to examine the water sample every day for a week. She added food for the Paramecia each day and counted the number of each species. Her findings are summarized in the table below.

Examples: 7th Grade Science NUMBER OF PARAMECIA IN POND WATER SAMPLE Day Species S Species T 1 50 50 2 60 80 3 100 90 4 150 60 5 160 50 6 160 30 7 160 20 Using the axes below, construct a graph showing the number of each species of Paramecium the student found each day. Be sure to label the axes.

Exercise-specific Scoring Rubric Complete (3): Student scales, plots, and labels the graph correctly. Essential (2): Student scales and plots S and T correctly; one or both axis labels or plot labels are missing (or incorrect). Partial (1): Student scales and plots either S or T correctly or scales and plots S and T combined (for example, adds data for each into one data point). No labels on graph. Unsatisfactory/Incorrect (0): Student fails to plot data for either S or T correctly or produces an illogical graph.

Sample Student Response: Complete (3/3)

Sample Student Response: Essential (2/3) “Provides correct scales and plots for both sets of data, but lacks a label for the y-axis.”

Sample Student Response: Essential (2/3) “Provides correct scales and plots for both sets of data, but lacks labels for both axes.”

Sample Student Response: Partial (1/3) “Using different types of graphs, both responses scale and plot only the data for species S. They also both lack labels for the axes.”

Sample Student Response: Unsatisfactory (0/3) “Illogical graph of the data.”

Examples: High School Science B1.2g Identify scientific tradeoffs in design decisions and choose among alternative solutions. AND B4.2h Recognize that genetic engineering techniques provide great potential and responsibilities. “Some people believe that recombinant DNA technology has serious disadvantages. Describe one disadvantage that might result from the use of recombinant DNA technology. Then describe a plan or a policy for dealing with the disadvantage that could be followed by research scientists, doctors, public officials, or other people who are involved with recombinant DNA technology and its uses.”

Exercise-specific Scoring Rubric Complete (3): Student response describes a reasonable disadvantage of recombinant DNA technology and provides a clear description of a plan for dealing with the disadvantage. Credited disadvantages (1 pt) include: • Regulation of new strains • Production of dangerous organisms • Genetic Similarity - loss of diversity • Regulation of applications/patents Credited acceptable plans (2 pts) include: • Informed consent • Regulation • Thorough testing • Oversight committee Essential (2): Student response describes a reasonable disadvantage and attempts a brief description of a plan for dealing with this disadvantage (e.g., test or observe, research further). OR Student response provides only a description of a plan. Partial (1): Student response describes a reasonable disadvantage of genetic technology but does not develop a plan for dealing with the disadvantage. Unsatisfactory/Incorrect (0): Student response states that there are no disadvantages, or states a disadvantage that is inaccurate or unreasonable.

Sample Student Response: Complete (3/3) “Student response states that making new kinds of viruses and mutations are a disadvantage, and outlines a plan that involves experimentation with human cells outside the body.”

Sample Student Response: Essential (2/3) “Student response discusses the production of dangerous viruses, and attempts a brief description of a plan.”

Sample Student Response: Essential (2/3) “Student response describes a disadvantage in general terms only, but does outline a plan that involves the setting of limitations.”

Sample Student Response: Partial (1/3) “Student response explains that the intermixing of genes could result in the production of nontreatable diseases. No plan is given.”

Sample Student Response: Partial (1/3) “Student response explains that the virus may have side effects, such as changing or altering a regular body function. No plan is given.”

Sample Student Response: Unsatisfactory(0/3) “Student response does not state a clear disadvantage or outline a coherent plan.”

Sample Student Response: Unsatisfactory(0/3) “Student response does not give a clear disadvantage, and states merely that these people should stop messing around with recombinant DNA technology.”

Activity #1: Write a Scoring Rubric Question 1 – 4th Grade Earth Science Question 2 – 5th Grade Life Science Question 3 – 7th Grade Physical Science Question 4 – 7th Grade Physical Science Question 5 – High School Biology Question 6 – High School Earth Science

Activity #1: Write a Scoring Rubric E.ST.04.22 Explain that the spin of the Earth creates day and night. “Everyone knows about day and night. Write what you think makes day and night. Draw a picture to show what you think.” NAEP Released Items http://nces.ed.gov/nationsreportcard

Activity #1: Write a Scoring Rubric “Everyone knows about day and night. Write what you think makes day and night. Draw a picture to show what you think.” 2 pts. The response indicates that the Earth turns so that the same face is not always facing the Sun. Example: “The Earth turns every 24 hours, and for 12 hours we are facing the Sun.” 1 pt. The response indicates that the Moon and Sun are on different sides of the Earth and the Earth rotates facing one and then the other. There is no implication that the Sun moves. Example: “In the day we face the Sun and in the night we turn to face the Moon.” 0 pt. The response indicates that the Sun moves (possibly across the sky) to cause night and day. Example: “The Sun moves and makes way for the Moon.” – PLUS – 1 pt. Diagram correctly illustrates Sun and Earth, with the side facing the Sun illuminated (day) and the side away from the Sun darkened (night). Adapted from Classroom Assessment for Student Learning, Stiggins, et. al. 2006

Activity #1: Write a Scoring Rubric L.OL.05.42 Explain how animal systems (digestive, circulatory, respiratory, skeletal, muscular, nervous, excretory, and reproductive) work together to perform selected activities. Sample Item: “When you exercise strenuously, your body produces excess heat. Describe at least two things your body does to help prevent your temperature from rising excessively, and explain why the body's response is effective.”

Activity #1: Write a Scoring Rubric Complete (3) Student indicates that the body keeps its temperature from rising through sweating or by blood vessels dilating AND states how these are effective – cooling by evaporation or transferring heat in blood to the air surrounding the skin. Partial (1-2) Student indicates one or two methods the body keeps its temperature from rising but does not explain fully how the mechanisms work. Unsatisfactory/Incorrect (0) Student provides little or no evidence of knowledge of any mechanism for losing heat during exercise.

Activity #1: Write a Scoring Rubric S.IP.07.12 Design and conduct scientific investigations. S.IP.07.13 Use tools and equipment (spring scales, stop watches, meter sticks and tapes, models, hand lens, thermometer, models, sieves, microscopes, hot plates, pH meters) appropriate to scientific investigations. Sample Item: “Explain how (Specify the Reasoning) you can find out the volume of a solid object, such as a small rock (Set the Context), using only water and either a measuring cup or a graduated cylinder (Point the Way).”

Activity #1: Write a Scoring Rubric Complete (3) Student correctly describes how to determine the volume of a solid object using water and a measuring cup or graduated cylinder. Comparison or change in volume of water should be explicit. Method A: Pour some water into the graduated cylinder Record the water level (1 pt.) Then put the rock in the graduated cylinder Record the water level again (1 pt.) The difference between the first and second volume measurements is the volume of the rock (1 pt.) Method B: Spillage Responses = Fill cup with water (to the top) (1 pt.), add rock, catch the water that overflows in separate container, and measure the overflow (1 pt.). Overflow = volume of the rock (1 pt.). Partial (1-2) Student gives the initial steps involved in measuring the volume of a solid object, but does not compare or discuss the change in water level, volume of overflow, or volume of the rock. Unsatisfactory/Incorrect (0) Student demonstrates no understanding of how to use water and a graduated cylinder to measure volume.

Activity #1: Write a Scoring Rubric S.IA.07.11 Analyze information from data tables and graphs to answer scientific questions. “One characteristic that can be used to identify pure metals is density. If you determine the density of a pure metal, you can determine what the metal is, as shown in the table below. Suppose that you determine that a metal ring has a density of 15.3 grams/cm3. Assume that the ring is a mixture of some combination of the metals listed in the table. What can you determine about its composition from its calculated density? Explain your answer.”

Activity #1: Write a Scoring Rubric Complete (3) Student response states that the ring is not pure gold but must contain some gold (1 pt.). Response may reason that the density of the ring (15.3 g/cm3) is less than the density of gold (19.3 g/cm3), but more than the density of any of the other metals (1 pt.). Response must also state that the identity of the other metals in the ring cannot be determined from the given information (1 pt.). Partial (1-2) Student response identifies gold as being one of the metals in the mixture and may choose others based on such factors as averaging densities. Response does not state that the identity of the other metals in the ring cannot be determined from the given information. Unsatisfactory/Incorrect (0) Student response does not identify gold as one of the metals in the mixture, or states that the ring is made of pure gold.

Activity #1: Write a Scoring Rubric B4.3g Explain that cellular differentiation results from gene expression and/or environmental influence (e.g., metamorphosis, nutrition). Sample Item: “Biologists know that nearly all cells in a person's body contain the same genes. For example, kidney cells contain the same genes as the cells that normally make hemoglobin. Given these facts, explain why kidney cells do not make hemoglobin even though they contain the hemoglobin gene.”

Activity #1: Write a Scoring Rubric Complete (3) Student response states that even though all cells in a person's body contain the same genetic information (1 pt.), different cells "use" different parts of this information at different times (1 pt.), i.e., the gene for hemoglobin may be "turned on" in hemoglobin-manufacturing (e.g. bone marrow) cells, but "turned off" (1 pt.) in kidney cells. Unsatisfactory/Incorrect Student response demonstrates little or no understanding of what causes different cells to perform different functions.

Writing Constructed Response Assessment Items: