Tools for Supporting Increased Rigor & High-Quality Assessment and Instruction

1. Tools for Supporting Increased Rigor & High-Quality Assessment and Instruction Verona Public Schools Karin K. Hess, Ed.D. www.karin-hess.com

2. Track your reflections as we work… Ways I am refining my thinking about DOK/rigor… • ? • ? Scaffolding strategies for getting students to deeper thinking… • ? • ?

3. Before we begin… • Take a minute to jot down words/phrases that come to mind when you think of “cognitive rigor” as it relates to instruction, learning, and/or assessment.

4. Let’s apply your rigor definitions Your class has just read some version of Little Red Riding Hood. • What is a basic comprehension question you might ask? • What is a more rigorous question you might ask?

5. The Hess Cognitive Rigor Matrix integrates Bloom+Webb Different states/schools/teachers use different models to describe cognitive rigor. Each addresses something different. • Bloom – What type of thinking (verbs) is needed to complete a task? • Webb – How deeply do you have to understand the content to successfully interact with it? How complex is the content?

6. Merging Bloom + Webb:The thinking behind the development of the Hess Cognitive Rigor Matrix …

7. Bloom’s Taxonomy [1956 ] & Bloom’s Cognitive Process Dimensions [2001]

8. Webb’s Depth-of-Knowledge Levels • DOK-1 – Recall & Reproduction - Recall of a fact, term, principle, concept, or perform a routine procedure • DOK-2 - Basic Application of Skills/Concepts - Use of information, conceptual knowledge, select appropriate procedures for a task, two or more steps with decision points along the way, routine problems applying 2+ concepts, organize/display data, interpret/use simple graphs • DOK-3 - Strategic Thinking - Requires reasoning, developing a plan or sequence of steps to approach problem; requires some decision making and justification; abstract, complex, or non-routine; often more than one possible answer or approach • DOK-4 - Extended Thinking - An original investigation or application to real world; requires time to research, problem solve, and process multiple conditions of the problem or task; non-routine manipulations, across disciplines/content areas/multiple sources

9. DOK Misconception #1: All kids can’t do this; or Kids don’t need scaffolding to get “up” there. Engaging in “a complex task” with supports/ scaffolding is an essential step along the way to proficiency (Vygotsky’s ZPD) • Do it with others first; DOK 3 and 4 are not meant to only be done alone/independently, especially at first • Oral language & meaningful discourse support deeper thinking and increase initial exposures to the content and student engagement. This is NOT cheating! • One strategy: Plan questioning & formative probes from DOK 1-2-3-4 over the course of a lesson or unit of study. Consider all DOK levels in your planning.

10. Vygotsky: Zone of Proximal Development(What a child can do with assistance today) What a child can do independently tomorrow/future What a child can do independently now: “ENTRY” Actual Development Area Potential Development Area The ZONE LEARNING PROGRESSIONS ZONE: Dynamic area Causes development to move forward Social interaction essential (scaffolding) Karin Hess (2008). Using learning progressions as a schema to monitor progress across grades.

11. DOK Misconception #2:Webb’s DOK model is a taxonomy • Bloom’s is a taxonomy, intended to be a hierarchy • Primary Weaknesses of Bloom: generic verbs (void of content); same verbs at different levels • Webb’s DOK model is nominative: • It names how you interact with content • It differentiates varying levels of engagement with content and suggests what tasks might look like • DOK 4 is not better than DOK 3, or DOK 2, or DOK 1

12. DOK Misconception #3:Bloom verbs & levels = Webb DOK The DOK “Wheel of Misfortune” implies that a DOK level is indicated by a particular verb or set of verbs. Norman Webb, “It’s what comes after the verb, that indicates the complexity of a task.”

13. DOK Misconception #4: DOK is about difficulty. • The intended student learning outcome determines the DOK level. What mental processing must occur? DOK = Complexity, not difficulty! • While verbs may appear to point to a DOK level, it is what comes after the verb that is the best indicator of the rigor/DOK level and complexity of the task. • Describe the information contained in graphics or data tables in the text; or the rule for rounding a number • Describe how the two story characters are alike and different. • Describe the data or text evidence that supports your solution, reasoning, or conclusions • Describe varying perspectives on global climate change using supporting scientific evidence, and identify the most significant effects it might have on the planet in 100 yrs.

14. DOK LEVEL 1 Recall and Reproduction DOK 2 Skills and Concepts DOK LEVEL 3 Reasoning DOK 4 Extended Thinking Remember Understand Create Apply Analyze Evaluate Recall, locate basic facts, definitions, details and events Select appropriate word when intended meaning is clear Explain how concepts relate to other content domains Explain relationships Summarize Central ideas Explain, generalize or connect ideas using supporting evidence (quote, text, evidence, data, etc.) Devise an approach among alternatives to research a novel problem Use language structure or word relationships (synonyms/anto-nyms) Use context to find meaning Obtain and use information in text features Use concepts to solve non-routine problems and justify solutions with evidence Compare literary elements, facts, terms and events. Analyze format, organization & text structures Analyze or interpret author’s craft (e.g., literary devices, viewpoint, or potential bias) to critique a text Analyze multiple sources or texts Analyze complex abstract themes. Identify information in a graphic, table, visual, etc. “UGs” Cite evidence and develop a logical argument for conjectures based on one text or problem. Evaluate relevancy, accuracy and completeness of information Brainstorm ideas, concepts, problems, or perspectives related to a topic Generate conjectures or hypotheses based on observations or prior knowledge Develop a complex model or approach for a given situation Develop an alternative solution Synthesize information across multiple sources Articulate a new voice, theme, perspective. Bloom’s Taxonomy + Webb’s DOK = the Hess CRM

15. The Hess Cognitive Rigor Matrix Applies Webb’s DOK toBloom’s Cognitive Process Dimensions

16. DOK Misconception #5:All DOK levels can be assessed with a multiple choice question • That’s just dumb! • Weak DOK 3 multiple choice items are possible; but does selecting the best option (e.g., locate supporting evidence for a theme) provide as much insight as seeing HOW a student formulates and reveals thinking? • By their nature, DOK 3 and 4 questions/tasks are more open-ended, generally take longer to respond to/solve, and may have more than one “appropriate right answer”

17. 2. The DOK Matrix Instructional Paths Instruction & Assessment Decisions… Selected Response DOK 4 Extended Thinking Constructed Response Each standard has an assigned Depth of Knowledge. DOK 3 Reasoning and Thinking DOK 2 Skills and Concepts Performance Tasks The DOK determines the cognitive level of instruction. DOK 1 Recall and Reproduction -Explain how concepts or ideas specifically relate to other content domains. Explain, generalize or connect ideas using supporting evidence (quote, evidence, data) Remember Explain relationships Summarize State central idea Recall, locate basic facts, definitions, details, events Devise an approach among many alternatives to research a novel problem Use concepts to solve non-routine problems and justify Select appropriate words for use when intended meaning is clearly evident. Apply Understand Use context for word meanings Use information using text features Analyze multiple sources or multiple text Analyze complex abstract themes Analyze . Analyze or interpret author’s craft (e.g., literary devices, viewpoint, or potential bias) to critique a text Evaluate relevancy, accuracy and completeness of information across texts or sources Cite evidence and develop a logical argument for conjectures based on one text or problem Evaluate Create . Develop a complex model or approach for a given situation Develop an alternative solution Synthesize across multiple sources/ texts Articulate a new voice, theme, or perspective

18. Let’s practice using the CRM-back to Little Red Riding Hood Your sample questions – a basic and more rigorous question Handout #2: Linking Research, Tool #1: CRM template for Close Reading (page 4)

20. Some other content examples… • Your class will be learning about… • Fractions/decimals; Data use • Data use; scientific investigation • Elements of art & principles of design • Come up with a basic understanding and more rigorous question you could pose. • Use a CRM Template to Plan Units

22. DOK Misconception #6:Higher order thinking = deeper learning • What we have thought of as “higher order” (analysis, evaluation, creative thinking) might only be engaging or fun…and not always deeper • Many critical thinking examples do not go deep or get to DOK 3 or 4 (e.g., interpret/solve and justify) • Shift our thinking from “higher order” to deeper learning, and that can mean: • deeper understanding • deeper application • deeper analysis, etc. The Hess CRM illustrates this shift

23. Some general rules of thumb… • If there is one correct answer, it is probably level DOK 1 or DOK 2 • DOK 1: you either know it (can recall it, locate it, do it) or you don’t know it • DOK 2 (conceptual): apply one concept, then make a decision before going on applying a second concept; express relationship (if-then; cause-effect) • If more than one answer/approach, requiring evidence, it is DOK 3 or 4 • DOK 3: Must interpret, provide supporting evidence and reasoning (not just HOW solved, but WHY it works– explain reasoning for each step/decision made) • DOK 4: all of “3” + use of multiple sources/data/ texts; initiate & complete an investigation

24. DOK Misconception #7:Multi-step or longer tasks, multiple texts, or complex texts always means deeper thinking • DOK 2 is not simply more than one step, it’s applying more than one concept; DOK 2 is still routine/typical (main idea, word problems, etc.) • Simply reading more complex texts, but NOT delving deeply into the text’s meaning/style/etc., is likely to still be DOK 1 or 2 • DOK 3 requires some aspect of open-endedness and interpretation with justification or support; DOK 4 = drawing from multiple sources

25. Reading Standards

26. Reading & Writing

27. Math Content Standards&MathPractices

28. For each assessment task (or rubric)…ask • What is its purpose? (What content/skill is being assessed? is there a ‘right’ answer?) • What is the implied/intended rigor? (What mental processing would you expect students to engage in? Use the CRM to find descriptors.) • Which standards does it REALLY assess? (content + intended rigor) • Does the scoring guide/rubric match content + intended rigor? • What would student responses tell a teacher if students could/could not do all or part of this task? (open-ended tasks, reasoning used) – next instructional decisions are clear

29. Take-Away Messages: Cognitive Rigor & Some Implications for Assessment • Begin with daily DOK3 classroom discourse! • Assessing only at the highest DOK level (the “ceiling”) will miss opportunities to know what students do & don’t know – go for a range; end “high” in selected/prioritized content • Performance assessments can offer varying levels of DOK embedded in a larger, more complex task • Planned formative assessment strategies and tools can/should focus on differing DOK levels

30. Some resources worth exploring… • www.nms.org – national math science initiative – gr 3 – HS; also has ELA and SS performance tasks with DOK designations • www.readworks.org – gr k-8; short literary & informational texts with CC Qs • http://www.louisianabelieves.com/resources/library/common-core-state-standards - LA gr 3-HS math & ELA tasks (some math samples are weak and not all math DOK levels are correct, but generally good examples) • http://ccgpsmathematicsk-5.wikispaces.com/K-5+Formative+Assessment+Lessons+%28FALs%29 - GA gr K-5 math • www.exemplars.com – sample math PAs for k-12; science PAs for k-8 • Dan Meyer blog – math PAs for MS-HS; kids have to build the problems by deciding what’s needed to solve them – good strategic thinking required • Hess & Gong (2014). Ready for college and career? Achieving the Common Core Standards and beyond through deeper, student-centered learning http://www.nmefoundation.org/resources/scl-2/ready-for-college-and-career • Links on www.karin-hess.com: • Karin’s YouTube video – “Text Complexity Tools” (qualitative text analysis) • Karin’s vimeoDOK video – http://vimeo.com/18281415 • Karin’s YouTube DOK video –excerpt from a CT workshop