Welcome!

1. Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1

2. Goals for the Afternoon • Share lessons learned from the ASK Project (an assessment design project funded by the National Science Foundation). • Designing assessment systems for science curriculum. • Understanding best practices. 2

3. Purposes Classroom assessment practices Develop assessment strategies and practices that lead to better student achievement and enhance instructional practices. Accountability Develop assessments with the technical quality needed to provide accountability information to districts. 3

4. Madison Charleston San Diego Fresno Vancouver Vista Phoenix Manhattan ME ST ME ID WA EM HB ST PS PS HB WA EM MS LP LF FN EV FN MD LF LP VB MX VB MD SE MX Advisory Board BEAR External Evaluator Formative Evaluator LHS Las Vegas MS ID LF SE EV 4

5. Two Reports from NRC 5

6. The Assessment Triangle OBSERVATION INTERPRETATION COGNITION 6

7. The Assessment Triangle OBSERVATION INTERPRETATION COGNITION 7

8. Cognition • The Cognitive Perspective • Focuses on how people develop structures of knowledge—content, reasoning, and problem solving. • The Situative Perspective (Sociocultural) • Focuses on how people learn to participate in the practices, goals, and habits of mind of a particular community. 8

9. Inquiry Constructs 9

10. Physics of Sound Constructs 10

11. Progress Levels Strategic Conceptual Recognition Notions 11

12. The Assessment Triangle OBSERVATION INTERPRETATION COGNITION 12

13. Observation—Item Development • Because you can’t see directly into a child’s mind, you have to find ways to infer evidence of learning. • Items are written • Panels review items • Researchers look at cognitive demands • Students and teachers test them out! • Empirical data informs further revisions 13

14. ASK Assessment Items • Embedded Assessment items (diagnostic and part of daily instruction) • Notebook entries • Notebook sheets • Response sheets • Benchmark Assessment items (evaluative, but also formative—given periodically) • Short Answer • Open Response • Multiple Choice/Multiple Answer 14

15. Benchmark Assessments • Before starting • After Investigation 1 • After Investigation 2 • After Investigation 3 • After Investigation 4 • After completing the module • Survey • Inv 1 I-Check • Inv 2 I-Check • Inv 3 I-Check • Inv 4 I-check • Posttest 15

16. The Assessment Triangle OBSERVATION INTERPRETATION COGNITION 16

17. Coding (scoring) guides— Based on “Progress Levels” 4 Strategic 3 Conceptual 2 Recognition 1 Notions 0 Makes no attempt (Coding guides are 0-2, 0-3 or 0-4 depending on the level of the question.) 17

18. Interpretation • Qualitative Methods • Teacher feedback, expert panels and cognitive analysis. • Quantitative Methods • Item Response Theory (statistical analysis) • ClassMap (computer program/reports) 18

19. The Assessment Triangle OBSERVATION (Item development) INTERPRETATION (Coding guides and measurement model) COGNITION (Construct Maps & Progress Maps) 19

20. Focus on Embedded Assessment How is embedded assessment enacted within a curriculum? Context: Physics of Sound 20

21. What students have learned before the lesson we’ll be doing Sound is caused by vibrations. • A sound source is an object that is vibrating. • A sound receiver detects sound vibrations. • Sounds have identifiable properties. • Pitch (a property of sound) is how high or low the sound is (volume = loud or soft). • The faster an object vibrates the higher the pitch (and vice versa). 21

22. Notes to Myself Prep for Inv 2 • Review the At-a-Glance chart • Read Background for the Teacher and Teaching Children About… • Read through the steps in Guiding the Investigation Review the I-Check for Investigation 2 Review the focus questions 22

23. Notes to Myself Prep for Inv 2 The three big ideas: • Sounds are caused by vibrations • Clarify distinction between pitch and volume • The length of the instrument affects the pitch • Shorter length = higher pitch • Longer length = lower pitch • Tension also affects the pitch • More tension = higher pitch • Less tension = lower pitch 23

24. Focus Question • How does length affect the pitch (and therefore the speed of vibrations)? • Record observations: • Waterphone • Xylophone • Kalimba • String Beam 24

25. Class Discussion • How does length affect the pitch (and therefore the speed of vibrations)? • Record observations: • Waterphone • Xylophone • Kalimba • String Beam 25

26. Notes to Myself Length/Pitch Key √ = Got it! rev = reversed ? = confused da = didn’t answer the question na = no answer Gen Cala Derek Halyn Badu Ada Edita Flavo 26

27. Notes to Myself Length/Pitch Can students distinguish high from low pitches? Do they know the rule? Why are they inconsistent? 27

28. Next Steps (Can students discriminate pitch?) • Activity: • Close your eyes. • Listen to the sounds. • If the sound has a high pitch, show thumbs up. • If the sound has a low pitch, show thumbs down. 28

29. What are our “rules?” • The longer the length, the ___________ the pitch, and the ___________ the vibrations. 29

30. What are our “rules?” • The shorter the length, the ___________ the pitch, and the ___________ the vibrations. 30

31. Next Steps • The longer the length, the lower the pitch, the slower the vibrations. • The shorter the length, the higher the pitch, the faster the vibrations. 31

32. 32

33. 33

34. Next Steps • The longer the length, the lower the pitch, the faster the vibrations. • The shorter the length, the higher the pitch, the slower the vibrations. 34

35. 35

36. Next Steps • What do the dots tell you about your use of the “rule?” • Rewrite any sentences you marked with a red or yellow dot. 36

37. 37

38. Benchmark Assessments • Purposes: • Summative (measure for giving grades) • Formative (continue the learning) • Procedures: • Take the test • Code, but don’t mark on papers • Use self-assessment strategies with students before revealing codes. 38

39. Students have Learned… • Sound is caused by vibrating objects. • Length affects pitch • Longer = lower pitch(slower vibrations) • Shorter = higher pitch(faster vibrations) • Tension affects pitch • Looser = lower pitch(slower vibrations) • Tighter = higher pitch(faster vibrations) 39

40. Benchmark Assessment • Take the I-Check for Investigation 2 40

41. Behind the scenes • After students take the benchmark, the teacher codes the items to determine what students know and what they still need help with. • The teacher then plans which items to use to help students self-assess their understanding. (we’ll come back to coding later) 41

42. Self-assessment Strategy Multiple-Choice Corners Look at item 13 • You will be assigned a letter A, B, or C • (In the classroom students would go to the corner that corresponds with their actual answer.) • Meet with your group to create an argument to convince the other groups in the room that your answer is the correct one. • If at some point you “disagree with yourself,” you are allowed to change corners. 42

43. Self-assessment Strategy Sentence Starters • I used to think….but now I think… • I should have gotten this one right, I just… • I know…but I’m still not sure about… • The most important thing to remember is… • Can you help me with… • Next time I will remember to… • I’m confused about… • Now I know… 43

44. Sentence StartersAndy 44

45. Sentence StartersAndy 45

46. Sentence StartersRachael 46

47. Sentence StartersRachael 47

48. Coding Benchmark Items 44 Note: Coding Guides may be 0–2, 0–3 or 0–4 48

49. Levels of Progress 4 Strategic 3 Conceptual 2 Recognition 1 Notions 0 No attempt 49

50. Code Inv 2 I-Check • Review the coding guide for first item. • Code responses for all students on that item • Go on to the next item 50