1 / 73

Developing Assessment Literacy: A workshop on the relationship between instruction, assessment & evaluation

Developed & Presented by Edward J. Caropreso, PhD Watson School of Education University of North Carolina Wilmington caropresoe@uncw.edu 910.962.7830 Director of Academics Marine Corps Combat Service Support Schools Camp Johnson, NC January, 2011.

betty_james
Download Presentation

Developing Assessment Literacy: A workshop on the relationship between instruction, assessment & evaluation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Developed & Presented by Edward J. Caropreso, PhD Watson School of Education University of North Carolina Wilmington caropresoe@uncw.edu 910.962.7830 Director of Academics Marine Corps Combat Service Support Schools Camp Johnson, NC January, 2011 Developing Assessment Literacy:A workshop on the relationship between instruction, assessment & evaluation

  2. Developing Assessment Literacy: Learning Outcomes • 1. Recognize/value the relationship between components of professional practice (Inst; Asmt; Eval) 2. Recognize/value the relationship between professional vocabulary and practice 3. Recognize/value the use of professional vocabulary • 1. Recognize/value the relationship between goals/objectives, instruction & assessment/evaluation 2. Recognize differences between objectives models 3. Write objectives using a specific model 4. Critique objectives using a specified evaluation system • 1. Recognize/value the role of appropriate information in decision-making 2. Recognize/value the relationship between types of assessment & resulting information 3. State the difference between objective & subjective (non-objective) assessment • 1. State the difference between traditional & alternative assessment models 2. State the difference between models of alternative assessment 3. State the purpose/s for rubrics in assessment 4. Identify appropriate applications of rubrics (in specific assessment settings/contexts) • 1. Describe formative assessment strategies 2. State appropriate applications for formative assessment/s • 1. Recognize/value the need for systematic planning for assessment 2. Evaluate item types for given applications/purposes 3. Construct items to meet specific Los in given learning domains 4. Evaluate items/tests as to purpose/use of assessment information • 1. Describe validity; reliability 2. Explain the relationship between high quality assessment and validity/reliability • 1. Recognize/value the role of instructors as student study supports 2. Recognize the relationship between test item types & specific study strategies

  3. Developing Assessment Literacy: Learning Outcomes • 1. Recognize/value the relationship between components of professional practice (Inst; Asmt; Eval) 2. Recognize/value the relationship between professional vocabulary and practice 3. Recognize/value the use of professional vocabulary • 1. Recognize/value the relationship between goals/objectives, instruction & assessment/evaluation 2. Recognize differences between objectives models 3. Write objectives using a specific model 4. Critique objectives using a specified evaluation system • 1. Recognize/value the role of appropriate information in decision-making 2. Recognize/value the relationship between types of assessment & resulting information 3. State the difference between objective & subjective (non-objective) assessment • 1. State the difference between traditional & alternative assessment models 2. State the difference between models of alternative assessment 3. State the purpose/s for rubrics in assessment 4. Identify appropriate applications of rubrics (in specific assessment settings/contexts)

  4. Developing Assessment Literacy: Learning Outcomes • 1. Describe formative assessment strategies 2. State appropriate applications for formative assessment/s • 1. Recognize/value the need for systematic planning for assessment 2. Evaluate item types for given applications/purposes 3. Construct items to meet specific Los in given learning domains 4. Evaluate items/tests as to purpose/use of assessment information • 1. Describe validity; reliability 2. Explain the relationship between high quality assessment and validity/reliability • 1. Recognize/value the role of instructors as student study supports 2. Recognize the relationship between test item types & specific study strategies

  5. Workshop Contents • Introduction & Context for Training • Knowledge-Practice Model • Professional Vocabulary & Concepts • Context for Instruction • Educational Goals & Objectives • Models of Objectives • Working with Objectives • Context for Assessment & Evaluation • Role of Information for Decision-making • Types of data related to types of decisions • Assessment Model • Types of Assessments • Objective vs Subjective, Etc. • Alternative Assessment: Performance Strategies Complementing Paper Assessment • Formative Assessment: Information about Teaching & Learning to Improve Instruction • Working with Assessment Tools/Strategies: Writing assessment plans; developing assessment items within local work context • Assessment & Test Construction: Technical Features & “Quality” Tools • Validity • Reliability • Supporting Student Learning through Assessment • KSU Study Strategies; Test Preps • Improving Performance; MIT Test Strategies

  6. Workshop Contents • Introduction & Context for Training • Knowledge-Practice Model • Professional Vocabulary & Concepts • Context for Instruction • Educational Goals & Objectives • Models of Objectives • Working with Objectives • Context for Assessment & Evaluation • Role of Information for Decision-making • Types of data related to types of decisions • Assessment Model • Types of Assessments • Objective vs Subjective, Etc.

  7. Workshop Contents • Alternative Assessment: Performance Strategies Complementing Paper Assessment • Formative Assessment: Information about Teaching & Learning to Improve Instruction • Working with Assessment Tools/Strategies: Writing assessment plans; developing assessment items within local work context • Assessment & Test Construction: Technical Features & “Quality” Tools • Validity • Reliability • Supporting Student Learning through Assessment • KSU Study Strategies; Test Preps • Improving Performance; MIT Test Strategies

  8. Workshop Contents • Introduction & Context for Training • Knowledge-Practice Model • Professional Vocabulary & Concepts

  9. Developing Assessment Literacy: Learning Outcomes • 1. Recognize/value the relationship between components of professional practice (Inst; Asmt; Eval) 2. Recognize/value the relationship between professional vocabulary and practice 3. Recognize/value the use of professional vocabulary

  10. 1. Introduction & Context for Training • Knowledge-Practice Model of Assessment & Evaluation • Professional Vocabulary for Concepts

  11. Knowledge-Practice Model Developing Professional Knowledge Knowledge of Learner Achievement Resulting from Instruction Knowledge of Instructional Practice: Strategies & Activites (incl. assessment) Dynamic Reciprocity

  12. Knowledge-Practice Model • Developing Professional Knowledge

  13. Knowledge-Practice Model(Discussion Activity) • What’s the current status of your professional knowledge? • How have you developed your knowledge of instruction, assessment and evaluation? • What are your expectations for developing your professional knowledge & practice? • What are your professional development alternatives?

  14. Professional Vocabulary for ConceptsActivity: Review/id concepts you know well; compare/contrast with colleagues’/groups’ Resources & Databases • New Horizons for Learning • http://www.newhorizons.org/strategies/assess/terminology.htm • Nat’l. Center for Research on Evaluation, Standards, & Student Testing (CRESST @ UCLA) • http://www.cse.ucla.edu/products/glossary.html • Coalition of Essential Schools: Defining Assessment • http://www.essentialschools.org/cs/resources/view/ces_res/124 • System of Adult Basic Education Supports • http://www.sabes.org/assessment/glossary.htm • Beyond Confusion: An Assessment Glossary, by Andrea Leskes • http://ctl.stanford.edu/Tomprof/postings/448.html • Continuous Assessment: Practical Guide for Teachers

  15. Professional Vocabulary(Activity) • Review the glossaries • Identify the terms & concepts you know • Mark the ones you know & understand • Compare your list to colleagues’/groups’ • What’s the same; what’s different? • Why are there differences? • What are the implications for professional practice given inconsistencies in the knowledge base?

  16. Workshop Contents • Context for Instruction • Educational Goals & Objectives • Models of Objectives • Working with Objectives

  17. Developing Assessment Literacy: Learning Outcomes • 1. Recognize/value the relationship between components of professional practice (Inst; Asmt; Eval) 2. Recognize/value the relationship between professional vocabulary and practice 3. Recognize/value the use of professional vocabulary • 1. Recognize/value the relationship between goals/objectives, instruction & assessment/evaluation 2. Recognize differences between objectives models 3. Write objectives using a specific model 4. Critique objectives using a specified evaluation system

  18. 2. Context for Instruction • Educational Goals & Objectives • Models of Objectives • Working with Objectives

  19. Educational Goals & Objectives • Educational Goals: • “…those human activities which contribute to the functioning of a society (including the functioning of an individual in society), and which can be acquired through learning.” (Gagne, Briggs, & Wagner, 1988, p. 39; cited in Nitko, 2001, p. 23) • Goals are stated in broad terms that give direction and purpose to planning overall educational activities.

  20. Educational Goals & Objectives • Educational Goals (or global objectives) : • Broad, complex learning outcomes that require substantial time and instruction to accomplish • Typically encompassing many specific learning objectives which require learner success on most/all to reach the general goal (from Airasian, 2001)

  21. Educational Goals & Objectives Educational Goals • UT Examples of Goals vs Objectives • More Examples of Goals vs Objectives • Airasian’s 3-Level Model

  22. Models of Objectives • Two Common Models: Mager & Gronlund • Mager Tips • Gagne & Briggs & ABCD Models • A Comparison of Models • Greenberg’s 6 Keys to Successful Performance Objectives

  23. Comparison of Formats[Seels & Glasgow (1990). Exercises in Instructional Design. Columbus, Ohio: Merrill.]

  24. Working with Objectives • Understanding the Context of Learning • How do you make sense of learning? • How do you identify/describe expectations? • Putting Bloom’s Taxonomy to Work • Three Types of Learning • What Drives Statements of Learning: Active Verbs

  25. Working with Objectives: Activity • Identify & State 1-3 TLOs (goals; in your area) • For each TLO (goal), using the course materials, identify & state 1-2 knowledge & 1-2 performance objectives (ELOs) in at least two models we’ve reviewed • Compare your objectives with a colleague’s • Using Greenberg’s Criteria to review your objectives, how do they hold up? • What’s similar to your colleague? • What’s different? • Keep your objectives for later; we’ll use them again!

  26. Workshop Contents • Context for Assessment & Evaluation • Role of Information for Decision-making • Types of data related to types of decisions • Assessment Model • Types of Assessments • Objective vs Subjective, Etc.

  27. Developing Assessment Literacy: Learning Outcomes • 1. Recognize/value the relationship between components of professional practice (Inst; Asmt; Eval) 2. Recognize/value the relationship between professional vocabulary and practice 3. Recognize/value the use of professional vocabulary • 1. Recognize/value the relationship between goals/objectives, instruction & assessment/evaluation 2. Recognize differences between objectives models 3. Write objectives using a specific model 4. Critique objectives using a specified evaluation system • 1. Recognize/value the role of appropriate information in decision-making 2. Recognize/value the relationship between types of assessment & resulting information 3. State the difference between objective & subjective (non- objective) assessment

  28. 3. Context for Assessment & Evaluation • Establishing vocabulary & conceptual framework • Assessment and Evaluation: • What do they mean? • Relationship between assessment, evaluation and decision-making: • The role of time • What about instructional outcomes? • Instruction, assessment and evaluation • Developing Criteria

  29. 3. Context for Assessment & Evaluation • Model of Assessment • Identifying & applying appropriate strategies to gather information for specific purposes &/or decisions • Determining appropriate assessment strategies • Appropriately applying strategies

  30. 3. Context for Assessment & Evaluation: Assessment Model Purposeful Decision Making Assess Instructional Practices: Methods, Materials, Strategies & Activities Assess Learner Achievement Resulting from Instruction Dynamic & Reciprocal

  31. 3. Context for Assessment & Evaluation • Types of Assessments by Purpose • Bloom’s Taxonomy as Context • Aim High • Objective Assessment • Selected vs Constructed Response • Subjective (Non-Objective) Assessment • Writing/Generating Responses • Role & Control of Subjectivity in Assessment • Alternative Assessment • Authentic Assessment • Performance Assessment

  32. 3. Context for Assessment & Evaluation [http://vudat.msu.edu/assessment/] • Objective Assessment • Objective assessments (usually multiple choice, true false, matching, short answer) have correct answers. • These are good for testing recall of facts and can be automated. • Objective tests assume that there are true answers and assume that all students should learn the same things.

  33. 3. Context for Assessment & Evaluation[http://vudat.msu.edu/assessment/] • Subjective (Non-Objective) Assessment • With subjective assessments, teacher's judgment determines the grade • These include essay tests. • Essay tests take longer to answer and they take longer to grade than objective questions and therefore only include a small number of questions, focusing on complex concepts. • Writing/Generating Responses • Role & Control of Subjectivity: Criteria &/or Rubrics

  34. Workshop Contents • Introduction & Context for Training • Knowledge-Practice Model • Professional Vocabulary & Concepts • Context for Instruction • Educational Goals & Objectives • Models of Objectives • Working with Objectives • Context for Assessment & Evaluation • Role of Information for Decision-making • Types of data related to types of decisions • Assessment Model • Types of Assessments • Objective vs Subjective, Etc. • Alternative Assessment: Performance Strategies Complementing Paper Assessment

  35. Developing Assessment Literacy: Learning Outcomes • 1. Recognize/value the relationship between components of professional practice (Inst; Asmt; Eval) 2. Recognize/value the relationship between professional vocabulary and practice 3. Recognize/value the use of professional vocabulary • 1. Recognize/value the relationship between goals/objectives, instruction & assessment/evaluation 2. Recognize differences between objectives models 3. Write objectives using a specific model 4. Critique objectives using a specified evaluation system • 1. Recognize/value the role of appropriate information in decision-making 2. Recognize/value the relationship between types of assessment & resulting information 3. State the difference between objective & subjective (non-objective) assessment • 1. State the difference between traditional & alternative assessment models 2. State the difference between models of alternative assessment 3. State the purpose/s for rubrics in assessment 4. Identify appropriate applications of rubrics (in specific assessment settings/contexts)

  36. 4. Alternative Assessment: Performance Strategies Complementing Paper Assessment • Alternative • To what? • Portfolio Assessment: An example • Authentic • With respect to what? • Performance • What types?

  37. 4. Alternative Assessment : Activity • Alternative • Possible examples? • Portfolio Assessment: Possible examples? • Authentic • Possible examples? • Performance • Possible examples?

  38. 4. Alternative Assessment: Alternative Methods to Document Learning • 3 Formal Models of Assessment: • Holistic: • Applying all criteria as a single judgment • Modified Holistic: • Adding levels of judgment • Analytical: • Judgment by criteria; weighting factors; scaling • Valid decision-making: • Matching the “tool” and the instructional outcome • Examples of Rubrics: Rubric Primer • Information Skills • Oral Presentation • Creating Rubrics

  39. Workshop Contents • Introduction & Context for Training • Knowledge-Practice Model • Professional Vocabulary & Concepts • Context for Instruction • Educational Goals & Objectives • Models of Objectives • Working with Objectives • Context for Assessment & Evaluation • Role of Information for Decision-making • Types of data related to types of decisions • Assessment Model • Types of Assessments • Objective vs Subjective, Etc. • Alternative Assessment: Performance Strategies Complementing Paper Assessment • Formative Assessment: Information about Teaching & Learning to Improve Instruction

  40. Developing Assessment Literacy: Learning Outcomes • 1. Recognize/value the relationship between components of professional practice (Inst; Asmt; Eval) 2. Recognize/value the relationship between professional vocabulary and practice 3. Recognize/value the use of professional vocabulary • 1. Recognize/value the relationship between goals/objectives, instruction & assessment/evaluation 2. Recognize differences between objectives models 3. Write objectives using a specific model 4. Critique objectives using a specified evaluation system • 1. Recognize/value the role of appropriate information in decision-making 2. Recognize/value the relationship between types of assessment & resulting information 3. State the difference between objective & subjective (non-objective) assessment • 1. State the difference between traditional & alternative assessment models 2. State the difference between models of alternative assessment 3. State the purpose/s for rubrics in assessment 4. Identify appropriate applications of rubrics (in specific assessment settings/contexts) • 1. Describe formative assessment strategies 2. State appropriate applications for formative assessment/s

  41. 5. Formative Assessment: Information about Teaching & Learning to Improve Instruction • Formative Assessment • Types of Formative Assessment • Objective assessments • Forms/versions of summative assessments that will be implemented for final evaluation of students • Alternative assessments • Forms/versions of summative assessments that could be implemented for final evaluation of students • Used to provide feedback on developing learning • Should not contribute to final grades • May or may not be “graded” assessments • Limited direct impact on final evaluation of student

  42. 5. Formative Assessment: Information about Teaching & Learning to Improve Instruction • Observation Strategies • Using explicit formal tools • Informal observations • Questioning Strategies • Taba Tables • Self & Peer Assessment Strategies

  43. 5. Formative Assessment: Information about Teaching & Learning to Improve Instruction • Alternative assessments • Used to provide feedback on developing learning • Examples: • Formative Evaluation Form • Summary Evaluation Form • Observation Record Form • Observation Guide Form • Observation Assessment Form • Open-ended Questions

  44. Workshop Contents • Introduction & Context for Training • Knowledge-Practice Model • Professional Vocabulary & Concepts • Context for Instruction • Educational Goals & Objectives • Models of Objectives • Working with Objectives • Context for Assessment & Evaluation • Role of Information for Decision-making • Types of data related to types of decisions • Assessment Model • Types of Assessments • Objective vs Subjective, Etc. • Alternative Assessment: Performance Strategies Complementing Paper Assessment • Formative Assessment: Information about Teaching & Learning to Improve Instruction • Working with Assessment Tools/Strategies: Writing assessment plans; developing assessment items within local work context

  45. Developing Assessment Literacy: Learning Outcomes • 1. Recognize/value the relationship between components of professional practice (Inst; Asmt; Eval) 2. Recognize/value the relationship between professional vocabulary and practice 3. Recognize/value the use of professional vocabulary • 1. Recognize/value the relationship between goals/objectives, instruction & assessment/evaluation 2. Recognize differences between objectives models 3. Write objectives using a specific model 4. Critique objectives using a specified evaluation system • 1. Recognize/value the role of appropriate information in decision-making 2. Recognize/value the relationship between types of assessment & resulting information 3. State the difference between objective & subjective (non-objective) assessment • 1. State the difference between traditional & alternative assessment models 2. State the difference between models of alternative assessment 3. State the purpose/s for rubrics in assessment 4. Identify appropriate applications of rubrics (in specific assessment settings/contexts) • 1. Describe formative assessment strategies 2. State appropriate applications for formative assessment/s • 1. Recognize/value the need for systematic planning for assessment 2. Evaluate item types for given applications/purposes 3. Construct items to meet specific Los in given learning domains 4. Evaluate items/tests as to purpose/use of assessment information

  46. 6. Working with Assessment Tools & Strategies (within local work context) • Writing assessment plans • Who’s responsible for developing plans? • What’s the basis for developing plans? • Developing assessment tools & items • Who develops summative assessments? • Who implements summative assessments? • Who interprets data from summative assessments? • Formative assessments • Who develops formative assessments? • Who implements formative assessments? • Who interprets data from formative assessments?

  47. 6. Working with Assessment Tools & Strategies • Developing Objective Tests & Assessments • Planning Assessments • Types of Objective Assessments • Sample Table of Specifications • Judging Assessments

  48. 6. Working with Assessment Tools & Strategies • Developing Objective Tests & Assessments • Resources online @ • Writing Test Items • Testing Primer • Matching Items to Objectives

  49. 6. Working with Assessment Tools & Strategies • Developing Objective Tests & Assessments • Test Construction Activity (online @ http://people.uncw.edu/caropresoe/)

  50. 6. Working with Assessment Tools & Strategies • Developing Subjective (Non-objective) Tests & Assessments • See Test Primer materials (online @ http://people.uncw.edu/caropresoe/)

More Related