MAINTAINING QUALITY IN BLENDED LEARNING: FROM CLASSROOM ASSESSMENT TO IMPACT EVALUATION PART I: DESIGNING AND EVALUATING

1. MAINTAINING QUALITY IN BLENDED LEARNING: FROM CLASSROOM ASSESSMENT TO IMPACT EVALUATIONPART I: DESIGNING AND EVALUATING LEARNING Suzanne Weinstein, Ph.D. Director of Instructional Consulting, Assessment, and Research The Schreyer Institute for Teaching Excellence The Pennsylvania State University

2. Part I Session Objectives As a result of this workshop, participants will be able to… • …write a student-centered, measureable learning objective • …map learning objectives to appropriate assessments • …use course-level evaluation data for program assessment

3. But first, some definitions

4. Assessment “Assessment is the ongoing process of establishing • clear, measurable expected outcomes of student learning, Suskie, L. (2009) Assessing student learning: A common sense guide. San Francisco: Jossey-Bass.

5. Assessment “Assessment is the ongoing process of establishing • clear, measurable expected outcomes of student learning, • ensuring that students have sufficient opportunities to achieve those outcomes, Suskie, L. (2009) Assessing student learning: A common sense guide. San Francisco: Jossey-Bass.

6. Assessment “Assessment is the ongoing process of establishing • clear, measurable expected outcomes of student learning, • ensuring that students have sufficient opportunities to achieve those outcomes, • systematically gathering, analyzing, and interpreting evidence to determine how well students learning matches the expectations and Suskie, L. (2009) Assessing student learning: A common sense guide. San Francisco: Jossey-Bass.

7. Assessment “Assessment is the ongoing process of establishing • clear, measurable expected outcomes of student learning, • ensuring that students have sufficient opportunities to achieve those outcomes, • systematically gathering, analyzing, and interpreting evidence to determine how well students learning matches the expectations and • using the resulting information to understand and IMPROVE student learning.” Suskie, L. (2009) Assessing student learning: A common sense guide. San Francisco: Jossey-Bass.

8. Evaluation “Evaluation is the analysis and use of data by faculty to make judgments about student performance. Evaluation includes the determination of a grade or a decision regarding pass/fail for an individual assignment or for a course.” Goldman and Zakel (2009)

9. A blended or hybrid course is one in which some face-to-face class periods are replaced with online activities +

10. Designing learning in a blended course

11. Designing learning in a (blended) course Wiggins and McTighe (1998), Understanding by Design http://pixel.fhda.edu/id/six_facets.html

12. Designing learning in a (blended) course Face-to -face or Online?

13. Designing learning in a (blended) course Face-to -face or Online? Integrate face-to-face and online elements

14. Designing learning in a (blended) course http://pixel.fhda.edu/id/six_facets.html

15. Adapted from Synthesis Worksheet: http://www.schreyerinstitute.psu.edu/pdf/Synthesis_Worksheet.pdf

16. Creating Learning Objectives

17. Identify what the learner should know or be able to do by the end of the course A learning objective should… • …focus on the learner • …be observable (i.e. measurable) http://www.schreyerinstitute.psu.edu/Tools/LearningObj/

18. Use Bloom’s taxonomy (1956) to develop learning objectives at various levels of learning Evaluation Synthesis Analysis Application Comprehension Knowledge Adapted from Eileen Herteis, The Gwenna Moss Teaching & Learning Centre

19. If you prefer, you can also use Krathwohl and Anderson version of the taxonomy Creating Evaluating Analyzing Applying Understanding Remembering Adapted from Eileen Herteis, The Gwenna Moss Teaching & Learning Centre

20. Chemistry Learning Objectives Students will be able to: • Identify the chemical deficiencies of the soil for a crop. • Distinguish between different types of chemicals based on the chemical deficiencies of the soil for a crop. • Develop a solution for the soil deficiencies in a proposal geared towards a business audience.

21. Evaluate the following learning objective 1. In this course you will understand the 3 major communication disorders.

22. Evaluate the following learning objective 2. In this course we will cover the religion of ancient Egypt.

23. Mapping Objectives to Assessment Strategies (mere examples) http://edorigami.wikispaces.com/Bloom%27s+-+Introduction

24. Mapping Objectives to Assessment Strategies (mere examples) http://edorigami.wikispaces.com/Bloom%27s+-+Introduction

25. One last, but not least, important consideration: Rubrics Rubrics for digital assignments http://edorigami.wikispaces.com/Rubrics+-+Bloom%27s+Digital+Taxonomy

26. Electronic assessment examples History course objective Students will be able to create personal histories that link to specific historical times. Family e-Album

27. Electronic assessment examples Information Science and Technology Students will be able to collaboratively developsummaries of research related to online games and virtual worlds. IST 440 Research Project

28. Give it a try Go to the following link. It will take you to a google form. The questions will walk you through the process of creating the appropriate assignment for a particular objective. https://spreadsheets.google.com/viewform?formkey=dDZSbS1XOHI4ZngtMzZLNno4ZkVnWHc6MQ

29. Let’s see the results • Blended Learning Workshop Activity - Spreadsheet

30. Use course-level assignments to measure program goals Psychology Program Goal Students will be able to understand and interpret psychological data, graphs and statistics. Course Objective Students will be able to correctly interpret a correlation coefficient.

31. Performance by student

32. Performance by learning objective

33. Resources • Hybrid Courses, University of Wisconsin at Milwaukee: http://www4.uwm.edu/ltc/hybrid/ • Schreyer Institute for Teaching Excellence, Penn State: http://www.schreyerinstitute.psu.edu/ • Bloom’s Digital Taxonomy: http://edorigami.wikispaces.com/Bloom%27s+-+Introduction • Bloom’s Digital Taxonomy- Rubrics http://edorigami.wikispaces.com/Rubrics+-Bloom%27s+Digital+Taxonomy • Evaluating Media Characteristics http://www.elearnspace.org/Articles/mediacharacteristics.htm

34. Suzanne Weinstein sweinstein@psu.edu 814-863-0689 http://www.schreyerinstitute.psu.edu http://www.assess.psu.edu CONTACT INFORMATION