Developing Effective and Measurable Learning Outcomes

1. Developing Effective and Measurable Learning Outcomes Stacey Hall, Ph.D. – University of New Hampshire Kathleen Hatch – Washington State University Don Stenta, Ph.D. – Ohio State University

2. Session Outcomes • Principles of Best Practice • 3 sources for Learning Outcomes • Writing Learning Outcomes • Implementing Learning Outcomes • Assessing Learning Outcomes • Utilizing Assessment of Learning Outcomes

3. Seven Principles for Good Practice in Undergraduate Education “The principles provide a common ground for faculty and students in their quest for meaningful learning. As a generally agreed-upon philosophy of ‘good’ teaching and learning, these principles establish fundamental guidelines for quality higher education and can be used as the building blocks for success by faculty, students, administrators and staff.”

4. Introduction • Arthur Chickering and Zelda Gamson developed the seven principles along with scholars who had conducted research on the impact of the college experience and on issues in higher education related to the organization, economics, and policy

5. Principles of Good Practice • Encourages student-faculty contact • Encourages cooperation among students • Encourages active learning • Gives prompt feedback • Emphasizes time on task • Respects diverse talents and ways of learning • Communicates high expectations

6. Assessment Cycle Identify outcomes Mission Purposes Improve learning Create learning opportunities Gather, interpret evidence Suskie, L. (2009). Assessing student learning: A common sense guide. San Francisco: Jossey-Bass.

7. Types of Outcomes • Operational outcomes • Learning outcomes • Program outcomes

8. Operational Outcome • Metrics that document how well the operational aspects of a program or activity are functioning, but do not document learning or overall impact of the program or activity. • Example: 200 students will participate in the Emerging Leaders Program by the end of the 2014-2015 academic year.

9. Program Outcome • A program outcome is the desired aggregate effect of a program, service, or intervention but is more specific than a goal. • 80% of all students will lead a student organization during their college career.

10. Learning Outcome Learning outcomes are statements of what is expected people will be able to value, do, or know (attitude, skills, and knowledge) as a result of participating in a learning activity (class, educational program, individual interaction, or using a resource).

11. Why Outcomes?

12. Student Learning Outcomes Options • Learning Reconsidered & CAS • LEAP - AAC&U • Tuning/Lumina

13. Efforts to Influence Student Development and Campus Climate NASPA & APCA (2004) “learning must be reconsidered – that new research, changing times, and the needs of today’s emerging generations of students require that our traditionally distinct categories of academic learning and student development be fused in an integrated, comprehensive vision of learning as a transformative process that is centered in and responsive to the whole student.” (p. 35) Keeling, R. (Ed). (January 2004). Learning reconsidered: a campus-wide focus on the Student experience. The National Association of Student Personnel Administrators and The American College Personnel Association. Retrieved from http://www.myacpa.org/pub/documents/learningreconsidered.pdf

14. Learning Reconsidered & CAS • Practical Competence • Cognitive Complexity • Humanitarianism and Civic Engagement • Interpersonal Competence • Intrapersonal Competence • Knowledge acquisition, construction, integration, and application

15. Essential Learning Outcomes toPrepare for 21st Century Challenges College Learning for the New Global Century. A Report from the National Leadership Council for Liberal Education & America’s Promise, (2007). Washington, D.C.: Association of American Colleges and Universities. AAC&U (Association of American Colleges and Universities). (2007). College learning for the New Global Century: A Report from the National Leadership Council for Liberal Education and America’s Promise. Washington, DC: AAC&U. Accessed at http://www.aacu.org/leap/documents/GlobalCentury_final.pdf

16. LEAP – AAC&U • Knowledge of Human Cultures & Physical and Natural World • Intellectual and Practical Skills • Personal and Social Responsibility • Integrative and Applied Learning

17. Tuning - Lumina • Applied Learning • Civic Learning • Intellectual Skills • Broad, Integrative Knowledge • Specialized Knowledge

18. Discussion • What learning outcome option is utilized on your campus?

19. Write Learning Outcomes • On the index card, write a learning outcome. • Give me your index card when you are done.

20. Good Outcomes • Using the index cards of outcomes, as a group develop a list of characteristics of good outcomes.

21. Developing Learning Outcomes • ABCD Formula • Audience • Behavior • Condition • Degree Heinrich, R., Molenda, M., Russell, J., & Smaldino, S. (2002). Instructional media and technologies for learning (7th ed.). Englewood Cliffs: Prentice Hall, Inc.

22. Key Elements • Audience (Who) • Who is the target of the outcome? • Behavior (What) • What should the audience be able to know, do, or value? • Condition (How) • What will facilitate the learning? • Degree (How much) • How much will be accomplished or demonstrated?

23. Developing Learning Outcomes Condition • As a result of participating in the leadership workshop, students • will demonstrate three of the five leadership criteria • as stated in Kouze’s and Posner’s The Leadership Challenge. Audience Behavior (verb in future tense) Degree

24. Developing Learning Outcomes SWiBAT Formula Learning outcome = SWiBAT + Action Verb + Condition Students will be able to differentiate between two styles of leadership as a result of attending the leadership workshop.

25. Action Verbs • Analyze • Apply • Argue • Arrange • Assemble • Calculate • Classify • Compare • Create • Criticize • Defend • Define • Describe • Develop • Differentiate • Discuss • Explain • Formulate • Implement • Manage • Operate • Outline • Translate

26. Words/Phrases to Avoid • Appreciate • Become aware of • Become familiar with • Know • Learn • Understand

27. Learning Outcome Examples • As a result of the team builder, students will remember the name of five newly introduced people. • As a result of living in the residence halls, students will describe strategies to balance freedom and responsible behavior. • As a result of participating in a student organization, students will cooperate in group activities. Adapted from http://www.nwlink.com/~Donclark/hrd/bloom.html

28. SMARRT Checklist • Specific • Measurable • Aggressive - but attainable • Results-oriented & Relevant • Time-bound University of Central Florida. (2004). Program assessment handbook. Orlando, FL. Author.

29. Revise outcomes • What types of revisions did you make? -Specific -Measureable -Aggressive, but attainable -Results-oriented -Time-bound -Revised action verb -Something else

30. Discussion • How are learning outcomes measured?

31. Discussion • How do you use the data gathered from assessing the learning outcomes?

32. Resources • Bloom’s Taxonomy – revised http://thesecondprinciple.com/teaching-essentials/beyond-bloom-cognitive-taxonomy-revised/ • Add more