Incorporating IDEA into Assessment of General Education Learning Outcomes

1. Incorporating IDEA into Assessment of General Education Learning Outcomes Stephanie Oetting, PhD Director of Institutional Research University of Saint Francis Fort Wayne, IN

2. General Aims • Incorporating IDEA student ratings into general education assessment • Using IDEA extra question process to collect data for general education assessment • USF GE Assessment Model • Mix of direct/indirect and internal/external measures • Custom reports versus aggregate data files • Using extra questions in many classes

3. Background • General education curriculum in place 20+ years • Intense work for revision initiated in 2003 • New GE Framework approved late Spring 2007 • Assessment plan work successfully avoided • Faculty Governance Structure • Committee On Assessment Student Academic Achievement • General Education Committee • Academic Council

4. General Education Goals • Develop personal awareness of diverse/global society. • Demonstrate leadership, service, social responsibility. • Communicate effectively in personal/professional interactions. • Demonstrate competence in applying technologies. • Demonstrate personal and social awareness of importance of lifelong health/wellness. • Think analytically, synthetically, critically, and creatively in the pursuit of knowledge. • Develop awareness/understanding of artistic/expressive aspects of the human experience. • Appreciate spiritual dimension of life, be conscious of own religious perspective within a community context.

5. Layers of Complexity 8 General Education Goals operationalized with 31 Learning Outcomes grouped into 19 Distribution Requirements

6. Challenges: Several significant administrative and structural changes New GE framework lacking assessment plan Exhaustion from GE framework revision No Clear leadership, accountability, authority for GE assessment and Committees searching for a responsible party! Strengths: Program assessment well-developed in some programs Emerging group of faculty knowledgeable about, invested in assessment Support, encouragement from University AQIP Systems Portfolio in progress Assessment “Opinion Leaders” vocalizing State of Assessment

7. Getting Started… “You don’t have to see the whole staircase, just take the first step.” --Martin Luther King, Jr.

8. Task Force Convened • Composition • Goals • BEGIN draft of assessment plan • Reduce fear and avoidance • Identify critical resources for implementation • Cultivate culture of assessment • Compensation & reward

9. Requirements of Assessment Plan! • Simple, seamless, sustainable • Produce meaningful data, quickly and efficiently • Use what’s already in place, no extra time or work! • Benefit students whenever possible • Empower instructors/dept. to generate artifacts • Need naturally occurring artifacts, not extra work • Compensate faculty who give their time! • No evaluation of faculty, course or program • No policing or punitive approach

10. IF ONLY…

11. Comforting thoughts… “It doesn’t really matter whether you can quantify your results. What matters is that you rigorously assemble evidence – quantitative or qualitative – to track your progress.” “What matters is not finding the perfect indicator, but settling upon a consistent and intelligent method of assessing your output results, and then tracking your trajectory with rigor.” Collins, J. (2005). Good to Great and the Social Sectors: A Monograph…(p. 7, 8)

12. GE Assessment Plan • Direct measures • Student demonstration of learning outcomes • Evaluated by faculty • Indirect measures • Student perceptions • Inferred measurements • Internal measures • Locally developed • External measures • Standardized assessments • Results comparisons

13. Indirect Measures • External • Student self-reported progress on IDEA objective that aligns with goal • Noel Levitz SSI • Internal • Student self-reported progress on learning outcome using IDEA extra question format • Other surveys and assessments

14. Align Goals to IDEA Objectives • Develop a personal awareness of our diverse and global society • Developing a clearer understanding of, and commitment to, personal values (#10) • Demonstrate leadership, service, and social responsibility • Acquiring skills in working with others as a member of a team (#5)

15. Direct Measure • Internal • Artifacts generated at course level • Assessed by faculty committee • Holistic rubric using learning outcome • Allows for comparison of faculty and student ratings

16. Goal 1 Develop a personal awareness of our diverse and global society LO3 LO1 IDEA Extra Q IDEA Extra Q Student satisfaction Noel Levitz SSI #62: Commitment to racial harmony on campus Artifact: iConnect & Society, Natural Environment Artifact: History MLK Survey Artifact: iConnect IDEA Extra Q Progress on IDEA objective #10: Understanding of & commitment to personal values LO2 = Direct Measure = Indirect Measure University of Saint Francis GE Assessment Model

17. 4-year Assessment Cycle • 2007-2008: Goals 1 and 2 • 2008-2009: Goals 3 and 4 • 2009-2010: Goals 5 and 6 • 2010-2011: Goals 7 and 8 • 2011-2012: Goals 1 and 2 • . . .

18. Preliminary Results • IDEA Objectives • Percent selecting as important/essential • Progress on relevant objectives • Learning Outcomes • IDEA Extra Questions – student perception • Artifact assessment – faculty assessment

19. Results - Learning Outcomes

20. iConnect Course Summary Ratings • Overall ratings for teacher and for course • Student comments examined • Faculty feedback sought

21. Actionable Results • Lessons Learned • New Directions

22. QUESTIONS? THOUGHTS? COMMENTS?