1 / 17

Using the Voluntary System of Accountability to Measure Learning Outcomes Issues and Opportunities

Using the Voluntary System of Accountability to Measure Learning Outcomes Issues and Opportunities. Christine M Keller VSA Executive Director. AASCU Academic Affairs Summer Meeting Portland Oregon ▪ July 28-31, 2011. Voluntary System of Accountability (VSA).

lobo
Download Presentation

Using the Voluntary System of Accountability to Measure Learning Outcomes Issues and Opportunities

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Using the Voluntary System of Accountability to Measure Learning Outcomes Issues and Opportunities Christine M Keller VSA Executive Director AASCU Academic Affairs Summer Meeting Portland Oregon ▪ July 28-31, 2011

  2. Voluntary System of Accountability (VSA) on the undergraduate student experience through a common web report – the College Portrait. Initiative by public 4-year universities to supply comparable, transparent information www.collegeportraits.org AASCU & APLU

  3. VSA Goals • Provide a streamlined college search tool for prospective students, families, and high school counselors • Provide a mechanism for public institutions to demonstrate transparency and accountability • Support institutions in the measurement and reporting of student learning outcomes through original research and by providing a forum for collaboration and exchange

  4. SLO Pilot Project 2007-08: Directly measure, publicly report student learning gains at institution level using a common method Skills: critical thinking, analytic reasoning, problem solving, written communication Tests: CAAP, CLA, ETS Proficiency Profile Reported: At, Above, Below Expected 2012: 4-year trial period ends 2011-12: Evaluate what works, what doesn’t work, next steps

  5. Evaluation Framework • SLO measurement and reporting will continue to be essential element of VSA • How can the section be improved to better meet the needs of participating institutions and external stakeholders within the current environment?

  6. Evaluation Activities • Interviews with key internal and external stakeholders • National Institute for Learning Outcomes Assessment (NILOA) • Focus Groups, Conference Sessions • Surveys • Advisory panels, working groups

  7. SLO Published on CP by Test • 34% or 109 College Portraits have published SLO results as of July 2011 ETS PP N=18 CLA N=80 CAAP N=11

  8. VSA SLO Survey • Short survey to gauge progress, gather feedback from VSA contacts • ~200 responses in June/July • 61% institutional research • 16% academic affairs • 13% assessment • 10% other

  9. Survey: Test Administration • 83% administered at least one test • 23% CAAP • 37% ETS PP • 51% CLA • 60% posted results on College Portrait • 92% plan to continue using one of tests

  10. Value of VSA Approach Helpful as overall indicator of student progress … powerful stimulus to other more local efforts Benchmarks to compare skill levels of our students with other campuses Good faith effort to be transparent Dovetails with institutional priorities Selected Comments VSA SLO Survey 7/5/2011

  11. Value of VSA Approach Lack of faculty buy in … small sample sizes provide limited value at operational level Testing protocol is negative because of cost (cash/staff time) with no useful results Unfortunately I don’t think many people look at this section Major limitation of VSA, please change this section … add AAC&U rubrics Selected Comments VSA SLO Survey 7/5/2011

  12. Survey: Other Assessments

  13. Next Steps • Gather feedback on efficacy of the SLO pilot study (Summer/Fall 2011) • Assemble brainstorming group to consider learning outcomes in current context (Fall 2011) • Convene technical workgroup to evaluate options (early Spring 2012) • Work with senior advisory panel and VSA Board to recommend next steps and future directions (late Spring 2012)

  14. Questions & Discussion • Questions about evaluation plan or current status of VSA/SLO • Your input on strengths, weaknesses, recommendations for future for student learning measurement and reporting within the VSA

  15. Discussion • What are the benefits of the current VSA approach to measuring/reporting SLO? • Has it been effective with external stakeholder groups? • What are some of the challenges or barriers of the current approach? • How can the current approach be expanded or modified to better address current needs, challenges, external demands, etc?

  16. More Information Websites www.voluntarysystem.org www.collegeportraits.org Christine Keller, VSA Executive Director ckeller@aplu.org Wendell Hall, VSA Assistant Director whall@aplu.org Elspeth Payne, VSA Project Coordinator epayne@aplu.org

More Related