1 / 84

Strategies, Tips, and Tools for Facilitating Learning Outcomes Assessment in Student Services

Student Learning Outcomes. Strategies, Tips, and Tools for Facilitating Learning Outcomes Assessment in Student Services. Jerry Rudmann, Irvine Valley College February 2008. Overview - Student Services. Fine-tuning assessment Tips for writing survey items Focus groups

demetriat
Download Presentation

Strategies, Tips, and Tools for Facilitating Learning Outcomes Assessment in Student Services

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Student Learning Outcomes Strategies, Tips, and Tools for Facilitating Learning Outcomes Assessment in Student Services Jerry Rudmann, Irvine Valley CollegeFebruary 2008

  2. Overview - Student Services • Fine-tuning assessment • Tips for writing survey items • Focus groups • Helpful technology tools • Clickers - promote active learning and record SLO information • Rubric generators - a way to measure most anything • PDF Acrobat forms - autoscoring and recording student input • Portfolios - making students responsible and reflective • Scanning - some ideas • Tracking software - organizing all this stuff • Several options / strategies for making SLOs meaningful • SSO versus SLO • Problem focus • Less is better • Use what you already have • Think of SLOs in the context of student development • Qualitative assessment in OK • Other…?

  3. Some Options / Strategies for Making SLOs Meaningful • Address “robust” SLOs (overarching outcomes) • Problem focus • Less is better • Share SLOs with students • Use what you already have • Think of SLOs in the context of student development • Qualitative assessment is OK • SSOs vs. SLOs…

  4. General Tip 1: Problem Focus Approach • What competencies do students have difficulty mastering? • Focus SLO activities on problem areas.

  5. General Tip 2: Keep It Simple But Meaningful • Corollary - Often, less is better.

  6. General Tip 3: Student Development Approach • Student development • Academic self-efficacy (Bandura) • Academic self-regulation • Campus involvement (Astin) • Mentoring professor studies • Student Services DO help student success

  7. Surveys

  8. Students self-rate their competencies on program or college level learning outcomes. Students’ satisfaction with various student services. Surveys - SLO Uses

  9. Types of Questions Open-ended – respondents answer in own words Closed-ended – respondent limited to a finite range of choices

  10. Types of Questions Open-ended Flexible Hard to code answers Good for preliminary work to finalize a survey Closed-ended Easier to code answers, process and analyze Hard to write good closed-ended items

  11. Item Format Visual Analogue Scale Food in the cafeteria is… Poor_ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _Excellent Likert Scale Food in the cafeteria isoutstanding! SDDN A SA (Strongly Agree) (Disagree) (Neutral) (Agree) (Strongly Agree)

  12. Nine tips for designing and deploying a survey Don’t call it a survey Provide a carefully worded rationale or justification at the beginning Group items by common format Start with more interesting items Put demographic items last Mix in negative wording to catch acquiescence (aka “response set”) Automate scoring when possible If asking for sensitive information, use procedures designed to assure anonymity Always, always, always pilot test first

  13. Survey Administration Methods Face to Face Written Group administration Mail Computerized http://research.ccc.cccd.edu Password protected Validation rules Branching and piping Telephone

  14. Focus Groups Focus groups can be especially insightful and helpful for program and institutional level learning outcome assessment. Have your college researcher provide some background materials. Focus Groups: A Practical Guide for Applied Research By Richard A. Krueger, Mary Anne Casey The RP Group sponsored several “drive in” workshops over the last few years.

  15. Goal for This Section • Technology Uses • Technology ToolsExpected Outcome: Be able to select and use technology-based approaches to assess student learning outcomes

  16. Assessment Challenges • Engaging Students in Self-Evaluation • More Efficient Assessment

  17. Some Technology Tools • Online Rubric Builders • eLumen (SLO Assessment/Tracking) • Classroom Responders (“Clickers”) • Scannable and Online Tests and Surveys • ePortfolios • Adobe Acrobat Forms • Excel Spreadsheets

  18. Rubrics • Way to measure the heretofore immeasurable: products and performances. • A rubric breaks the assessment into important components. • Each component rated along a scale well-labeled scale.

  19. Let’s Develop an Assessment Rubric for a Resume

  20. Chocolate Chip Cookie Rubric Chocolate Chip Cookie Rubric

  21. Rubrics are Good! • Facilitate staff dialogue regarding satisfactory performance. • Create a more objective assessment. • Make expectations more explicit to the student. • Encourage metacognitive skill of self-monitoring own learning. • Facilitate scoring and reporting of data.

  22. Online Discussion Rubric http://www.uas.alaska.edu/sitka/IDC/resources/onlineDiscussionRubric.pdf

  23. Design Your Own Rubric • Please work in groups and use the worksheet in your packet to design a scoring rubric for assessing one of the following: • Coffee shops • Syllabi • Customer service at retail stores • Grocery stores • Online courses

  24. Online Rubric Builders • Rubrics to guide and measure learning • Tools • Rubistar http://rubistar.4teachers.org • Landmark Rubric Machine http://landmark-project.com/rubric_builder

  25. Rubistar Art History Rubric Rubistar

  26. Rubric Builder Screen Shot

  27. Adobe Acrobat Forms • Make form using MS Word • Import form and save as PDF form • Adjust the fields • Add fields to tally sub scores and total scores

  28. How Do You Report Results?

  29. eLumen to Assess SLOs • Reduce Time Spent Creating Reports • Assess Course, Program, and/or Degree-Level Outcomes • Share Assessment Rubrics Across Classes and Programs • View Individual or Aggregated Results • Use Online or Offline http://www.elumen.info

  30. Use Online or Offline

  31. Criterion-Based Assessment • Rubrics are attached to each SLO Excerpted from eLumen: A Brief Introduction by David Shupe, July 2007

  32. Rubrics Describe Criteria • Writes prose clearly Excerpted from eLumen: A Brief Introduction by David Shupe, July 2007

  33. Library of Degree-Level SLOs Excerpted from eLumen: A Brief Introduction by David Shupe, July 2007

  34. And Rubrics Link to SLOs Excerpted from eLumen: A Brief Introduction by David Shupe, July 2007

  35. from the Science committee from the Biology Department from the faculty committee on critical thinking from the faculty committee on communication skills Science and Gen Ed SLOs/Rubrics Excerpted from eLumen: A Brief Introduction by David Shupe, July 2007

  36. Scorecard for All Students in the Course Excerpted from eLumen: A Brief Introduction by David Shupe, July 2007

  37. Class Scores by Student Excerpted from eLumen: A Brief Introduction by David Shupe, July 2007

  38. Aggregated Data for Course Excerpted from eLumen: A Brief Introduction by David Shupe, July 2007

  39. Course Aggregates by Program Excerpted from eLumen: A Brief Introduction by David Shupe, July 2007

  40. Classroom Responders • Engage students • Monitor student understanding • Quickly and easily collect and store assessment data • Use publisher item banks or create your own

  41. Renaissance Classroom Response System PBS Demo

  42. Most valuable tip is… • Finding ways to use technology to make SLO and SSO assessment easier and more efficient • Concentrating SLO work on skills students have difficulty mastering • Building SLOs around student development (self-efficacy, goal clarity, etc.)

  43. Renaissance Learningfor clicker training resources http://www.renlearn.com

  44. Scanning Technology • A way to gather survey input from students • A way to test students’ knowledge http://www.scantron.com and http://www.renlearn.com

  45. Surveys and Tests • Online or Scannable • Surveys • Pre and post surveys of student self evaluation of progress • Gather stakeholder (faculty, business community leaders, advisory groups) input on expected learning outcomes • Student satisfaction with service (SSO) • Quizzes/Tests • Practice and graded

  46. Some Survey Software Options • Scannable surveys and quizzes - Optical Mark Reader by Remark (OMR Remark) • Need software and a Fujitsu scanner • Use word processor to create scannable bubble-in surveys or answer sheets. • Produces item analysis output. • Online survey tools • eListen (Scantron Co.) • SelectSurvey.net http://www.classapps.com/SelectSurveyNETOverview.asp

  47. Excel Spreadsheets Example of autoscoring and record keeping in a Japanese Program.

More Related