Before you start . . .

1. Before you start . . . Click the space bar to continue • Go to the Assessment Site on the LSC-PA website. It’s in the pull-down menu just above the MyLamarPAlogin button, or go to http://www.lamarpa.edu/main.asp?url=/SACS/quick_link_docs/default.asp • Find and print the following documents: • “Reporting Test & Exam Data” • “Reporting Project & Essay Data” • “Reporting Pre-test/Post-test Data” • “Reporting Indirect Assessment Data” • “Artifacts and Assessment Module Quiz” • “Artifacts and Assessment Module Survey”

2. assessment is a machine that never stops Click the space bar to continue

3. After this slide, the presentation will advance slides automatically. Click the space bar to start

4. Creating & ReportingArtifacts for Assessment Lamar State College – Port Arthur March 17, 2010

5. Expected Outcomes of this Presentation • By the end of this presentation, you should be able to: • Identify at least two types of direct or indirect evidence from your courses • Report accurately at least two types of direct or indirect evidence from your courses • Plan your courses to help you generate easily-reported evidence

6. All assessment relies on YOU. Notice that at the tip of the pyramid, the things that hold up the entire system, are individual assignments in your classes.

7. Pop Quiz! True or False They can just do the program assessment without my course data. False. Every course LSC-PA offers falls within a program, and program assessment can only be accurate if everyone participates.

8. Definitions of Indirect and Direct Assessment

9. Examples of Data Used for Indirect and Direct Assessment at the Course Level

10. Examples of Data Used for Indirect and Direct Assessment at the Program Level

11. Some Costs of Indirect and Direct Assessment

12. Some Benefits of Indirect and Direct Assessment

13. We want to use more direct than indirect evidence.

14. Tests and Test Questions

15. Finding Artifacts/Evidence You probably already have multiple lines of direct evidence in the assignments and exams you currently administer. You’ll need • The Program Student Learning Outcomes for your program (on the LSC-PA Assessment Site) • Your course assignments or exams • Your test maps

16. Use your Test Map If you created a test map, matching Program Student Learning Outcome (PSLO) to individual questions is already done for you, by you. If you have not created test maps for the assignments you are using, consider creating them as you prepare for next semester.

17. New Tests As you create new tests for your classes, create maps for them as well.

18. No Map? No Problem. Okay, not much of a problem. If you don’t have a test map, you can still find evidence that can be used for program assessment. Here’s the solution:

19. Pick one PSLO or subPSLO Pick one of the PSLOs or subPSLOs for your program. Read through your exam, question by question. Mark which questions directly relate to the PSLO.

20. Make a Record • Administer the tests as normal. • Record the total number of test-takers. • Record the responses to the questions. • For Objective tests, record the number of right vs. wrong answers. • For Subjective tests, create a scoring rubric for the question.

21. Save Your Data • Save 10 student responses to the exam/exam question you used as evidence. • The 10 copies should be representative. • If it’s an essay or short answer question, scored with a rubric, keep some with high scores, some with middle scores, some with low scores (and the rubrics). • If it’s an objective test, save half with right answers and half with wrong answers.

22. Save and Report Your Data On the Assessment website, you will find a form, “Reporting Test & Exam Data” to download, fill out, and save to your computer. Save the student responses to your computer. You may need a secretary or print shop operator to scan the documents for you. Report your data to the assessment coordinator for your program when requested. Please do not email unsolicited data.

23. Plan for the Future We will all be reporting this kind of data from now on, so try to let this record keeping become part of your regular routine. Plan to embed questions by placing newly created questions, designed to measure certain knowledge or skills, into existing exams.

24. Things to Remember Not every question on your existing tests will necessarily measure every PSLO. Pick questions that clearly measure the PSLO. Plan ahead and create new questions to embed into your existing tests.

25. And… The best tests or test questions to use for gleaning data are from comprehensive final exams. If your program has a CAPSTONE course with a comprehensive exam, you’ve struck the mother lode.

26. Pop Quiz! True or False • To report accurately, I have to create new questions for my tests. False. Your existing tests most likely have perfectly good questions that can be used to measure PSLOs. • I should send all results to David immediately after compiling them. False. Save the documents until they are requested. At that point, you will send them to the program assessment coordinator.

27. Reports, Projects, Essays, Presentations, Productions

28. Finding Artifacts/Evidence You probably already have multiple lines of direct evidence in the reports, projects, essays and essay questions, presentations, and productions you already assign. You’ll need • The Program Student Learning Outcomes for your program (on the LSC-PA Assessment Site). • Your course assignments or exams. • Your project maps.

29. Use your Project Map If you created a project map, matching Program Student Learning Outcome (PSLO) to discreet parts of your project is already done for you, by you. If you have not created project maps for the assignments you are using, consider creating them as you prepare for next semester.

30. New Reports, Projects, Essays, Presentations, and Productions As you create new reports, projects, essays, presentations, or productions for your classes, create maps for them as well.

31. No Map? No Problem. Okay, not much of a problem. If you don’t have a project map, you can still find evidence that can be used for program assessment. Here’s the solution:

32. Pick one. Pick one of the PSLOs or subPSLOs for your program. Read through your project, element by element. Identify which project elements directly relate to the PSLO.

33. Make a Record • Record which elements you will monitor on your project. • Administer the projects as normal. • Record the total number of project completers. • Record the responses to the project elements. • You should probably score the project with some sort of rubric specific to the PSLO/subPSLO.

34. Save Your Data • Save your rubrics with scoring on them. • Save 10 copies of the element of the project you used as evidence. • Mark the part of the project that you have determined measures the PSLO/subPSLO. • The 10 copies should be representative. • Keep some with high scores, some with middle scores, some with low scores.

35. Save and Report Your Data On the Assessment website, you will find a form, “Reporting Project and Essay Data” to download, fill out, and save to your computer. Save the student responses and your completed rubrics to your computer. You may need a secretary or print shop operator to scan the documentsfor you. Report your data to the assessment coordinator for your program when requested. Please do not email unsolicited data.

36. Plan for the Future We will all be reporting this kind of data from now on, so try to let this record keeping become part of your regular routine. Plan to embed measurement tools by determining what PSLOs or subPSLOs are measured by each element of your new projects.

37. Things to Remember Not every element of your existing projects will necessarily measure every PSLO or subPSLO. Pick elements that clearly measure a specific PSLO or subPSLO. Save your scoring rubrics and copies of the element being assessed.

38. Also… The best reports, projects, essays, essay questions, presentations, and productions from which to draw your assessment are those that represent a culmination of course information – a final project, a final essay exam or essay question, a final presentation.

39. Pop Quiz! True or False • Projects that are subjective, or projects that have multiple elements, should be scored by using a rubric. True. Scoring with a rubric helps to make more concrete the assessment of material that otherwise seems subjective. • I should send all results to David immediately after compiling them. False. Save the documents until they are requested. At that point, you will send them to the program assessment coordinator.

40. Pre-Tests/Post Tests

41. Value-Added is “any method of analyzing student test data to ascertain students’ growth in learning by comparing students’ current level of learning to their own past learning.” (Evergreen Freedom Foundation) One significant way of measuring value-added is to administer pre-tests and post-tests.

42. In a Nutshell Basically, pre-tests and post-tests test the same information at different times of the semester. A pre-test is usually administered in the early part of the semester, and a post-test is typically given at the end of the semester.

43. How do I do this pre-testing and post-testing? Use questions or elements of projects that measure specific PSLOs/subPSLOs. Select the questions or elements of projects from tests or projects you will administer at the end of the semester, preferably the final exam. Administer the same questions or elements of a project to the students at the beginning of the semester, ideally before you have started teaching the material.

44. Track Student Response Record student scores on the pre-test in your grade book or wherever you keep your data. Record student scores on the post-test in your grade book or wherever you keep your data.

45. Do a Little ‘Rithmetic Subtract the pre-test score from the post-test score. A positive difference indicates a gain in added value, whereas a negative difference suggests no gain. Post-test - Pre-test = Value added 86 - 54 = 32 good 72 - 85 = -13 not good

46. But I Used a Rubric! The same principle works if you used a rubric for scoring a subjective project or numbers to grade an objective test. Post-test (rubric) - Pre-test (rubric) = Value Added 3 - 3 = no value added not good 3 - 2 = + 1 value added good Convert your rubric score into your regular grading scale as needed.

47. Pop Quiz! True or False I can assume my students don’t know my content at the beginning of the class, so I can administer just a post-test and get similar results. False. We never know what knowledge students bring with them the first day. Also, it’s impossible to measure any gain without a baseline score, which the pretest provides.

48. Save and Report Your Data On the Assessment website, you will find a form, “Reporting Pre-test/Post-test Data” to download, fill out, and save to your computer. Save the student responses and your completed rubrics to your computer. You may need a secretary or print shop operator to scan the documentsfor you. Report your data to the assessment coordinator for your program when requested. Please do not email unsolicited data.

49. Portfolios

50. What’s a Portfolio? A student portfolio is a systematic collection of student work and related material that depicts a student’s activities, accomplishments, and achievements in one or more school subjects. The collection should include evidence of student reflection and self-evaluation, guidelines for selecting the portfolio contents, and criteria for judging the quality of the work. The goal is to help students assemble portfolios that illustrate their talents, represent their […] capabilities, and tell their stories of school achievement. (Venn, 2000)