1 / 20

Helping Program Assessment Using Automated Text Processing

Helping Program Assessment Using Automated Text Processing. Anne Gilman SoTL Brown Bag 25 Sept 2013. Finding questions we hadn't thought to ask. ..in what students say. Assesment Tools. The Law of the Instrument:

ananda
Download Presentation

Helping Program Assessment Using Automated Text Processing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Helping Program Assessment Using Automated Text Processing Anne Gilman SoTL Brown Bag 25 Sept 2013

  2. Finding questions we hadn't thought to ask... ..in what students say.

  3. Assesment Tools • The Law of the Instrument: “Give a small boy a hammer, and he will find that everything he encounters needs pounding.” -Abraham Kaplan

  4. Assesment Tools • Are there tools that can help us • ..to articulate goals we take for granted? • ..to point in new directions?

  5. Cool Hand LIWC

  6. Data • LiM participant essays • 57 from 2007-2012 • Holistic ratings • 23 Director’s Cut • Totals: • 9 Excellent • 11 Good • 3 Just barely makin’ it

  7. LiM Assignment In a paper of at least two pages, discuss what the Language in Motion experience has meant for YOU. Be as specific as possible and provide examples where relevant. Consider the following: A. Why did you choose to participate in Language in Motion? What personal goals, both short and long term, did you have for this experience? Were they met? If so, how well? If not, why not? B. In what areas have you improved your skills and/or knowledge? Consider what you learned about the following: • Your second language, if any, • The topic(s) of your presentations, • Your home culture, • Your second culture, • American education, • Presentation skills: Organization, ability to give clear directions, adjustment of content to audience (age and knowledge level), etc. • Yourself: Communication or cross-cultural communication abilities, time management, self-confidence, flexibility, leadership, etc. C. Was there something you wanted to learn through this experience that you didn’t? If so, what was it and why did you not learn it? Is there something we should do differently?

  8. What did LIWC say?

  9. What did liwc say?

  10. Next Steps: • Analyze by paragraph • Consider value of enjoyment • Look under the hood

  11. Can we look under the hood?

  12. Topic sorting c. 1974

  13. Self-organizing maps c. [guess!]

  14. Are these more familiar? • Hoping the XPS viewer works in our room!

  15. Sample LiMWordles

  16. What's a trigram? • Demo

  17. Results: Cluster-to-Category Match • Better: • Talking less about your own country • zzt, zto, top • TBD: • Less: tha • More: ent, eth

  18. Next Steps • Get more ratings • Add more trigrams • Compare to LiM, ILAC goals • Consider POEs

  19. Many Many Thanks • LiM Director: Dr. Deb Roney • Gilman Lab Assistants • LIWC project: Helen Hu, Aric Koestler, Olivia Moody, Sungouk Park, Seth Weil • Trigram project: Tori Rehr, Tori Buser • Dr. Kim Roth • The Juniata Department of Psychology • The SoTL Leadership & the Lakso Fund

  20. Gory Details • > aggregate(limtrunc[-1], by=list(cluster=fittrunc.km$cluster), mean) • cluster age all and ati den • 1 1 0.002255296 0.002815929 0.007260874 0.006236264 0.002169403 • 2 2 0.004112582 0.002038445 0.006445021 0.007287239 0.003207497 • 3 3 0.001498536 0.002247922 0.007557460 0.006659985 0.004155337 • ent ere erseseest • 1 0.007423606 0.003938757 0.002885631 0.003315418 0.003273650 • 2 0.010172528 0.003758447 0.001438849 0.005238279 0.003749762 • 3 0.010626493 0.004926549 0.002321635 0.004856809 0.003323920 • eth eve for. hat her • 1 0.003958753 0.002694240 0.003256768 0.005716957 0.003714643 • 2 0.002908776 0.002441420 0.002277172 0.004412824 0.002856234 • 3 0.003351811 0.002440801 0.003763736 0.002332821 0.003069785 • ing ion ztozztver • 1 0.008014828 0.005659334 0.001344568 0.001344568 0.003019872 • 2 0.008223715 0.009158820 0.001807482 0.001807482 0.001872784 • 3 0.005631146 0.006514531 0.004178863 0.004178863 0.003330204 • tertha the thitio • 1 0.002892176 0.005949131 0.01666473 0.003739603 0.005432872 • 2 0.002505902 0.004485792 0.01535570 0.003083640 0.008802554 • 3 0.002784424 0.003212007 0.01513814 0.004036677 0.006142843 • top tthntsonsopi • 1 0.002193457 0.003513021 0.002240106 0.002399789 0.001670029 • 2 0.002822616 0.002456230 0.002990221 0.002544836 0.002204109 • 3 0.005067310 0.002195498 0.003912161 0.002463628 0.004443421 • ore pia pre rea res • 1 0.002220215 0.001284905 0.003154103 0.003108067 0.004400721 • 2 0.001488311 0.001586731 0.005186367 0.002198381 0.005775334 • 3 0.002816267 0.004117230 0.004174962 0.003289315 0.005404926 • sth • 1 0.002374196 • 2 0.002569780 • 3 0.002009747

More Related