1 / 39

Assessing our Assessment: Failures and Successes at UT-Austin

Assessing our Assessment: Failures and Successes at UT-Austin. AJ Johnson, Michele Ostrow, Meghan Sitar Library Instruction Services University of Texas Libraries. FAILURE 1 Didn’t ask for help at the beginning. http://www.flickr.com/photos/inju/246717376/.

jaxon
Download Presentation

Assessing our Assessment: Failures and Successes at UT-Austin

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Assessing our Assessment: Failures and Successes at UT-Austin AJ Johnson, Michele Ostrow, Meghan Sitar Library Instruction Services University of Texas Libraries

  2. FAILURE 1Didn’t ask for help at the beginning http://www.flickr.com/photos/inju/246717376/

  3. FAILURE 2Didn’t recognize the difference between faculty and librarian culture http://fc04.deviantart.com/fs16/f/2007/160/7/f/WilliamSnape2_by_WilliamSnape.jpg http://www.hermionegrangerswand.com/

  4. FAILURE 3Didn’t anticipate low response rates www.houseofzing.com/book/images/ignore.jpg http://www.houseofzing.com/book/images/ignore.jpg

  5. FAILURE 4Didn’t consider how faculty view librarians http://www.flickr.com/photos/kaishin/1660238046/ http://www.flickr.com/photos/bibliona/238979054/

  6. FAILURE 5 Didn’t hire an army of instruction librarians and statisticians "United States Army, The." Online Photograph. Encyclopædia Britannica Online. 22 May 2008  <http://www.britannica.com/eb/art-82331>.

  7. FAILURE 6Didn’t pre-test

  8. FAILURE 7Still didn’t pre-test

  9. FAILURE 7Still didn’t pre-test

  10. FAILURE 8 Designed our post-test around faculty characteristics

  11. FAILURE 8 Designed our post-test around faculty characteristics

  12. FAILURE 9Didn’t match pre-tests to post-tests

  13. FAILURE 9Didn’t match pre-tests to post-tests

  14. FAILURE 10 Project design didn’t reflect flexibility of our instruction programs

  15. FAILURE 10 Project design didn’t reflect flexibility of our instruction programs

  16. FAILURE 11 Didn’t define a control group “No controle (in control)” by renatotarga on Flickr, http://www.flickr.com/photos/rtarga/223776150/

  17. FAILURE 12 Thought writing questions would be the easy part How would you . . . What three things would you . . . If you had to choose . . . Why does the . . . ARGH!

  18. FAILURE 13Didn’t base questions on learning outcomes

  19. FAILURE 13Didn’t base questions on learning outcomes OLD: Question - You found an article in a database but it isn’t full-text. How do you find the full text of the article somewhere else? (Circle all that apply) • Search the Library Catalog. Use the article title as your search terms. • Search the Library Catalog. Use the title of the journal, magazine or newspaper as your search terms. • Follow the “FIND IT UT” link

  20. FAILURE 13Didn’t base questions on learning outcomes NEW:Learning Outcome - Students will be able to determine where scholarly material will be found. Question - You are doing research on border security along the U.S. Mexico border. Choose the BEST place to find a research study by a political science professor. Library article database www.google.com Published encyclopedia Don’t know

  21. FAILURE 14 Forced to use multiple choice tool to measure critical thinking skills

  22. FAILURE 14 Forced to use multiple choice tool to measure critical thinking skills • Old: • When evaluating web sites to see if you should use them in your research, which factors should you consider? (Choose all that apply) • Site was on the first page of results in Google • Author expertise • Page design • Bias • Don’t know

  23. FAILURE 14 Forced to use multiple choice tool to measure critical thinking skills Newer: You find a web page about media mergers and know nothing about the author. What would be your best clue as to whether the page was a reliable source to use for your paper? • date the page was last updated • the URL (http://www...) • the quality of the page’s graphic design • the advertisements on the page • don’t know

  24. FAILURE 14 Forced to use multiple choice tool to measure critical thinking skills Newest: You are doing research on the effect of illegal immigration on border violence. You find a website from an organization called The Illegal Immigration Research Coalition, which includes statistics on incidents of border violence. List at least three criteria you would use to evaluate this site for bias and credibility.

  25. FAILURE 15Created questions with more than one right answer 1. Google searches will find information from library databases True False Don’t know 2. Choose the BEST place to find a recent report from a media watchdog organization library article database www.google.com published encyclopedia don't know

  26. FAILURE 16Created trick questions You are doing Internet research about border security. Choose the domain of a website most likely to contain a U.S. presidential candidate's stance on immigration issues .org .edu .gov .com don't know

  27. FAILURE 17Questions didn’t mean the same thing to students as they did to us

  28. FAILURE 18Didn’t define success before we started http://www.easywebcreator.com/applications/WebBlogManager/inc_webblogmanager.asp?CategoryID=227&ItemID=1061&action=refer

  29. FAILURE 19Didn’t identify all the variables http://www.flickr.com/photos/jdawg/172131301/

  30. FAILURE 20Can’t really use the data

  31. FAILURE 20Can’t really use the data

  32. FAILURE 20 Can’t really use the data

  33. Group ActivityWriting Assessment Questions Tied to Student Learning Outcomes

  34. Student Learning Outcomes 1. Students will be able to explore general information sources to increase familiarity with a topic.

  35. Student Learning Outcomes 2. Students will be able to identify the purpose and audience of popular and scholarly resources.

  36. Student Learning Outcomes 3. Students will be able to differentiate between the types of sources cited and understand the elements of a citation.

  37. Student Learning Outcomes 4. Students will be able to differentiate between searching by keyword and by subject.

  38. Student Learning Outcomes 5. Students will be able to distinguish between the free Web and online, fee-based library resources.

  39. Presentation and Questions Created by Participants available at: http://www.lib.utexas.edu/services/instruction/ostrow.html

More Related