The Library Impact Data Project

1. NAG Conference 7-8 September 2011 The Library Impact Data Project Graham Stone Information Resources Manager http://eprints.hud.ac.uk/11389/ #lidp #jiscad This work is licensed under a Creative Commons Attribution 3.0 Unported License

2. …to improve existing services …to gain insights into user behaviour …to measure the impact of the library Using Usage Data since 2005…

3. Non/Low Use Projectdigging deeper into data 3

4. Measuring Library Impact2008/9 honours graduates Analysis of the results consistently revealed a correlation between e-resource use, book borrowing and student attainment This appears to be the case across all disciplines

5. Results • Not a cause and effect relationship • Never proven statistically significant • Potential for collaboration on future projects

6. JISC Activity Data Call • Obtained funding from the JISC Activity Data Call • 6 month project (Feb-Jul 2011)

7. Library Impact Data Project

8. To prove the hypothesis that… “There is a statistically significant correlation across a number of universities between library activity data and student attainment”

9. Data requirements • For each student who graduated in a given year, the following data was required: • Final grade achieved • Number of books borrowed • Number of times e-resources were accessed • Number of times each student entered the library, e.g. via a turnstile system that requires identity card access • School/Faculty

10. Legal issues • Consultation with JISC Legal, University legal officer and data protection officer • Ensured that any identifying information is excluded before it is handled for analysis • Excluded any small courses to prevent identification of individuals e.g. where a course has less than 35 students and/or fewer than 5 of a specific degree level • Received guidance from the Using OpenURL Activity Data

11. Data issues • Anticipated that there may be problems in getting enough data to make the project viable • Potential partners were asked to confirm that they could provide at least 2 of the 3 measures of usage as well as student grades • Some partners ran into issues with data collection, but it was felt that there was still enough information to prove the hypothesis one way or another

12. Library Impact Data Projectbook loans (2009/10)

13. Library Impact Data Projectbook loans & Athens (2009/10)

14. Library Impact Data Projectlibrary PC logins & visits (2009/10)

15. Can we prove the hypothesis? • Due to the data not being continuous, a correlation cannot be calculated http://www.flickr.com/photos/jepoirrier/2043728206/

16. Further statistical tests (1) • Running a Kruskal-Wallis test • to indicate whether there is a difference between values e.g. between levels of e-resource usage across degree results • THEN we analyse the data visually to check which variables to compare

17. Further statistical tests (2) Running a the Mann-Whitney U test to see whether there is a significant difference between variables tested Matching these findings with previous tests using ANOVA and the Student T test

18. Can we prove the hypothesis? • The relationship and variance means that you can believe what you see • And you can believe it across a range of data • Subjects • Partners • So library usage does impact on students attainment Not a cause and effect relationship http://www.flickr.com/photos/princesstheater/3530252342/

19. Linking back to non/low usage • Our research shows that for books and e-resource usage, there appears to be a statistical significance across all partner libraries • If we know that there is a link between usage and attainment • We can link this back to non/low usage

20. Measuring Library Impact2008/9 – library visits 15.5% of students who gaineda 1st never visited the library 34% of students who gaineda 3rd never visited the library 20

21. Measuring Library Impact2008/9 – MetaLib usage 10.5% of students who gained a1st logged in more than 180 times 70% of those who gained a3rd logged in to e-resources20 times or less over 3 years 21

22. Measuring Library Impact2008/9 – book loans 15% of students who gaineda 1st never borrowed a book 34% of students who gaineda 3rd never borrowed a book 22

23. Scenarios If the hypothesis is correct, does cutting the library budget mean that attainment will fall? Is this something that can be used at director level to protect resource budgets/subject librarians? Can we actually demonstrate that the library adds value? E.g. if a student enters university with average UCAS points and attains a first class degree having used library resources to a high level, does this prove the library has added value to the student achievement? Have we done anything? Do they need us? If students are not using the library or the resources, what can we do to change their behaviour? Is non-use a resourcing issue or an academic/information skills issues? How could gender, culture, nationality and socio-economic background affect library usage and how could this be addressed? Are there scenarios where we should NOT try to increase library use? What further work could we do to help us understand more about student use and the impact on the processes we use to acquire material? What about print and e-Book use? Will direct links from reading lists help improve usage? What about Demand Led Acquisition?

24. Profiling non/low users • Investigate the use of reading lists • Matching attainment with use of essential, recommended and wider reading • Check the amount and type of contact subject teams have had with the specific courses • to compare library teaching hours to attainment • Baseline questionnaire or exercise for new students • To establish the level of information literacy skills for new students • Target our users by concentrating staff resources at the right point

25. Top 15 TitlesAugust to March 2010/11 • 2010/11 usage • Exploring corporate strategy 1671 • An introduction to criminological theory 333 • Teaching in lifelong learning 303 • Approaches to psychology 299 • Events management 228 • Operations management 218 • Postdramatic theatre 209 • Criminology and social policy 203 • Theoretical approaches in psychology 198 • Marketing communications 193 • Stadia 192 • Health psychology 186 • The A-Z of social research 185 • A companion to early 20th century Britain 185 • Media and crime 185 PDA selected titles in red average of 19.3 views per title

26. Usage of PDA Selected Titlescurrent academic year 2010/11 27% of titles have above average usage (more than 19.3 views) 32% unused titles

27. Usage of PDA Selected Titlescurrent academic year 2010/11 13% of titles have above average usage (more than 19.3 views) 57% unused titles

28. Usage of PDA Selected Titlescurrent academic year 2010/11 • Non-PDA titles are almost twice as likely to have not been viewed as PDA titles • PDA titles are twice as likely to have above average viewing stats than non-PDA titles

29. Acknowledgements Dave Pattern and Bryony Ramsden Phil Adams, Leo Appleton, Iain Baird, Polly Dawes, Regina Ferguson, Pia Krogh, Marie Letzgus, Dominic Marsh, Habby Matharoo, Kate Newell, Sarah Robbins, Paul Stainthorp

30. Thank you Graham Stone g.stone@hud.ac.uk @Graham_Stone This work is licensed under a Creative Commons Attribution 3.0 Unported License • http://library.hud.ac.uk/blogs/projects/lidp/ • http://eprints.hud.ac.uk/11389/