480 likes | 501 Views
Building a Journal Collection. 2-Step Map Evaluation Approach. Tohoku University Library, Kanako Katsumoto Aiko Watanabe. OUTLINE Tohoku University Questions on Research Evaluation Using Citation Metrics 2-Step Map Application to Library’s Collection Development Effectiveness and Issues.
E N D
Building a Journal Collection 2-Step Map Evaluation Approach Tohoku University Library, Kanako Katsumoto Aiko Watanabe
OUTLINE Tohoku University Questions on Research Evaluation Using Citation Metrics 2-Step Map Application to Library’s Collection Development Effectiveness and Issues
Sendai ●Tokyo Tohoku University • Sendai City, Miyagi Prefecture • National university founded in 1907 • Research-oriented university with 10 undergraduate departments and 15 graduate schools • Student Enrollment: 17,900; Number of Faculty and Staff: 5,000 • “Research First”; ”Open Door”
Sendai – “Mori no Miyako” naturevol.442, no.10, p714-716 (2006.8.10)
Library System • Main Library,4 Branch Libraries,13 Division Libraries • Collection: Approx. 3.7 million Volumes • No. of Staff: 130
Electronic Resources Environment • Site-wide license • Web of Knowledge (Web of Science, JCR, ESI) • OVID (MEDLINE, Current Contents, PsycINFO, ERIC, Biological Abstracts, CINAHL, EBMR) • … • Resources maintained by faculty research fund • SciFinder Scholar, CrossFire, INSPEC, … • Others • Scopus, PubMed, …
Paper version+ EJ Package EJ EJ Package EJ Electronic Journal Environment Paper version + Paper version+ … +EJ Which are “essential journals”?
Electronic Journal Environment What is the “fair cost burden”?
Citations and Research Evaluation • The need for research evaluation • Introduction of “Web of Science” in 2001 • Utilization as a research evaluation tool • The number of citations received by papers • Impact factor of the journalpublishing those papers Perhaps a little excessive?
Cautions for Use of Citation Metrics • Scope of journals included in the DB (Mostly European and American, Core journals only) • Field-dependent differences (Circumstances surrounding paper/citations) • The significance of citations (excessive self-citations, courtesy/critical citations) • Slippage loss of counting due to variants and journal title changes
Cautions for Use of Impact Factors • Average number of citations in the journal(not each paper) • Different citation trends by document type(Review journals) • Calculated for the past 2 years(Basic v.s. applied fields) • Differences between general and specialized journals
2-Step Map • Draw an arrow to the 2 highest citing journals (exclude self-citation) • Avoid overlapping of the arrows • Present each journal proportionate to its publication size A 1.012 Receiving many arrows = Core B 2.345 C 3.242
Advantages of 2-Step Map • Relationships between journals can be observed visually • Tendencies that are often missed with citation metrics can be recognized • Journals that receive many arrows = Core journals ≠ journals with high impact factors Narin F, Carpenter MP, Berlt NC “Interrelationships of scientific journals” Journal of the American Society for Information Science vol.23, p323-331, 1972
Materials Science, Multidisciplinary (JCR 2004) 177 Journals More than 100 papers Top 50 journals by IF Review journals not included
Materials Science, Multidisciplinary (JCR 2004) Physical Review B Journal of Applied Physics Applied Physics Letters
Materials Science, Multidisciplinary (JCR 2004) 6 7 10 3.075 8 1 2 9 13 2.255 4 5 4.308 3 4 10
Materials Science, Multidisciplinary (JCR 2004) 8 1 2 9 4 5 3 10 NANO CHEMISTRY
Materials Science, Multidisciplinary (JCR 2004) 6 7 Acta Materialia Materials Science & Engineering A Physical Review B
Materials Science, Multidisciplinary (JCR 2004) 6 7 10 3.075 8 1 2 9 13 2.255 4 5 4.308 3 4 10
Review Review Review Review Review 177 Journals More than 100 papers Top 50 journals by IF Creating a 2-Step Map
Journal Details Basic Information Categories Detailed Calculation
Journal Details founded in 2002?
Journal Details Source data for 2-Step Map! Review journal verification
Journal Details Record top 3 citing journals
User Observation Usage Statistics Surveys Impact Factor 2-Step Map? Application to Collection Development Which are “essential journals”?
Mapping Based on Subscriptions 209titles OPAC JCR
Journals subscribed at Agricultural Library (JCR 2004) Agricultural Library On campus Unpurchased Drawn by Aiko WATANABE
Journals subscribed at Agricultural Library (JCR 2004) The Journal of Biological Chemistry Proceedings of the National Academy of Sciences, USA Nature Drawn by Aiko WATANABE
BIOCHEMISTRY GEOPHYSICS MEDICINE ANIMAL CELL FOOD ANALYSIS PLANT FISHERIES AGRONOMY ECOLOGY Journals subscribed at Agricultural Library (JCR 2004) Drawn by Aiko WATANABE
Advantages of Maps Based on Subscriptions • Degree of availability of journals receiving many arrows • Gaining an understanding of the importance of a journal • Relationships between fields and balance of journals purchased
Assessment by Researchers • Gaps between the arrow concentrations and their professional perceptions • Are maps based on subscriptions limited?
Example: Fisheries Science Limnology and Oceanography Marine Ecology Progress Series
What is Causing This Gap? • Citation data for non-subscribed journals also needed? • Citation data for related fields also needed? • Comparison of this map to another map including all journals in the field
Fisheries + Marine & Freshwater Biology + Oceanography (JCR 2004) Limnology and Oceanography Drawn by Aiko WATANABE
2-Step Map of the Overall Field • Map of all journals in the field + subscription information • Facilitates the holistic picture of the field • Helps to recognize essential journals that have not been subscribed
Effectiveness of 2-Step Map • Visually recognize the trends that are difficult to be drawn by citation metrics alone • Journals with high IF values ≠ Journals that receive many arrows • Observe the relationships between journals and categories • Observe the research trends within fields • Utility in journal collection development by adding subscription information • Utility for journal evaluation supplementing IF values Researchers show high level of interest!
Issues Surrounding 2-Step Map • Limitations due to JCR • For 1-year sets of data, journals cited over a long period of time are not well presented • Journals included in the JCR are limited • Journals after title change are treated as new journals
Issues Surrounding 2-Step Map • Difficulty in Extracting Appropriate Field • Research fields becoming interdisciplinary and segmentalized • JCR Category ≠ Research focus of your institution • Complex nature of data extraction and map creation
Difficulty in Extracting Appropriate Field • Graduate School of Agricultural Science(Plant sciences) • Plant Sciences → ×(Too broad) • Agronomy → ×(Content is irrelevant) • Essential to know the research focus before starting!
Field Analysis Based on Journal Usage Survey • Journals in Use (2005) • Journals in which our faculty publishedtheir work • Journals that were cited in their work • Analysis of the use frequency and the fields of thesejournals
Extraction of Journals in Use Publication Year +Affiliation Journal that Published Paper Citation Information
43 categories Analysis of Extracted Data EJ available: 100 Not available on campus: 10 Items not indexed in JCR: 7 1634titles 123 titles
Creating a Map of Extracted Fields Used more than 10 times Used more than once REPRODUCTIVE BIOLOGY (JCR 2005) Non-subscribed journals Human Reproduction
Fisheries + Marine & Freshwater Biology + Oceanography 6 2 24 6 4 4 55 10 4 1 4 2 10 3 2 1 24 15 1 3 15 2 3 (JCR 2004) Drawn by Aiko WATANABE
Effective Application to Library CollectionDevelopment • Determine the field(s) you wish to analyze • Create a 2-Step map with all journals in the field(s) • Add subscription information • Add usage frequency information
Multiple Indicators for Collection Development • Use a variety of indicators to make multi-faceted and comprehensive evaluation • Associate with the characteristics of the research focus • Also keep in mind the characteristics of each journal • 2-Step Map • Impact Factors (2, 5, 10… years) • Statistics on Electric Journals usage (COUNTER) • Researcher surveys • …
Conclusions • Overcoming excessive reliance in impact factors • Providing researchers with accurate information • Enhancement of the JCR – for utilization of multiple indicators • Facilitating easier data extraction • IF values longer than 2 years • Refining category schemes • Handling journal title changes