New developments in electronic publishing and bibliometrics

1. New developments in electronic publishing and bibliometrics Henk F. Moed CWTS, Leiden University, Netherlands Elsevier, Amsterdam, Netherlands

2. Contents

3. Contents

4. Journal impact measures are no good predictors of an individual paper’s actual citation impact Partly based on International Mathematical Union’s Report ‘Citation Statistics’ (2008)

5. Length boys vs. adults

6. Citations to P-AMS vs. T-AMS

7. Normal vs. skewed distributions

8. What is the probability that ....... Almost zero a randomly selected PAMS paper is cited at least as often as a randomly selected TAMS paper? 62 %

9. Probabilities still substantial for high JIF journals

10. ‘Free’ citations

11. Thomson/JCR Journal Impact Factor Citations to all docs # Citable docs

12. Citable vs. non-citable docs

13. The problem of “free” citations - 1 Cites + + + + + Docs + + + + +

14. The problem of “free” citations - 2 “Free” Citations Cites + + + + + Docs + +

15. All three publication lists have a Hirsch Index of 5 Author 1 Author 2 Author 3 30 P1 10 P2 8 P3 6 P4 5 P5 1 P6 0 P7 30 P1 10 P2 8 P3 6 P4 5 P5 4 P6 4 P7 4 P8 4 P9 100 P1 70 P2 8 P3 6 P4 5 P5 1 P6 0 P7 1 2 3 4 5 6 7 8 9 H=? H=? H=? 5 5 5

16. Different bibliometric distributions have the same H-Index

17. Indicators are becoming more informative

18. Contents

19. University ranking positions are primarily marketing tools, not research management tools

20. Research assessment methodologies must take into account… [EC AUBR Expert Group] • Inclusive definition of research / output • Different types of research and its impacts • Differences among research fields • Type and mission of institution • Proper units of assessment • Policy context, purpose and user needs • The European dimension • Need to be valid, fair and practically feasible

21. Types of outputs (SSH)

22. Top-down institutional analysis Select an institution’s papers using author addresses (incl. verification) Categorize articles into research fields Calculate indicators Compare with benchmarks

23. Bottom-up institutional analysis (CWTS) Compile a list of researchers Compile a list of publications per researcher (incl. verification) Aggregate researchers into groups, departments, fields, etc. Calculate indicators; compare with benchmarks

24. Secondary analyses of ‘ranking’ data are informative

25. Contents

26. Case study: A national Research Council • Proposals evaluated by committees covering a discipline • Reports from external referees • Committee members can be applicants

27. Affinity applicants – Committee 0 Applicants are/were not member of any Committee • Co-applicant is/was member of a Committee, but notof the one evaluating • Firstapplicant is/was member of a Committee, but not of the one evaluating • Co-applicant is member of the Committee(s) evaluating the proposal • Firstapplicant is member of the Committee(s) evaluating the proposal

28. For 15 % of applications an applicant is a member of the evaluating Committee (Affinity=3, 4)

29. Probability to be granted increases with increasing affinity applicants-Committee

30. Logistic regression analysis:Affinity Applicant-Committee has a significant effect upon the probability to be granted MAXIMUM-LIKELIHOOD ANALYSIS-OF-VARIANCE TABLE (N=2,499) Source DF Chi-Square Prob ------------------------------------------------------------- INTERCEPT 1 18.47 0.0000 Publ Impact applicant 3 26.97 0.0000 ** Rel transdisc impact applicant 1 0.29 0.5926 Affinity applicant-Committee2 112.50 0.0000 ** Sum requested 1 45.47 0.0000 ** Institution applicant 4 25.94 0.0000 ** LIKELIHOOD RATIO 199 230.23 0.0638

31. The future of research assessment exercises lies in the intelligent combination of metrics and peer review

32. Contents

33. Effects of editorial self-citations upon journal impact factors[Reedijk & Moed, J. Doc., 2008] Editorial self-citations: A journal editor cites in his editorials papers published in his own journal Focus on ‘consequences’ rather than ‘motives’

34. Case: ISI/JCR Impact Factor of a Gerontology Journal (published in the journal itself)

35. Decomposition of the IF of a Gerontology journal Editorial self citations Free citations

36. One can identify and correct for the following types of strategic editorial behavior Publish ‘non-citable’ items Publish more reviews Publish ‘top’ papers in January Publish ‘topical’ papers (with high short term impact) Cite your journal in your own editorials Excessive journal self-citing

37. Contents

38. Journal articles Not deposited in OA rep. (no) Deposited in OA rep (o) ?><= Average Impact (CPPo) Average Impact (CPPno)

39. Three effects[Kurtz et al., 2005]

40. ArXiv, Cond Mat Phys [Moed, JASIST 2007] ArXiv papers appear earlier EarlyView Effect 100 Quality Bias: Better authors use ArXiv

41. Age distribution of citations to Arxiv and non-ArXiv papers Move curve by 6 months to the right

42. Early view effect: Citations to papers deposited in ArXiv-CM start about 6 months earlier

43. More research questions • Early view effect also visible in a non-OA environment? • Citation impact measured in biased sample?

44. Contents

45. Downloads vs. Citations More downloads more citations or More citations more downloads?

46. Relation between citations and internet hits for 153 papers in volume 318 of the BMJ (1999)Figure 1 from: Perneger, TV. BMJ. 2004, 329 (7465): 546–547. Relation between online “hit counts” and subsequent citations: prospective study of research papers in the BMJ

47. Analogy Model

48. Age distribution downloads vs. citations[Tetrahedron Lett, ScienceDirect; Moed, JASIST, 2005] Downloads % Citations Age (months)

49. Ageing downloads vs. citations: Two factor vs. single factor model Downloads % Citations Age (months)

50. Citations lead to downloads[Moed, J. Am Soc Inf Sci Techn, 2005] Paper B published; it cites A Paper C published; it cites A and B Paper A published Download of A increases