Self-citation and its impact on research evaluation: Literature review. Part II
https://doi.org/10.33186/1027-3689-2022-3-85-104
Abstract
This review summarizes papers which analyze the impact of self-citation on research evaluation. We introduce a generalized definition of self-citation and its variants: author, institutional, country, journal, discipline, and publisher selfcitation. Formulae of the basic self-citation measures are given, namely self-citing and self-cited rates. World literature on author, institutional, country, and journal self-citation is studied in more detail. Current views on the role and impact of self-citation are compiled and analyzed. It is found that there is a general consensus on some points: a) excessive self-citation and its total absence are both seen as pathological; b) self-citation has low impact on large research entities but may be critical for the analysis of individual researchers; c) share of self-citations is generally higher for entities with poor bibliometric performance, while top scientists, institutions, journals receive the majority of their citations from outside. This review also considers how bibliometric tools and databases respond to the challenge of possible manipulation by self-citations and how some bibliometric indicators are adjusted by them. The first part of the review presented here deals with the fundamental terms and definitions, and the most discussed and studied type of the self-citation, author self-citation.
This second and final part of the review considers institutional, country and journal self-citation. It also examines new bibliometric indicators which adjust for self-citation.
Supporting agencies: The paper was funded by RFBR, project number 20-111-50209.
About the Author
V. V. Pislyakov
Higher School of Economics National Research University
Russian Federation
Russian Federation
Vladimir V. Pislyakov – Cand. Sc. (Physics & Mathematics), Deputy Director, Library, Higher School of Economics National Research University; Editorial Board Member, Journal of Informetrics
Moscow
References
1. Pislyakov V. V. Samotsitirovanie i ego vliyanie na otsenku nauchnoy deyatelnosti: obzor literatury. Chast 1 // Nauch. i tehn. b-ki. 2022. № 2. S. 49–70.
2. Garfield E., Welljams-Dorof A. Citation data: Their use as quantitative indicators for science and technology evaluation and policy-making // Science and Public Policy. 1992. Vol. 19 (5). P. 321–327. URL: https://doi.org/10.1093/spp/19.5.321.
3. Hendrix D. Institutional self-citation rates: A three year study of universities in the United States // Scientometrics. 2009. Vol. 81 (2). P. 321–331. URL: https://doi.org/10.1007/s11192-008-2160-2.
4. van Raan A. F. J. Bibliometric statistical properties of the 100 largest European research universities: Prevalent scaling rules in the science system // Journal of the American Society for Information Science and Technology. 2008. Vol. 59 (3). P. 461–475. URL: https://doi.org/10.1002/asi.20761.
5. Moed H. F., Burger W. J. M., Frankfort J. G., Van Raan A. F. J. The use of bibliometric data for the measurement of university research performance // Research Policy. 1985. Vol. 14 (3). P. 131–149. URL: https://doi.org/10.1016/0048-7333(85)90012-5.
6. Moed H. F., Burger W. J. M., Frankfort J. G., van Raan A. F. J. A comparative study of bibliometric past performance analysis and peer judgement // Scientometrics. 1985. Vol. 8 (3–4). P. 149–159. URL: https://doi.org/10.1007/BF02016933.
7. Shehatta I., Al-Rubaish A. M. Impact of country self-citations on bibliometric indicators and ranking of most productive countries // Scientometrics. 2019. Vol. 120 (2). P. 775–791. URL: https://doi.org/10.1007/s11192-019-03139-3.
8. Bakare V., Lewison G. Country over-citation ratios // Scientometrics. Vol. 113 (2). P. 1199–1207. URL: https://doi.org/10.1007/s11192-017-2490-z.
9. Larivière V., Gong K., Sugimoto C. R. Citations strength begins at home // Nature. 2018. Vol. 564 (7735). P. S70–S71. URL: https://doi.org/10.1038/d41586-018-07695-1.
10. Khelfaoui M., Larrègue J., Larivière V., Gingras Y. Measuring national selfreferencing patterns of major science producers // Scientometrics. 2020. Vol. 123 (2). P. 979–996. URL: https://doi.org/10.1007/s11192-020-03381-0.
11. Baccini A., De Nicolao G., Petrovich E. Citation gaming induced by bibliometric evaluation: A country-level comparative analysis // PLoS ONE. 2019. Vol. 14 (9). Art. no. e0221212. URL: https://doi.org/10.1371/journal.pone.0221212.
12. Garfield E., Sher I. H. New factors in the evaluation of scientific literature through citation indexing // American Documentation. 1963. Vol. 14 (3). P. 195–201. URL: https://doi.org/10.1002/asi.5090140304.
13. Garfield E. Citation analysis as a tool in journal evaluation // Science. 1972. Vol. 178 (4060). P. 471–479. URL: https://doi.org/10.1126/science.178.4060.471.
14. Nisonger T. E. Use of the Journal Citation Reports for Serials Management in Research Libraries: An Investigation of the Effect of Self-Citation on Journal Rankings in Library and Information Science and Genetics // College and Research Libraries. 2000. Vol. 61 (3). P. 263–275. URL: https://doi.org/10.5860/crl.61.3.263.
15. McVeigh M. E. Journal self-citation in the Journal Citation Reports–Science Edition (2002). [Philadelphia]. 2004. URL: https://clarivate.com/webofsciencegroup/essays/journal-self-citation-jcr/ (data obrashcheniya: 31.05.2021).
16. Huang M.-H., Lin, W.-Y.-C. The influence of journal self-citations on journal impact factor and immediacy index // Online Information Review. 2012. Vol. 36 (5). P. 639– 654. URL: https://doi.org/10.1108/14684521211275957.
17. Pislyakov V. V. Vliyaet li samotsitirovanie na pozitsiyu zhurnala v reytinge rossiyskih izdaniy. Bechichi, 2006. URL: https://www.elibrary.ru/projects/conference/montenegro2006/presentations/autociting.ppt (data obrashcheniya: 07.05.2021).
18. Frandsen T. F. Journal self-citations — Analysing the JIF mechanism // Journal of Informetrics. 2007. Vol. 1 (1). P. 47–58. URL: https://doi.org/10.1016/j.joi.2006.09.002.
19. Andrade A., González-Jonte R., Campanario J. M. Journals that increase their impact factor at least fourfold in a few years: The role of journal self-citations // Scientometrics. 2009. Vol. 80 (2). P. 515–528. URL: https://doi.org/10.1007/s11192-008-2085-9.
20. Campanario J. M. Journals that Rise from the Fourth Quartile to the First Quartile in Six Years or Less: Mechanisms of Change and the Role of Journal Self-Citations // Publications. 2018. Vol. 6 (4). Art. no. 47. URL: https://doi.org/10.3390/publications6040047.
21. Chorus C., Waltman L. A Large-Scale Analysis of Impact Factor Biased Journal Self-Citations // PLoS ONE. 2016. Vol. 11 (8). Art. no. e0161021. URL: https://doi.org/10.1371/journal.pone.0161021.
22. Davis P. The Emergence of a Citation Cartel. 2012. URL: https://scholarlykitchen. sspnet.org/2012/04/10/emergence-of-a-citation-cartel/ (data obrashcheniya: 30.05.2021).
23. Heneberg P. From Excessive Journal Self-Cites to Citation Stacking: Analysis of Journal Self-Citation Kinetics in Search for Journals, Which Boost Their Scientometric Indicators // PLoS ONE. 2016. Vol. 11 (4). Art. no. e0153730. URL: https://doi.org/10.1371/journal.pone.0153730.
24. Teodorescu D., Andrei T. An examination of “citation circles” for social sciences journals in Eastern European countries // Scientometrics. 2014. Vol. 99 (2). P. 209–231. URL: https://doi.org/10.1007/s11192-013-1210-6.
25. Clarivate. InCites Indicators Handbook. [Philadelphia], 2018. URL: https://incites. help.clarivate.com/Content/Resources/Docs/indicators-handbook-june-2018.pdf (data obrashcheniya: 27.06.2021).
26. Krell F. Losing the numbers game: Abundant self-citations put journals at risk for a life without an impact factor // European Science Editing. 2014. Vol. 40 (2). P. 36–38.
27. Clarivate. Title Suppressions. [Philadelphia], 2021. URL: https://jcr.help. clarivate.com/Content/title-suppressions.htm (data obrashcheniya: 15.07.2021).
28. West J., Althouse B., Rosvall M., Bergstrom C. Eigenfactor Score and Article Influence Score: Detailed methods. [Washington], 2008. URL: http://www.eigenfactor.org/methods.pdf (data obrashcheniya: 27.06.2021).
29. West J. D. Eigenfactor: ranking and mapping scientific knowledge. A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy. University of Washington, 2010. URL: https://jevinwest.org/papers/West2010Dissertation.pdf (data obrashcheniya: 27.06.2021).
30. Pislyakov V. V. Bibliometricheskie indikatory: praktikum. Moskva : NFPK, INFRA-M, 2014. URL: http://ntf.ru/sites/default/files/Last%20Edition.pdf (data obrashcheniya: 07.05.2021).
31. González-Pereira B., Guerrero-Bote V. P., Moya-Anegón F. A new approach to the metric of journals scientific prestige: The SJR indicator // Journal of Informetrics. 2010. Vol. 4 (3). P. 379–391. URL: https://doi.org/10.1016/j.joi.2010.03.002.
32. Guerrero-Bote V. P., Moya-Anegón F. A further step forward in measuring journals’ scientific prestige: The SJR2 indicator // Journal of Informetrics. 2012. Vol. 6 (4). P. 674–688. URL: https://doi.org/10.1016/j.joi.2012.07.001.
33. Pislyakov V. V. Bibliometricheskie indikatory v resursah kompanii Clarivate // Rukvo po naukometrii: indikatory razvitiya nauki i tehnologii, vtoroe izd. / M. A. Akoev, V. A. Marcusova, O. V. Moskaleva, V. V. Pislyakov [pod red. M. A. Akoeva]. Ekaterinburg : izdvo Ural. un-ta, 2021. S. 177–220.
34. Szomszor M., Pendlebury D. A., Adams J. How much is too much? The difference between research influence and self-citation excess // Scientometrics. 2020. Vol. 123 (2). P. 1119–1147. URL: https://doi.org/10.1007/s11192-020-03417-5.
Review
For citations:
Pislyakov V.V. Self-citation and its impact on research evaluation: Literature review. Part II. Scientific and Technical Libraries. 2022;(3):85-104. (In Russ.) https://doi.org/10.33186/1027-3689-2022-3-85-104
Views:
858
Email this article 