ScienceDirect is a database that contains over 25% of the worlds science, technology and medicine full text and bibliographic information (journals and books) published by Elsevier and other publishers. 2019; DAntuono and Ciavarella 2019; Ioannidis et al. Metrics of Social Impact Based on Twitter and Correlation with Traditional Metrics of Scientific Impact", "Measuring Up: Impact Factors Do Not Reflect Article Citation Rates", "Ditching Impact Factors for Deeper Data", "The effect of open access and downloads ('hits') on citation impact: a bibliography of studies", "Comparing the Impact of Open Access (OA) vs. Non-OA Articles in the Same Journals", "Citation Advantage of Open Access Articles", "Ten-Year Cross-Disciplinary Comparison of the Growth of Open Access and How It Increases Research Citation Impact", "Self-Selected or Mandated, Open Access Increases Citation Impact for Higher Quality Research", "Open access publishing, article downloads, and citations: randomised controlled trial", "The impact factor of an open access journal does not contribute to an article's citations", "Universality of citation distributions: Toward an objective measure of scientific impact", "Scholarometer: A Social Framework for Analyzing Impact across Disciplines", "What a difference a colon makes: how superficial factors influence subsequent citation", 20.500.11755/2fd7fc12-1766-4ddd-8f19-1d2603d2e11d, "Evaluation by Citation: Trends in Publication Behavior, Evaluation Criteria, and the Strive for High Impact Publications", "Psychology and Wikipedia: Measuring Psychology Journals' Impact by Wikipedia Citations", "Psychology and Wikipedia: Measuring journals' impact by Wikipedia citations", "Research Quality, Publications and Impact in Civil Engineering into the 21st Century. The narrow definition would require a specific authors name on both citing and cited documents; the broad view would describe a self-citation as any instance in which any author name appears on the two (scientist B, without scientist A as an author, citing a paper by scientist A and B), sometimes called all-author to all-author self-citation. *Neurosciences: The ranking is calculated from total citations for all journals in the Frontiers Neuroscience journal series, in the Neuroscience subject area (CiteScore 2020 edition). average, a paper from a Geosciences research field published in 2008 Analysis of the world's 20 largest publishers by volume, ranked by average number of citations in 2020 received by articles published in 2017, 2018 and 2019 (Dimensions, 2021). In total, 9,141 from 1900-2015 and 4,593 from 2016-2020 endometrial cancer related studies were identified with the USA having the largest numbers of publications, citations, institutions, as well as the highest country-specific h-Index concerning endometrial cancer research in both periods. The Journal Impact Factor is the average number of citations received in the last year to articles published in the previous two years. A Cited Reference Search will provide the number of times a particular publication has been cited in the Web of Science database since its publication. will be below the citation rate for all years. CiteScore metrics: A family of eight metrics and useful features. The dataset used for this analysis was the Web of Science list of Highly Cited Researchers for 2019. The citation advantage of linking publications to research data Percentile values are baselines for evaluating research impact. interpreted as representing the central tendency of the distribution. https://doi.org/10.22452/mjlis.vol22no1.1. Merton later included humility in one description of the norms of scientific research (Merton 1957). The metric tide: Report of the independent review of the role of metrics in research assessment and management. By the end of January, more than 50 research papers had been published about the outbreak [2]. Fields that lie outside the main distribution (Pharmacology, Mathematics and Space Science) have been labelled. Such reasons imply behaviors that conform both to Mertonian norms and also to the social constructivist interpretation of citation theory (Merton 1942; Kaplan 1965; Moravcsik and Murugesan 1975; Gilbert 1977; Cozzens 1989; Bonzi and Snyder 1991; White 2001; Bornmann and Daniel 2008; Davis 2009; Erikson and Erlandson 2014; Tahamtan and Bornmann 2018, 2019; Aksnes et al. Previous studies have looked at citation rates for preprints of Covid-19 papers [17] and quality of evidence contained within published Covid-19 articles [18, 19]. https://doi.org/10.1007/BF02019738. The comparative median rates of a self-referencing and b self-citation analysed by Essential Science Indicator field and the comparative lower thresholds for indicative outliers in those fields. 2016; Heneberg 2016; Fong and Wilhite 2017; Zaggl 2017; Scarpa et al. https://doi.org/10.1002/asi.23728. This analysis, like others before, has been about individual self-citation by researchers. Larivive, V., Gingras, Y., Sugimoto, C. R., & Tsou, A. period. 2019; Peroni et al. For the 2019 Highly Cited Researchers analysis, the papers included in the preliminary data development were those published and cited during 20082018. So the RCR would be 9 citations per year / 6 citations per year = 1.5. Types of items Journals included Time periods for counts Average Citation Rates Thresholds Types of items counted Papers are defined as regular scientific articles, review articles, proceedings papers, and research notes. Publications of the Astronomical Society of the Pacific,98(610), 13471348. The self-referencing median rates and outlier thresholds are similar across fields but these are more variable for the self-citation indicators. A discussion of measuring the top-1% most-highly cited publications Measuring science: Basic principles and application of advanced bibliometrics. It is important to recognize that citation counts grow over Bakare, V., & Lewison, G. (2017). https://doi.org/10.13140/RG.2.1.4929.1363. The key to interpretation is the need to determine the relationship between summary citation impact and fundamental research influence. Oakland, CA. (2015). The negative effects of citing with a national orientation in terms of recognition: National and international citations in natural-sciences papers from Germany, the Netherlands, and UK. 2 was repeated with the other ESI fields. https://doi.org/10.3145/epi.2018.mar.03. papers that meet the criteria for inclusion in Essential Science Indicators. "[61][62], According to Mario Biagioli: "All metrics of scientific evaluation are bound to be abused. It is measured each year by the Web of Science Group and reported in the Journal Citation Reports (JCR). Isis,79, 606623. Journal of the American Society for Information Science and Technology,61(7), 15141515. PLoS ONE, 11(4), article number e0153730. Negative binomial regression output reporting independent variable effects on citation count, PLOS is a nonprofit 501(c)(3) corporation, #C2354500, based in San Francisco, California, US. Non-citation rates vary enormously by field. van Raan, A. F. J. Evidence of complex citer motivations. Highest rates of self-citing in some categories that are still within that central range. Two common field normalized citation metrics types include the Relative Citation Ratio (RCR) from iCite and Field Weighted Citation Impact from Scopus. Journal self-citations: Analyzing the JIF mechanism. Journal of the Association for Information Science and Technology,68(4), 10361047. Porter, A. L. (1977). Szomszor, M., Pendlebury, D.A. Welcome to the Next Generation Essential Science Indicators. So the FWCI would be 23 / 18 = 1.28. [11] By 2014 it had accumulated more than 305,000 citations. In every field-based profile, some researcher portfolios are on a steeper part of the curve with values above but continuous with the central range. Reasons behind the major increase in citations for Covid-19 articles seem straightforward. Example: An article published in 2017 has been cited 5 times in 2017, 10 times in 2018, 5 times in 2019, and 12 times in 2020. A high self-citation rate may reflect the extent of contributions and the researchers focus. On the skewness of journal self-citations and publisher self-citations: Cues for discussion from a case study. The linear analysis provides an accessible dataset for initial interpretation of citation rates. https://doi.org/10.1371/journal.pone.0161021. Can the quality of published academic journal articles be assessed with New insights into the relationship between the h-Index and self-citations? https://doi.org/10.1016/j.joi.2018.07.009. We thank our ISI colleagues, particularly Ross Potter, Gordon Rogers and Josh Schnell, for their advice and suggestions during the development of this work. The AWCR is very useful, but it only works if publishers enter the dates of their online materials correctly. San Francisco, CA. Much has changed from the late 20th century to the early 21st. Citation impact - Wikipedia Like any metric, field normalized citation metrics have their limitations. Journal of the American Society for Information Science,27, 292306. The distributions of the average percentage of a self-references in highly cited papers and b self-citations from highly cited papers for 3517 Highly Cited Researchers identified in the Web of Science for 20082018 publications. Macfarlane, B., & Cheng, M. (2008). Is there a clubbing effect underlying Chinese research citation increases? What do citation counts measure? So the RCR would be 9 citations per year / 6 citations per year = 1.5 (2009). The time period for citation counts is 10 years, plus partial year counts for Article PLoS Biology, 17(8), article number e3000384. The Journal Impact Factor is the average number of citations received in the last year to articles published in the previous two years. Young, J. S., & Brandes, P. M. (2020). Median and range were used to represent continuous variables (not normally distributed) while we used frequencies and percentages to represent categorical variables. A negative binomial regression model was used to assess for potential confounding variables (Table 3). Their study found no penalty for self-citation but diminishing returns with respect to generating citations from others beyond a certain point. (2018). Insights into the relationship between the h-Index and self-citations. To our knowledge, this study is the first to quantify the rate at which a large volume of peer-reviewed Covid-19 articles are being cited, on average, in major medical journals and compare these results to non-Covid-19 articles in the same journals. Note, in this paper, that we refer to self-references and to self-cites and that they are not the same thing. The following chart shows the differences between Citation Impact of various subject categories. Each graph shows the frequency distribution of the original data for the researchers and a computed log-normal fit of these data to a negative binomial distribution. LibGuides: InCites Benchmarking & Analytics: Baselines Michael D. Brandt, Are 90% of academic papers really never cited? Reviewing the literature Galvez, R. H. (2017). However, the spread of outlier values that rise above the high threshold may be clearer evidence of unusual citation behaviour well outside the normal range. Trimble, V. (1986). Fong, E. A. The field-specific self-citation profiles, showing a data point for each Highly Cited Researcher and following the style of Fig. An RCR of 1 is considered average, while an RCR greater than 1 is above average. 2019; Kacem et al. Field Normalized Citation Metrics - Impact Metrics - Research Guides at When negative binomial regression is used to minimize the influence of other variables such as article number variation and field of research, Covid-19 papers have still experienced more than 80% increase in citations relative to non-Covid-19 papers. https://www.who.int/emergencies/disease-outbreak-news/item/2020-DON229, https://www.nature.com/articles/d41586-020-00253-8, https://www.who.int/director-general/speeches/detail/who-director-general-s-opening-remarks-at-the-media-briefing-on-covid-1911-march-2020, https://www.webofscience.com/wos/woscc/basic-search, https://jcr.clarivate.com/jcr/home?Init=Yes&SrcApp=IC2LS, Corrections, Expressions of Concern, and Retractions. The fourth age of research. 2019; DAntuono and Ciavarella 2019; Peroni et al. As such, researchers are eager to add their contributions to what is known and what can be done to combat Covid-19 and, to do this, are citing earlier works to support their approaches. https://doi.org/10.1002/(SICI)1097-4571(198909)40:5%3c342:AID-ASI7%3e3.0.CO;2-U. This holds even when article volume is equated using a 1:1 sample of non-Covid to Covid articles as median citation is six times greater for the Covid-19 articles. Quantifying the Impact of My Publications: Citations The method to calculate the impact factor of The Social Science Journal is as follows. [34][35][36][37][38][39][40][41] Some experimental and observational studies have found that articles published in OA journals do not receive more citations, on average, than those published in subscription journals;[42][43] other studies have found that they do. These data are also summarised in the Appendix Table3. Engqvist, L., & Frommen, J. G. (2008). Journal of Informetrics,10(2), 365391. This suggests that a propensity to unusually high self-citation rates is not distributed uniformly across all research areas. Given the refereeing system that controls the quality of the scientific literature in the better known journals, the high publication count could be achieved only if the person had a lot to say that was at least marginally significant. https://doi.org/10.1016/j.respol.2017.02.009. 2004, 2006; Glnzel and Thijs 2004a, b; Schubert et al. As examples: for the ESI field of Chemistry (Fig. These results are similar to where the pay gap stood in 2002, when women earned 80% as much as men. Malaysian Journal of Library & Information Science,23(2), 2546. The 919,038 papers received a total of 4,667,747 citations. To examine this more closely, we randomly selected three journals included for this paper and manually searched each journals website for Covid-19-related articles that the journal itself had categorized as an original piece of research over three months. https://doi.org/10.1016/j.joi.2016.04.010. These rates of self-citation are evidently much lower than those of self-referencing, as expected for Highly Cited Researchers because their broad peer influence attracts many cites from others (van Raan 2008), and they are relatively more variable between fields compared to self-referencing. When quoting verse, maintain original line breaks. different disciplines and time periods so it cannot be used effectively to compare entities that are in different subjects or years. In some instances, citation analysis may not provide the entire story of the impact of a research project. the journals of the American Society for Microbiology. Citation: Brandt MD, Ghozy SA, Kallmes DF, McDonald RJ, Kadirvel RD (2022) Comparison of citation rates between Covid-19 and non-Covid-19 articles across 24 major scientific journals. Open access meets discoverability: Citations to articles posted to Academia. Inclusion of these would undoubtedly have increased the volume of papers and affected the results, though the direction or magnitude of change is unclear. Cumulative advantage and the symbolism of intellectual property. Coercive citation in academic publishing. For instance, many more scientists work in neuroscience than in mathematics, and neuroscientists publish more papers than mathematicians, hence neuroscience papers are much more often cited than papers in mathematics. Looking at all categories and journals that were included, Covid-19 articles published between January 1, 2020 and December 31, 2021 are being cited at considerably higher rates than the non-Covid-19 papers. The average citation per article for these publications was 8.1. https://doi.org/10.1087/095315108X378712. [56] Also it is found that scholars engage in ethically questionable behavior in order to inflate the number of citations articles receive.[57]. https://doi.org/10.1007/BF02017571. One of the most basic citation metrics is how often an article was cited in other articles, books, or other sources (such as theses). https://doi.org/10.1023/B:SCIE.0000045117.13348.b1. Journal of the American Society for Information Science and Technology,60(6), 12831285. Self-citation can inflate h-index. We describe both a generic level of median self-referencing rates, common to most fields, and a graphical, distribution-driven assessment of excessive self-citation that demarcates a threshold not dependent on statistical tests or percentiles (since for some fields all values are within a central normal range). Physics (194 Highly Cited Researchers) is a similar sized field to Engineering to which its research culture might also be linked. The smaller sets had too few articles to analyze, however, so the final number of field/year combinations analyzed was slightly less. In doing this, we demonstrate the sharp contrast between citation rates for Covid-19 and non-Covid-19 articles published in 24 months during the rise and height of the global pandemic. If there are indeed a significant number of individuals who abuse the conventional research system by inappropriate practice then why does the system not detect and sanction this at review before publication? What do citations count? The influence of international collaboration on the impact of research results: Some simple mathematical considerations concerning the role of self-citations. The following sample from a Percentiles The term percentile designates a citation threshold at or above that at which a fixed fraction of the papers fall. Citation averages, 2000-2010, by fields and years We had reason to question this approach when we noted that China was leading in citation rates https://doi.org/10.1002/leap.1235. The field or subject area: t: Year: d: Document Type: n: The number of subjects to which a paper is assigned: i: . To address this gap in the literature we use WOS to examine citation rates for Covid-19-related articles published between January 1, 2020 and December 31, 2021 across 24 major scientific journals. https://clarivate.libguides.com/incites_ba. 3: Key Measures of Academic Influence | Impact of Social Sciences Ideas in Ecology and Evaluation,7(1), 12. Scopus also offers a Citation Tracker feature that allows for a listing of articles, web sources and patents that cite a particular publication since 1996. Frontiers journals improved their Journal Impact Factor ranking in 30 out of 39 categories in the 2021 Journal Citation Report. Brooks, T. A. Research Policy,46(5), 970983. (2020). A note on self-citation rates in astronomical papers. Egghe, L. (1987). https://doi.org/10.1007/BF02016632. [8][9] Similarly, review papers are more often cited than regular research papers because they summarize results from many papers. The prevalence of synchronous self-citation practices at the institutional level. No other fields accounted for significant difference in citation rates amongst Covid-19 and non-Covid-19 papers published within the same field. The median home value across these five cities, on average, is about $137,900. It follows that 90% of Microbiology papers published in 2011 received fewer than 14 citations. ESI surveys more than 11,000 journals from around the world to rank authors, institutions, countries, and journals in 22 broad fields based on publication and citation performance. We first need to add up the number of citations the article has received the year it was published and the 3 calendar years following its publication (i.e., 2017 to the end of December, 2020). [16] Publish or Perish is a software program that retrieves and analyzes academic citations. https://doi.org/10.1002/asi.21042. This means that other publications such as letters, editorials, perspectives, and opinions were not included in our search criteria which leaves a large number of publications out of the study. The field of Pharmacology and Toxiciology stands out for a different reason. In an isolated instance, citation counts will be highly inflated, leading to overestimates of the influence the scientist has in his field. While Garfield doubted self-citation would be a problem in research evaluation, Narin imagined the possibility. Journal of Education and Social Policy,2, 7579. Zaggl, M. A. The 2020 Journal Impact Factors, published in the 2021 Journal Citation Report, are based on citations in 2020 to articles published in 2018 and 2019. These are only labelled on the first graph. This may also be the reason why papers with shorter titles get more citations, given that they are usually covering a broader area. Data covers a rolling 10-year period and includes bimonthly updates to rankings and citation counts. This is most easily interpreted on the linear plot, whereas statistical sense would suggest that the data should be log-plotted because of the skewed negative binomial distribution. Use the Cited References feature to find a work, then select Find Citing Articles. https://doi.org/10.1016/j.joi.2013.11.001. https://doi.org/10.1007/s11192-011-0523-6. The citation rate for all years is the total number of citations Measuring research. 6, 7, 8, 9). Shehatta, I., & Al-Rubaish, A. M. (2019). (2019) were right to draw attention to the statistical properties of these distributions: they have regular and testable properties. For example, the top decile of author self-citation could be within normal range for some fields and yet not for others. Using a negative regression model to analyze the entire data, Covid-19 papers have 84% more citations than non-Covid-19 papers when controlling for field and article number discrepancies. ScienceDirect offers a search feature that displays cited references (journal articles or books) for a specific author or publication. Ioannidis et al. They are not statistical tests but point to rather than definitively identifying possible cases of excessive self-citation. Citation Analysis - Measuring Your Impact: Impact Factor, Citation