In a time in which money is the dominating remunerative incentive for guiding behavior on the labor market, journal publications have assumed the role of the new “currency” for scientists. Decisions about faculty positions, contracts, salaries, or grant applications depend on a scientist’s publication record. The publication of research studies in scientific journals is the mechanism by which the latest discoveries, interesting information, and new knowledge are formally disseminated to the scientific community. Besides the total number of publications, their quality is a decisive prerequisite for academic success. Evaluating the quality of a manuscript is a complex undertaking, but in practice has been increasingly linked to the journal impact factor (JIF).
The identification and evaluation of research studies of high scientific merit is an important but difficult task. Therefore, quantitative measurements of journal article quality, such as the JIF, have become increasingly popular as a surrogate measure of scientific quality. For a particular journal, the JIF is defined as the number of citations within a given year (2021) cited to all papers published in that journal during the previous 2 years (2020, 21) divided by the total number of papers published in that journal during those 2 years. Taking the JIF as the measure of scientific quality has been intensely criticized, and there is a lively and ongoing debate about how the JIF compromises academia in general. Despite the debate on the validity of the JIF as the main criterion for scientific quality and its impact on science, its use is growing, and there is empirical evidence that it influences decision-making in academia. Indeed, this relatively new measure has become deeply entrenched in science and in the evaluation of academic success, be it on a personal, institutional, or national level.
Thus, it seems a logical adaptation of the prevailing principle in science to treat the JIF as a value of a publication just as numbers represent the value of a bill. The ratio has been used to judge the quality of individual research articles, as well as the quality of individual journals. In some countries, the JIF has been used as a criterion for the assessment of research funding, in the appraisal of research staff performance, and in considering job promotions and salary bonuses. However, one single factor cannot measure the scientific credibility of journal articles, journal quality, individuals, specific research projects, or research institutions. Indeed, for this and other reasons, there have been a number of major reviews in the literature criticizing the use of JIFs as a measure of journal article quality and journal quality. Nevertheless, the JIF continues to be used as a surrogate measure of scientific quality in many countries.
Career Impacts
In an increasingly competitive job market, hiring, promotions, grant funding, and bonuses are inextricably linked to a measure, the value of which few people understand completely. Despite the drumbeat of editorials and reviews criticizing its value—there are many examples published between 1997 and 2017 which suggests that JIF has become increasingly institutionalized around the world. At some universities, job applications lacking at least one first-author publication in a high-impact journal isn’t even seriously considered; also it is well documented that JIF is extensively used by many universities for providing financial incentives to scientists who publish in high-impact journals, long known in China but now practiced in more than a dozen countries, including the United States.
Usage of JIF by the scientific community as a predictor of impact has also increased, even while evidence of its predictive value has eroded; both correlations between article citation rate and JIF and proportions of highly cited articles published in high-impact journals have declined since 1990. Because digitization of journal content and proliferation of open-access articles have profoundly changed how relevant literature is located and cited. Having reviewed its history, a Web of Science search was carried out for articles published last year relevant to JIF; of 88 articles, about half are critiques of JIF, yet the other half, for the most part, are journal editorials touting a new or higher impact factor for the year.
Hiring and promotion decisions are too important to be subject to the influence of a metric so vulnerable to manipulation and misrepresentation. Journals can boost their JIF by encouraging selective journal self-citation and by changing journal composition through a preferential publication of reviews, articles in fields with large constituencies, or articles on research topics with short half-lives. JIF has degenerated into a marketing tool for journals as illustrated by the use of “Unofficial Impact Factors” in promotional material for journals that are not even indexed in Web of Science; also they are marketing tools for academic institutions as illustrated by the practice of Clarivate Analytics (which now owns Science Citation Index) of awarding paper certificates and electronic “badges” for scientists determined to be Highly Cited Researchers (HCRs, #HighlyCited) by virtue of publishing papers in the top 1% by citations for their field and publication year.
Such recognition lacks transparency in terms of the criteria used to calculate citation rates and are not particularly predictive of career trajectory in as much as the designation is lifelong. Moreover, this kind of recognition fails to account for the increasing importance of collaborative “team science” that perforce must depend on non-HCRs who, as a consequence of authorship order, don’t earn badges or hashtags in recognition of their contributions. Such practices undercut the efforts to train and reward early-career investigators who participate in team science.
Enacting Change
In Science, it has been widely noted that using JIF as a proxy for scientific excellence undermines incentives to pursue novel, time-consuming, and potentially groundbreaking work: “Only the very bravest of young scientists can be expected to venture into a poorly populated research area.” It has also been criticized that “many pre-tenure researchers believe that the number of papers in top-tier journals is the key to professional success and happiness.” Such a perception isn’t entirely without some foundation. According to a report about 40% of research-intensive universities in the United States and Canada mentioned JIF or related terms in their review, promotion, and tenure documents, overwhelmingly positively. Almost two-thirds of these institutions link JIF with quality. Promotion and tenure committees are contributing to the publication delays by encouraging early-career professionals to submit manuscripts to journals based not on their appropriateness but rather on the “average popularity” of articles therein. That’s a risky way for an early-career investigator to establish a reputation for excellence within a discipline, a criterion ostensibly valued by promotion and tenure committees. Moreover, pressuring early-career scientists to publish in high-impact multidisciplinary journals may also force them to squeeze their best work into a restrictive publication format (limiting page numbers and graphical elements) that, ironically, can reduce its ultimate impact.
Although various scientific journals which are concerned about the vitality and productivity of the community of early-career scientists has created Brief Reports, in part, to provide a venue for early-career professionals to showcase significant discoveries in their early stages, before full mechanistic details requiring years of additional work are completed. Beyond research reports, they also publish Profiles, Questions and Answers, Core Concepts, Inner Workings, and other content relating to the scientific enterprise in general to inform and educate readers, particularly those in the early stages of their career. Because the excessive use of the JIF as the predominant incentive and indicator for scientific quality might be dysfunctional in motivating scientists to “publishing well rather than often”, many journals have signed on to the San Francisco Declaration on Research Assessment (DORA), issued in 2013 to encourage reforms in evaluations of scientific quality by publishers, funding agencies, institutions, organizations that create and provide metrics, and individual scientists.
Further, the extensive use of extrinsic rewards such as money, but also surrogates such as the JIF, could compromise intrinsic motivation and curiosity. It is intriguing to recognize that most of the criticism of the JIF is about finding “better measures in town” to quantify scientific excellence, whereas relatively little attention, is devoted to thinking about the general implications of using these metrics. Accordingly, I wish to close with a reference to Werner’s recent comment that “many negative effects of bibliometrics come not from using it, but from the anticipation that it will be used. When we believe that we will be judged by silly criteria, we will adapt and behave in silly ways”.
(Author is Postdoctoral Associate-NIH, University of Nebraska–Lincoln, USA. Mail: drtajamulmumtaz@gmail.com)