Introduction

Citation indicators have become very popular in appraising the published work of scientists. However, they are also known to have major caveats that can lead to misleading conclusions [1]. To confuse matters more, many different citation indicators currently compete for prominence. Besides the total citation count, perhaps the most popular indicator is the Hirsch H index (H) [2], and there have been long debates about its relative merits and drawbacks [3–6]. Moreover, it is increasingly recognized that in most scientific fields there is an increasing number of co-authors per paper [7–9]. Some fields, such as particle physics, are operating routinely with large-scale collaborations with hundreds of authors. Many fields in biology, genomics, epidemiology, and medicine also have many papers with large numbers of authors. Therefore, citation indicators have been proposed that account for co-authorship [10–12]. One may also want to separate citations according to whether they come from single-authored work or whether the scientist has a leading role as suggested by authorship placement, e.g., first or last author. Of course, with the exception of single-authored work, the relative contribution of authors cannot be fully ascertained from authorship position alone. However, empirical surveys suggest that, except when listing of authorship is alphabetic, first and last authors have typically had the greatest contributions in a multi-authored paper [13–16].

There are numerous surveys that investigate one or a few of the proposed citation indicators in samples of scientists limited to specific scientific disciplines. It would be more useful, however, to assess a large number of the most prominent citation indicators across all the tens of thousands of scientists with substantial citation impact across all scientific fields. This could help us understand to what extent different citation indicators are correlated with each other or offer independent information. Furthermore, one could assess whether these correlation patterns differ across different scientific fields. Finally, one could generate composite indicators that incorporate information from multiple citation indicators. A composite may be more appropriate, because it would not be that much influenced by extreme values in only one indicator.

Here, we performed such an evaluation of 84,116 influential scientists across all scientific disciplines around the world. For further details, see Methods. All citation indicators, unless specified otherwise, refer to the citation impact of the scientists’ work in the calendar year 2013. Scopus contains very complete information, including reference lists, for documents published since 1996. Coverage for documents published before 1996 is somewhat (~15%) less, and Scopus is only now adding reference lists for documents published from 1970 to 1995. This limits our ability to assess the full citation profile of a scientist over his/her entire career for careers that were initiated prior to 1996. Perhaps more importantly, one would like to focus on the recent citation impact of a scientist, as this may be more reflective of both his/her current visibility and his/her future citation rate (e.g., the number of citations in a given year is the strongest predictor of citations in future years [17]) and perhaps also the potential for continuing to perform influential work in the near future. However, in principle, the same approach as we outline in this manuscript can also be applied in evaluating citation metrics of scientists covering multiple years (if this is considered desirable for whatever reason) or even the entire career (if databases have coverage for the respective time span).

Results

Correlation between Citation Indicators

For analysis, each of the six indicators listed above was log-transformed. Table 1 shows Pearson correlations between these log-transformed indicators across all 84,116 scientists. As shown in Table 1, the correlation of NC and H is strong (r = 0.88), suggesting that most of the time, with some exceptions, total citations and H-index gave the same impression about the ranking of a scientist. The exceptions occurred mostly because of scientists who had (co)-authored only a few papers with very large citation impact. Exclusion of 8,481 scientists with fewer than 30 papers indexed in Scopus resulted in an even stronger correlation (r = 0.91) between NC and H.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Correlations between various citation indicators and other metrics across all 84,116 scientists.

https://doi.org/10.1371/journal.pbio.1002501.t001

Conversely, there was no correlation between the total number of citations and the co-authorship adjusted Hm (r = 0.00), and there was a negative correlation between total number of citations and each of the three author-order-based citation indicators (r = -0.43, r = -0.22, and r = -0.04 for NS, NSF, and NSFL, respectively, p < 0.001 for all). Some of the most-cited authors (in terms of citation volume, i.e., total citations or H index) had published very few or even no influential papers as single, first, or last authors. For example, of the 100 most-cited authors according to total citations (NC = 7,457–19,245 Scopus citations received in 2013), almost half of them (n = 48) had received only 0–20 citations to papers or other scholarly documents for which they were the single author, and 15 of them did not have even a single cited single-authored paper or other scholarly document. Moreover, of the 100 most-cited authors according to total citations, 12 had received fewer than 100 citations in papers for which they were the single or first author, and 11 had received fewer than 1,000 citations to all papers for which they were single, first, or last author. Thus, selection of top scientists based on total citations only may include many scientists without a leading, key contribution to the published work.

Indicators of citation volume (total number of citations and H) were somewhat strongly correlated with the number of papers. However, they had no substantial correlation with the mean number of citations per paper (Table 1). Visual correlations between log-transformed pairs of the six individual indicators and composite indicator are shown in S1 Fig.

Field-Specific Analyses

Each of the published items indexed in Scopus was allocated into one of 12 fields (physics, mathematics, computer science, chemistry, earth sciences, engineering, biology/biotechnology, infectious disease, medicine, brain research, health sciences, social sciences) that have been defined in previous work [19,20]. Then, we allocated each of the 84,116 scientists to the scientific discipline that had the largest share of his/her published work indexed in Scopus. We separately calculated the correlations of NC with all other indicators in each scientific discipline that included at least 1,000 scientists. As shown in Fig 1, the correlation between NC and H was always strong across all disciplines (r = 0.776–0.890). NC was negatively correlated with NS in all disciplines and was most strongly negatively in physics. Physics, which has the strongest tradition of extremely multi-authored papers, stands out from the other disciplines in that NC was negatively correlated with all four author-adjusted or author-order indicators (NS, NSF, NSFL, and Hm index). Excluding physics, the correlation of NC with the co-authorship-adjusted Hm was substantial, although quite variable across disciplines (r = 0.21–0.76) and always smaller than the correlation between NC and H. NC correlated positively with NSF in all disciplines but biology and physics, with correlations ranging from weak (medicine, r = 0.02) to strong (social sciences, r = 0.74). Correlations of NC with NSFL were much stronger, above 0.7 for all disciplines except biology, infectious disease, and medicine. The correlation between NC and the number of papers varied moderately across disciplines (from r = 0.37 in biology to r = 0.58 in brain sciences and mathematics). There were no substantial correlations between NC and the number of citations per paper in any discipline, except for mathematics (r = 0.25).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Correlation between number of citations and various citation indicators and other metrics in each of 12 different scientific fields.

Abbreviations: PHYS, physics; MATH, mathematics; CS, computer science; CHEM, chemistry; ENG, engineering; EARTH, earth sciences; BIO, biology/biotechnology. INFDIS, infectious disease; MED, medicine; BRAIN, brain research; HEALTH, health sciences; SOC, social sciences. No data are shown on humanities, for which there are too few papers and too few citations in Scopus to allow meaningful analysis.

https://doi.org/10.1371/journal.pbio.1002501.g001

Capturing Nobel Laureates of 2011–2015 with Different Metrics

Table 2 shows, for each of the Nobel laureates of the years 2011–2015 in physics, chemistry, physiology/medicine, and economics, whether he/she belongs to the top 14,150 scientists according to the composite score and, similarly, to the top 14,150 according to total citations, H index, or Hm index (as above, we know that the ranking is inclusive for the top 14,150 ranked scientists). It also shows whether they are among the top 30,000 scientists according to any of the six metrics included in the composite score.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Capturing Nobel laureates of 2011–2015 by top rank in 2013 with different metrics.

https://doi.org/10.1371/journal.pbio.1002501.t002

As shown, the composite score captures 31 of the 47 laureates, as opposed to only 15 captured by the total citations, 18 by the H index, and 26 by the Hm index. A total of 40 of the 47 laureates were in the top 30,000 scientists in 2013, according to at least one of the six metrics included in the composite score. The seven Nobel laureates who were not captured by any of the six indicators are useful to scrutinize in order to understand why these indicators failed. The seven exceptions included two economists (Lars Peter Hansen and Thomas J. Sargent) for whom Scopus had poor coverage of citations for their work (they are both highly cited in Google Scholar, which provides >10-fold more citations for them than Scopus); two scientists with a major contribution in the development of important antiparasitic drugs (Williams C. Campbell and Youyou Tu), whose work is not that highly cited; and three physicists (Takaaki Kajita, Arthur B. McDonald, Hiroshi Amano) who are very highly cited, but their citations peaked >10 y earlier (moreover, the most-cited papers for Kajita and McDonald had alphabetic authorship).

Number of Papers and Citations per Paper

In contrast to the six indicators that are included in the composite indicator, for which higher values are better than lower values, the number of papers and the citations per paper are not necessarily better when they are higher. In fact, by definition, these two metrics are inversely related. Excellent scientists may publish few papers or many papers, and this will respectively increase or decrease their citations per paper. There was tremendous variation in these two metrics across the 84,116 scientists: the number of papers varied from 1 to 2,533, and citations per paper varied from 0.08 to 5318. Even among the top 1,000 scientists with the highest composite score, the number of papers varied from 26 to 2,533, and the citations per paper varied from 1.55 to 288.9.

Fig 2 shows the distribution of citations per paper for authors in various fields. As shown, for most fields, a minority of scientists had extremely high values, and another minority had extremely low values. Perhaps one could wish for investigators to avoid having a low or very low number of citations per paper (and this might also be tailored according to field), while the exact value above that threshold would not make a difference. For example, values below 5 were seen in 42,984 of the 84,116 (51%) scientists, but in only 145 of the 1,000 with the highest composite C scores.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Cumulative proportion of scientists (among total n = 84,116) with citations per paper greater than a given number.

Data are shown separately for each of 12 scientific disciplines. Scientific discipline abbreviations are the same as in Fig 1.

https://doi.org/10.1371/journal.pbio.1002501.g002

Impact of Self-Citations

Fig 3 shows the percentage of self-citations for each of the analyzed scientists. Among the 84,116 scientists, the mean was 5.5% and the median was 3.3% (interquartile range, 0.7% to 7.4%). Among the top 1,000 scientists based on the composite score, the mean was 4.2% (median 3.0%, interquartile range 1.3% to 5.4%). The proportion of self-citations exceeded 30% for only 1,206 of the 84,116 scientists (1.45%). Furthermore, only 7 of the top 1,000 scientists based on the composite score (0.7%) have such an extreme self-citation rate.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Percentage of self-citations for each of the 84,116 analyzed scientists.

https://doi.org/10.1371/journal.pbio.1002501.g003

Misallocation of Credit Due to Alphabetic Ordering of Authors

Indicators based on author order, such as NSF and NSFL, would be misleading in papers for which authorship order is selected alphabetically rather than reflecting the relative value of contributions. In general, the proportion of papers that are expected to have alphabetic order of authors is 1:k! (where k is the number of authors), i.e., 100%, 50%, 16.67%, 4.17%, 0.83%, 0.14%, 0.02%, 0.002%, and ~0% for papers with k = 1, 2, 3, 4, 5, 6, 7, 8, and >8 authors, respectively. When the proportion of papers with alphabetic order exceeds 1:k!, the excess provides an estimate of the frequency of papers for which authors are ordered alphabetically, not because this reflects their relative contributions, but because this is a chosen authorship listing strategy. The excess was 0%, 3.73%, 4.04%, 2.25%, 1.19%, 0.65%, 0.48%, 0.38%, and 0.40% for the respective values of k. It was 1.97% when data are considered across all articles and all k.

Table 3 shows the proportion of papers with alphabetic listing of authors for k = 3–5 for each of the 12 main scientific fields (plus humanities). As shown, there was a small excess (<1%) from the expected 1:k! in 9 of the 12 scientific fields and a substantially larger excess for physics, mathematics, social sciences and humanities, with mathematics showing the highest incidence of alphabetic ordering.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Proportion of papers with authors in alphabetic order in different scientific fields (data are shown for papers with 3, 4, and 5 authors).

https://doi.org/10.1371/journal.pbio.1002501.t003

Discussion

Multiple citation indicators may be considered in appraising scientific impact, including those addressing total impact, co-authorship adjustment, and author order. We have demonstrated the complex correlation patterns between these indicators across 84,116 top scientists and separately across 12 scientific fields. Indicators of total impact and co-authorship/author order may be used in a complementary fashion to generate a composite score. No citation indicator, single or composite, can be expected to select all the best scientists. However, the use of a composite may allow for capturing not only citation influence but also aspects of co-authorship and relative credit in multi-authored papers that currently represent the large majority in most scientific fields.

Perusal of the data on the top 1,000 scientists based on the composite indicator shows that they are generally well-known, leading, high-profile scientists. This applied also to the scientists who were top-ranked based on the composite score, but would have been ranked much lower based on total citation count. For example, many Nobel laureates do not necessarily have the highest number of total citations, but they tend to have a pivotal role as single, first, or last authors of some extremely influential works. The vast majority of Nobel laureates of 2011–2015 ranked at extremely high ranks in 2013 by at least one of the six metrics; the few exceptions were scientists with earlier citation peaks, poor coverage of their field in Scopus, and only two great discoverers without any extremely high citation metrics. Furthermore, the composite score included in its top-most ranks a higher number of the Nobel laureates of 2011–2015 than any other metric alone in their respective top-most ranks. In particular, ranking using a traditional metric such as total citations brought to the top ranks large numbers of scientists without any cited papers as single, first, or last authors. An in-depth evaluation of a sample of them suggested that many of them were collaborators in large projects and others were probably auxiliary researchers. For those for whom we could not find their affiliation and rank online, it is even more unlikely that they would have been leading drivers in the published research.

Of course, it is not possible to ever reach certainty that scientists ranked on the top based on the composite score are more influential than others with modestly lower ranking. There is no gold standard for appraising scientist merit. Nevertheless, the selected top-ranked scientists represent a cohort of highly influential scientists who can be used for other empirical assessments [21]. There can also be some debate about whether co-author/author order components should carry similar, less, or more weight as compared with total citation impact. Different weight and approaches to forming a composite citation indicator may select more collaborative types of scientists or more individual leaders [22]. For example, the Thomson Reuters approach for selecting the most influential scientists does not consider author order and multi-authorship (except for extremely multi-authored papers in physics that are excluded) [23]; as a result, the most influential scientist in the world in 2014 and 2015 according to the Thomson Reuters list is a scientist who has not been the first or last author of any highly cited paper.

The proposed composite indicator does not include the number of papers or the citations per paper. We see no point in rewarding a scientist for publishing more papers [24]. While some promotion or funding systems reward such plain productivity, this can stimulate salami publication of least publishable units and sloppy, low-quality work [25]. As for citations per paper, a wide range of values can be equally acceptable. Some investigators have argued in favor of using citations per paper as an indicator of investigator quality [26]. However, this metric depends a lot on the specific sub-field and numbers of co-authors, and values can be markedly affected by outliers (one or a few extremely cited papers). Moreover, over-emphasis on citations per paper may lead investigators to avoid publication of negative, high-quality results and may make them pursue reporting extreme results and exaggerated narratives that draw attention. Nevertheless, it may be reasonable to ask for investigators to maintain citations per paper above a relatively low given threshold. Such an approach would sufficiently discourage salami publication and the publication of least publishable units without discouraging publication of good studies with negative results.

As we admitted up front, citation indicators have limitations, and they should be used with great caution [1]. They should be complemented with in-depth qualitative appraisal of scientific work. However, citation indicators are also here to stay. Our approach suggests how one can gain insights about their behavior and patterns across science and within specific scientific disciplines. Instead of using only one citation indicator, systematically examining multiple indicators offers a more complete, granular picture. Previous empirical assessments have examined the correlation between certain indicators [27,28], but these have been limited mostly to H (and variants of H), NC, and number of papers (NP), without consideration of co-authorship-adjusted and author order indicators. Availability of large-scale citation data on all publishing scientists may also allow for placing these indicators in the context of the entire global scientific community and the scientific community of large disciplines or even smaller focused sub-disciplines. Here, we worked with data on 80,000+ influential scientists, including the ones who are largely shaping the current scientific literature [29], and we placed emphasis on statistics based on author order, which has rarely been done before. However, one can expand the same principles to the many millions of scientists publishing scientific work, provided that the citation databases used adequately cover the specific scientific field being evaluated; otherwise, results may be misleading [28]. We should caution that it is difficult to differentiate the ranking of scientists with small differences in their indicators, and this applies to both single and composite indicators.

Correction for self-citation and fractionalization of citation counts could also be considered [1], although we have not done so here. We have nevertheless explored the extent of the challenge offered by self-citations. Within a reasonable range, self-citation is not inappropriate and may be even considered necessary for transparent attribution to prior work. However, there are exceptions of extreme self-citation rates, and perhaps these should be “penalized” in the ranking. For example, “penalizing” scientists with >30% self-citation rate may affect a substantial number of scientists. Our data suggest that such a rule for selecting highly self-cited authors would have a negligible imprint on the scientists ranked at the top, based on the composite score.

New indicators are continuously proposed, and there is the challenge of how to incorporate them in bibliometric assessments of scientists, institutions, and journals [30]. With a rapid proliferation of citation metrics, there are additional indicators besides NC and H focusing on total citation impact (e.g., the Egghe g index [31]), but NC and H are still more widely used. Moreover, there are other indicators besides Hm that apply fractionalization based on the number of authors [32,33]. We used Hm, which is an index that has been widely cited since presented in 2008 (over 230 citations for the journal article and the respective arXiv article), and because some others (e.g., the h_I index [32]) have the undesirable paradox that they can even decrease over time (e.g., if multi-authored papers in a scientist’s curriculum vitae become more cited than her papers with few authors). Similarly, various other indicators may try to capture the citation impact of papers at specific author positions; e.g., one can calculate an H index limited to papers as single and/or first and/or last author. However, such author-position-specific H indicators may not be as informative as author-specific-position citation counts, because the number of papers for which a scientist has a specific position is limited; thus, these H indicators will mostly be very small. An H = 2 for papers as single author may refer to two papers with two citations each or two papers with 3,000 and 2,000 citations.

In several scientific fields, it has been shown that, over a researcher’s career, one moves from doing mostly implementation of research (first author) to advising, designing, and financially supporting research (last author) [34]. Moreover, authorship order reflects publication strategies of investigators [35,36]. However, attributing credit fairly according to author position may stumble upon the use of author-listing practices that do not use the first and last author positions based on maximal contributions but based on alphabetic considerations. Alphabetic listing of authors has been studied in a number of empirical evaluations, and it has been previously noted that some fields, such as economics and physics, may be particularly affected [37–39]. Our evaluation is based on a very large number of articles across all scientific fields. We show that alphabetic authorship does occur, but its impact would be tiny in most scientific fields in the life, biological, and medical sciences. A larger prevalence is seen in social sciences, humanities, physics, and mathematics. Overall, for most sub-disciplines, consideration of single, first, and last authorship offers meaningful information. However, as previous studies have shown, physics and social sciences are heterogeneous in their citation practices and in the extent to which they use alphabetic ordering. Therefore, some sub-disciplines within these fields may use alphabetic ordering far more frequently. In these sub-disciplines, use of indices based on first and last author position would be problematic. Differences between sub-disciplines may also affect the correlation pattern between different indicators. It has also been shown that in some disciplines, such as economics, first authors whose names start from ABC get more cited than those whose names start from XYZ [40]. However, the overall distortion due to this phenomenon seems very small in absolute magnitude.

The 12 key fields that we addressed here have generally good coverage at Scopus (social sciences are the one possible exception), although fields such as computer science and engineering may be at a relative disadvantage, since they have fewer citing references listed per paper. One advantage of the composite indicator is that it may help dampen differences between fields, because one type of indicator gives an advantage to some fields, but another type of indicator may counter that advantage. For example, total citations give an advantage to disciplines like empirical work in particle physics that have many multi-authored papers, but Hm dissipates this advantage, and author-order indicators even penalize this discipline. However, comparisons between fields should remain cautious. It is not even known what the ideal proportional representation of scientists from different fields among those claimed to be top scientists should be.

One other caveat is that we did not consider the age of investigators, since birth year or year of obtaining graduate degrees is generally not available. Very young investigators may have few papers and thus would be at a disadvantage to compete in these indicators against investigators with a long-standing publication record. However, in practice, it is unlikely that one would ever like to compare a first-year postdoctoral fellow against a senior professor for any reason. If insights are needed for the impact of recently published work only, one can limit the evaluation of citation indicators to the impact of papers published in the last few years. It is also well-known that for young scientists who have just published their first few papers, accrued citations don’t give a picture of their full potential [3]. Other approaches are needed to understand the potential of very young scientists [41].

We should also acknowledge that single citation indicators may have the advantage of being easier to understand and more transparent than composite indicators. For example, it is relatively straightforward to see whether a single indicator such as total citation count is very high because of a single paper with extreme citations. Conversely, a composite indicator requires more complex calculations, and the impact of single extremely-cited publications may be more difficult to discern. In some cases, the distinction between composite and non-composite indicators is not straightforward. However, a composite indicator may be more informative than any of its constituents if it captures the strengths of these constituents and does not compound their deficiencies. This may be the reason why it can also outperform single indicators in the ability to identify widely acknowledged major researchers.

Acknowledging these caveats, we argue that instead of using isolated single indicators, insights can be improved if indicators are generated systematically for the entire scientific community, and multiple such indicators are examined in parallel and in composite scores that integrate volume of citations, co-authorship, and author order. All indicators need to be used with knowledge of both their strengths and weaknesses, since none of them is perfect.

Methods

For all citation analyses, we used the Scopus database. An XML version of the complete Scopus database was obtained in September 2014, and these data were parsed and loaded into an SQL database that was used to construct a variety of indicators at the author level. Paper-level data are available since 1900. We have considered all Scopus-indexed papers, regardless of whether they are primary research, reviews, or editorials and comments. For the 2013 citing year, 95.1% of documents cited were articles, conference papers, or reviews, and these accounted for 96.5% of the citations. Thus, so-called non-citable documents play a negligible role.

Scopus author IDs were used for all author-based analyses. We limited the analysis to citations received by scientists from papers published in 2013. A total of 10,730,738 author IDs (scientists) received at least one citation during 2013. For detailed analysis, we considered all scientists who were within the top 30,000 (roughly top 0.28%) in any of the six key indicators of interest: total number of citations received in 2013 (NC), total number of citations received in 2013 to papers for which the scientist is single author (NS), total number of citations received in 2013 to papers for which the scientist in single or first author (NSF), total number of citations received in 2013 to papers for which the scientist is single, first, or last author (NSFL), Hirsch index for the citations received in 2013 (H), and Schreiber co-authorship adjusted index for the citations received in 2013 (Hm). Hm is computed similarly to the H index after the citations of each paper have been divided by the number of authors. We also recorded the number of papers by each scientist published up to and including 2013 (NP), and the respective number of citations per paper in 2013 (CPP).

A total of 84,565 potentially eligible unique author identifiers were thus compiled that scored among the top 30,000 of each of the six key indicators. Of the 10,000 with the largest numbers of publications, we excluded 278 upon scrutiny, since they seemed to reflect cases of polysemy. Some scientists in the list had multiple author identifiers. After merging, a total of 84,116 scientists remained.

For analysis, each of the six indicators listed above was log-transformed, after inspection of the distribution of values. Several evaluations [42–44] have shown that the distributions of asymptotic number of citations to papers published in a single journal, by researchers in a single academic department, or by the same researcher all follow a discrete lognormal transformation. This offers some further justification for our use of log-transformed indicators.

Correlation analysis between the log-transformed indicators uses Pearson correlation coefficients. Analyses were performed for all scientists regardless of field and separately for each of 12 scientific fields (physics, mathematics, computer science, chemistry, earth sciences, engineering, biology/biotechnology, infectious disease, medicine, brain research, health sciences, social sciences). No analysis of indicators was performed for the humanities, since there are very few such papers in Scopus and they receive very few citations. Many scientists publish in multiple scientific fields; however, most of them have a heavy concentration in only one field. Therefore, each of the 84,116 scientists was assigned to only one field based on where he/she had the largest share of his/her published work indexed in Scopus. The median (interquartile range) for the percentage of this largest share was 76.4% (58.2% to 89.5%). The basis for assignment is the UCSD journal classification system [20]. Briefly, 554 disciplines from the UCSD map of science were aggregated into 13 high-level fields based on natural visual groupings within the map. Later, the biology and biotechnology fields were combined, and physics was split into physics and math for this study, since these two fields have widely differing authoring and citing cultures. Humanities is not represented in this analysis of top researchers.

The composite indicator is the sum of the standardized six log-transformed citation indicators (NC, H, Hm, NS, NSF, NSFL). For each standardization, a value of 1 is given to the scientist with the highest raw value for the respective indicator. Log-transformations ensure that there are no major outlier values. For parsimony, we applied equal weights to all six log-transformed indicators included in the composite. However, if, for whatever reason, one or more of these indicators are considered more essential in a particular field, one can weigh them more compared with the others.

Scientists with values for each the six metrics just below the top 30,000 that we selected may still outperform some of the selected 84,116 scientists in their composite score. We estimated what the composite score of a scientist would have been if he/she had barely missed the threshold in all six metrics. Such a scientist would still not be able to outperform the top 14,150 scientists that we have ranked according to the composite score. Therefore, the list of the top 14,150 scientists is all-inclusive, and we have not missed any scientists with higher composite scores (with the very rare exception of scientists with missed split ID records), while at lower ranks there could be additional scientists with such high composites who are not among the 84,116. We used the all-inclusive top 14,150 ranks of the composite score (and, for comparison, also of other metrics) to assess whether they capture in their lists the 47 Nobel laureates in physics, chemistry, physiology/medicine, and economics awarded in 2011–2015. While we used Nobel laureates as a measure of “ground truth” for evaluation of the different measures, there is really no perfect, gold standard on who the best scientists are, and there are limitations about any “ground truth” standard [45].

All presented indicators include self-citations. Results on the percentage of self-citations across all 84,116 scientists pertain to self-citations by the same author rather than self-citations by all authors involved in each paper.

Alphabetic order of authors is expected simply by chance in 1:k! for papers with k authors; e.g., for papers with three authors, the chances of alphabetic order of the authors simply by chance are 1/(3x2x1) = 1/6 = 16.67%.

Supporting Information