Introduction

Studies focusing on the international comparative performances of countries or territories have a long tradition. Leydesdorff and Wagner [1] ascertain that “the USA is still outperforming all other countries in terms of highly cited papers and citation/publication ratios, and it is more successful than the EU in coordinating its research efforts in strategic priority areas like nanotechnology” (p. 23). The Science and Engineering Indicators of the US National Science Board [2] report developments in international and US science and technology based on a comprehensive data base. This report shows that the combined share of published articles of researchers in the EU and the USA “decreased steadily from 69% in 1995 to 58% in 2009. In little more than a decade, Asia’s world article share expanded from 14% to 24%, driven by China’s 16% average annual growth dominated world article production” (p. 0–10). With regard to citation impact, the analyses of the National Science Board [2] reveal that “U.S. articles continue to have the highest citation rates across all broad fields of S&E” (p. 5–43). On the excellence level, the USA has published 76% more articles than expected among the top−1% most frequently cited articles in 2010; scientists from the EU have published top−1% articles 7% less than expected.

Two recently published reports, although based on different data bases (Web of Science, WoS, of Thomson Reuters and Scopus of Elsevier) came to the same conclusion: In comparison with other prolific countries, the UK ranks top (first and second, respectively) in citations per paper [3]. According to Adams [4] “UK performance is on a rising trajectory, whereas the USA research base has at best plateaued in performance and – on some estimates – is now in decline … the UK has now overtaken the US on average and … remains well ahead of Germany and France. Japan has dropped well behind its G7 comparators and its average impact is now similar in performance to China” (p. 8). Whereas Adams [4] used WoS data, Elsevier [5] reports the following results based on Scopus data: “The UK’s field-weighted citation impact … is well above the world benchmark, but is slightly lower than that of the US, ranking it 2nd in the G8 and in the comparator group but with a growth rate of 1.1% compared to the US’s −0.5% per year since 2006” (p. 28).

In this study, the citation impact analyses of Adams [4] and Marshall and Travis [3] are extended by (1) the consideration of a longer publication window (1981 to 2010 instead of 1991 to 2010), (2) a differentiation in four broad subject areas (instead of an analysis across all areas and some selected fields), and (3) the use of statistical procedures to investigate the citation impact trends across the years. Following Adams [4] and Marshall and Travis [3], the analyses of this study refer to the six most prolific countries: China, Japan, France, Germany, the United States, and the UK. Adams [4] justifies the selection of these six countries as follows: they “now account for almost two-thirds of the global total of research publications indexed in Thomson Reuters Web of Knowledge^SM. France and Germany are the UK’s major partners in the European Research Area. The USA and Japan are its key collaborators in other regions. China is the new dynamic factor in the global geography of science, as we have reported previously. Its rapid rise demonstrates a different model in research policy and demands a reanalysis of the certainties that underpinned the trans-Atlantic research axis of the last half-century” (p. 3).

This study is based on data from the relatively new InCites tool of Thomson Reuters, which facilitates national comparisons across long time periods using normalized citation impact values.

Methods

Results

Figure 1 shows, for six countries, citation impact values calculated relatively to the four subject areas. Two points should be considered in the interpretation of the results: (1) Larger citation impact differences between two following years for one country can rather be the effect of lower paper numbers than of significant performance differences. Since authors from the USA publish more papers than all other countries, the variation is accordingly low. (2) The longer the citation window, the more reliable the performance estimation for a paper is [15]. Therefore, the most recent publication years in Figure 1 should be interpreted with due care (see, e.g., the outliers for 2010 in the citation impact values in the case of agricultural sciences).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 1. Citation impact of six countries (China, UK, Germany, France, Japan, and USA) calculated relatively to four subject areas (natural sciences; engineering and technology; medical and health sciences; and agricultural sciences).

For each country, the Spearman’s rank-order coefficient for the correlation between publication year and citation impact is given; an asterisk signifies a statistically significant coefficient (the “true” coefficient, the population coefficient, is likely to be above 0). A high correlation coefficient indicates an increasing or decreasing trend in citation impact values. Source: InCites^TM Thomson Reuters [9].

https://doi.org/10.1371/journal.pone.0056768.g001

If we compare the results for the different subject areas in Figure 1 with the figures published by Marshall and Travis [3] and Adams [4] across all subject areas, the correspondence is largest for the natural sciences. In this subject area we see a similar pattern with a sharp increase in the impact of papers from the UK during the last four years. The UK is the only country which surpasses the performance of the USA across the last thirty years. It is interesting to see that in the case of the natural sciences all countries have very high correlation coefficients with the exception of the USA (China: r_s = .89, UK: r_s = .84, Germany: r_s = .98, France: r_s = .98, Japan: r_s = .77). This points to increasing trends of citation impact values. This trend is especially visible for the European countries (France, Germany, and the UK). On a low performance level, the trend is also visible for China but it has been declining for the last two years. Studies in the next few years will show whether this trend will continue or not.

For engineering and technology, a different pattern is visible in Figure 1 than for natural sciences. Not only Germany (r_s = .96) and China (r_s  = .65), but also the USA (r_s = .70) show an increasing citation impact trend on a relatively high level (these three countries have the highest correlation coefficients in the graph). The reverse trend is visible for Japan (r_s = −.49, but this coefficient is statistically not significant). In the next graph in the figure, the medical and health sciences show significantly increasing trends for Germany (r_s = 1.0) and France (r_s = 1.0) but a decreasing trend for the USA (r_s = −.64). However, the decreasing trend of the USA occurs on a high impact level across the past thirty years, whereas Germany and France improved their impact from a very low performance level in the 1980s to a very high level (25% above the subject area average) during the last two years. Agricultural sciences – as the last subject area considered in Figure 1 – are characterized by increasing trends of China (r_s = .66), France (r_s = .91), and Germany (r_s = .99), but a decreasing trend of the USA (r_s = −.87). Across the whole time period, the UK performs on a very high level of at least 25% above the subject-area average.

Table 1 shows the minima, maxima, means, and standard deviations of citation impact values for the different subject areas and countries. Although the USA has in most of the subject areas in Figure 1 a statistically significant trend (increasing or decreasing), the standard deviations are in all subject areas very small for this country (SD<.1). This means that there is only a low level of variability between the publication years. A different situation is indicated for Germany (SD is between .15 and .25) and China (SD is between .13 and .16). These countries have a high level of variability and a dynamically increasing trends (as Figure 1 points out).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Minimum, maximum, mean, and standard deviation of citation impact values by subject area and country (N is the number of publication years included in the analysis).

https://doi.org/10.1371/journal.pone.0056768.t001

Discussion

In broad agreement with other studies (described in the Introduction section), this study shows an increasing trend of normalized citation impact values for France, the UK and especially for Germany across the last thirty years in four subject areas. For France and Germany, this may also be an effect of internationalization. Scientists in both countries have published increasingly in English. Thus, one measures not only scientific performance, but also a shift of performance. The citation impact of papers from China is still on a relatively low level (mostly below the world average), but the country follows an increasing trend line.

The USA shows a relatively stable pattern of high citation impact values across the years. With small impact differences between the publication years, the US trend is increasing in engineering and technology but decreasing in medical and health sciences as well as in agricultural sciences. Similar to the USA, Japan follows increasing as well as decreasing trends in different subject areas, but the variability across the years is relatively small. In most of the years, papers from Japan perform below or approximately at the world average in each subject area.

As a first limitation of this study, the use of arithmetic average rates of citations should be mentioned. There are dangers in focusing on the measures of central tendency: in the face of non-normally distributed citation data, the arithmetic mean value can give a distorted picture [16], “and it is a rather crude statistic” ([17], p. 1). As the distribution of citation frequencies is usually right skewed, distributed according to a power law, arithmetic average rates of citations show mainly where papers with high citation rates are to be found [18].

Particularly in bibliometric analysis the use of percentile rank scores for evaluative purposes is very advantageous, as no assumptions have to be made about the distribution of citations [19]. Although some national comparison studies additionally report results based on percentiles [2], the most commonly used approach is a presentation of (normalized) average citation rates. Since InCites does not provide percentiles in the global comparison module, this study followed this approach. For future national comparisons, however, studies based on percentiles are very desirable (see here, e.g., the new Leiden ranking at http://www.leidenranking.com/ for the comparison of institutions) [20].

A second limitation of our study concerns the use of four broad fields to normalize the results. Performing normalization at the level of these broad fields means that a high citation impact of a country in a particular field may be a consequence of the fact that most of the activity of the country in this field takes place in subfields with a relatively high citation density. So differences in citation impact may merely reflect differences in the research portfolios of countries (rather than differences in countries’ actual scientific impact). Furthermore, an increase (or decrease) of a country’s citation impact in a particular field may also be a consequence of a shift of the country’s research activities from subfields with a lower (higher) citation density to subfields with a relatively higher (or lower) citation density.

Conclusion

With the global comparisons module, InCites offers a unique opportunity to study the normalized citation impact of countries. Although the approach has its limitations, (1) the consideration of a long publication window (1981 to 2010), (2) a differentiation in (broad) subject areas, and (3) the use of statistical procedures allow for an insightful investigation of national citation impact trends across the years.