http://orcid.org/0000-0003-0716-9872
Figures
Abstract
Systematic reviews represent the core and backbone of evidence-based medicine (EBM) strategies in all fields of medicine. In order to depict a first global sketch of the international efforts in the Cochrane database systematic reviews (CDSR), we analyzed the systematic reviews of the Cochrane database. Our global maps of systematic reviewing offer intriguing structural insights into the world of EBM strategies. They demonstrate that for the CDSR, the UK and Commonwealth countries take the lead position. Since patients, care providers and health systems all over the world benefit from systematic reviewing, institutions in other countries should increase their commitment.
Citation: Groneberg DA, Rolle S, Bendels MHK, Klingelhöfer D, Schöffel N, Bauer J, et al. (2019) A world map of evidence-based medicine: Density equalizing mapping of the Cochrane database of systematic reviews. PLoS ONE 14(12): e0226305. https://doi.org/10.1371/journal.pone.0226305
Editor: Wisit Cheungpasitporn, University of Mississippi Medical Center, UNITED STATES
Received: July 8, 2019; Accepted: November 22, 2019; Published: December 13, 2019
Copyright: © 2019 Groneberg et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The bibliometric data is owned by and have been obtained from the Web of Science database. Therefore, authors are not allowed to share the data publicly of privately. Any researcher with access to the Web of Science database can obtain the data using the methods that the authors used, as described in the paper. Readers that do not have access to Web of Science should contact Clarivate Analytics to obtain a license (https://clarivate.com/webofsciencegroup/solutions/contact-us/). As per Clarivate Analytics terms of use, the authors may also be able to provide limited access to data, subject their agreement.
Funding: The authors received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Background
Systematic reviews constitute the backbone of evidence-based medicine (EBM) in all fields of medicine. Among the different study types that rely upon evidence-based medicine, the Cochrane Database of Systematic Reviews (CDSR) plays a key role in this important field of medicine [1]. Besides the benefits of CDSRs, there has to be some critical issues to be raised. To be named are the lack of comparability, the risks of bias, indirectness, imprecision, and inconsistency [2]. While the CDSR has nearly published 10,000 systematic reviews so far, the underlying architecture has not been analyzed so far using advanced scientometric tools. In order to address this issue and to depict a first sketch of the global landscape of systematic reviews, we used the NewQIS platform, which was established in 2007/8 [3, 4]. So far, about 50 different biomedical entities were analyzed on the basis of NewQIS ranging from infectious diseases such as Ebola to cancer or even health care policy issues [5–7]. To generate maps of systematic reviewing, the Web of Science (WoS) Core Collection database was used since it also allows citation analysis and density equalizing projections, established by Gastner and Newman [8]. All CDSR articles were analyzed and reviewed with a focus on the question arises who dominates systematic reviewing processes from a global viewpoint. Since US institutions are known to play a prominent role in almost every area of medicine [9], it was hypothesized that they also dominate systematic reviewing in the CDSR.
Methods
NewQIS protocol
We used the previously established NewQIS platform to analyze all CDSR publications present in the Web of Science (WoS) Core Collection database. The WoS was chosen because of the ability to analyze citations.
Search protocol
We identified all relevant articles by searching the WoS for the following string: PUBLICATION NAME: Cochrane Database of Systematic Reviews, Refined by DOCUMENT TYPES: review or article, Timespan: all years.
Density-equalizing mapping
Density-equalizing mapping projections (DEMPs) were used: In brief, DEMPs algorithms based on Gastner and Newman’s algorithm [8] were used to re-size country territories with regard to the following variables: 1) number of published items of each country, 2) the number of institutions in each country, 3) the number of citations for each country, 4) the country-specific modified Hirsch index. The original Hirsch (H)-index for authors was defined by Jorge Hirsch, representing a quantifying index (h): h is the maximum number from the total number of articles written by a given author where each one of these h articles have been at least h times cited. Presently, a modified h index was used to quantify citations of countries as previously performed in the NewQIS studies.
Network analysis
A matrix with all participant countries was computed and transformed into a vector graphic. The vector thickness illustrates the numbers of co-operations between two countries. A threshold of at least two collaborations was set.
Contextual factors
Socio-economic data was obtained from the World Factbook [10]. The values of the Gross Domestic Product (GDP) per publishing country were related to publication activities. The number of researchers in full time equivalent (FTE) per mill. inhabitants was extracted from the UNESCO database [11]. Consequently, these number were also set in relation to the number of CDSRs.
Results
General activity
A total of n = 9,765 CDSR articles were identified using the above described search methodology. When these publications were ranked for the highest country activities, the UK was ranked no. 1 with a total of n = 4,261 articles. It was followed by Australia with a total of n = 1.596. At third position, the USA was listed with n = 1,309 articles, followed by Canada (n = 1,094). Active countries were also the Netherlands (n = 699), China (n = 576), Italy (n = 466), Germany (n = 423), New Zealand (n = 376), Brazil (n = 292), Switzerland (n = 260), Denmark (n = 223), India (n = 208), and Ireland (n = 201). When all countries are analyzed using density equalizing mapping, a global landscape appears, which is largely distorted towards Europe with the UK, Northern America with Canada and the USA, and Australia (Fig 1A). While Africa nearly disappears, Eastern Asia is visible due to Chinese contributions (n = 576).
A) Density equalizing map projection of the number of articles per country. B) Density equalizing map projection of the number of citations.
Quality parameters
To approach (semi-)qualitative aspects of CDSR research, citation analysis was performed using a variety of benchmarks. In the absolute citation ranking, the UK was ranked no.1 (c = 63,974 citations), followed by Australia (c = 25,894), Canada (c = 21,213), the US (c = 18,971), the Netherlands (c = 9,559), New Zealand (c = 6,884), Italy (n = 6,602), Germany (c = 6,587), Switzerland (c = 4,261), and Norway (c = 3,566), China (c = 3,555), and Denmark (c = 3,547). Density equalizing mapping leads to a distortion of the world map (Fig 1B) that largely follows the total publication activity mapping. The citation rate analysis shows a different ranking: The highest number of citations per review (cr) is found for Norway (cr = 32.41), followed by Canada (cr = 19.39), and New Zealand (cr = 18.3), underlying a threshold of at least 30 CDSRs. The UK has a citation rate of about cr = 15 and the USA of cr = 14.5 (Fig 2). The country-specific h-index (hI) analysis lists the UK at hI = 89, followed by Australia (hI = 69), Canada (hI = 66) and the US (hI = 60) (Fig 2).
A) Density equalizing map projection of the citation rates. B) Density equalizing map projection of the country-specific h-indices.
International networking
Networking analysis of all articles shows that there are strong bonds between the UK (1,937 out of all 4,261 UK reviews are performed as international collaborations) and Australia, the US and Canada. Interestingly, the US has a much higher percentage of collaborative articles (903/1,309 = 69%) than the UK (45.5%) or Australia (54.6%). There are also countries such as Sweden (94.3%), which publish nearly all reviews within an international collaboration (Fig 3).
Contextual factors
When the economic strength of the highly industrialized countries is related to the systematic reviewing activities of these countries. The following data are obtained (Table 1): New Zealand publishes a calculated 2.151 reviews per 1000 billion US$ Gross Domestic Product (GDP), the UK 1,528 reviews per 1000 billion US$ GDP and Australia 1,342. By comparison, the USA publishes about 71 systematic reviews per 1000 billion US$ GDP, Germany 106 and China 27 systematic reviews per 1000 billion US$ GDP, respectively. Secondly, the number of reviews can also be related to the number of researchers (FTE) per mill. inhabitants (researchers per capita): Here, the UK leads with 0.97 reviews per researchers per capita, followed by India with 0.96 reviews per researchers per capita, and Colombia with 0.83 reviews per researchers per capita. China has 0.47 reviews per researchers per capita ranking 10th. The US is ranked 12th with 0.31 reviews per researchers per capita.
GDP Gross Domestic Product; bn billion; USD United States Dollar; HI high-income country; UMI upper-middle-income country; LMI lower-middle-income country, FTE full time equivalents. Sources: GDP (bn USD) [10], Number of Researcher (FTE) per mill. inhabitants [11],–no data available.
In consideration of the socio-economic parameters GDP and total population of the publishing countries a clear economic related distribution appears (Fig 4).
Discussion
The area of evidence-based medicine has increasing importance in all areas of medicine. We analyzed the Cochrane database systematic reviews using the NewQIS platform and density equalizing approaches and found striking contrasts to the usual global research patterns: A large study analyzed a number of 5,527,558 publications and showed the usually clear dominance in biomedicine of the USA. The USA was by far the most productive country with nearly 1.9 million reviews. Japan ranked second with 573,473 reviews. The third most active country was Germany (n = 444,775), followed by the UK (n = 415,499). When oncology is focused, a similar lead position for the US is present in numerous cancers including ovarian cancer [6], breast cancer [12], pancreatic cancer [13] or laryngeal cancer [14], as previously demonstrated. In contrast, our findings show the UK being by far the leading country within the area of systematic reviews. Thus, our primary hypothesis needs to be rejected. Out of the 9,765 CDSRs included in our analysis, 4,261 originated from the UK and just 1,309 articles from the USA. A further interesting finding is that China appears to be strong with 576 reviews, in contrast with Japan with just 81 reviews. Even Malaysia (n = 87) appeared to be more active in this area of the world. Noteworthy, Japanese research activity is currently regarded to be decreasing in contrast to China [15]. As reported by N. Philips [16], Japan's Council for Science, Technology and Innovation conceded in on its Fifth Science and Technology Basic Plan (2015) that its world standing in science and technology was falling.
The total number of systematic reviews might represent a good marker for overall activities in the field of clinical medicine, in which evidence based diagnosis and treatment are of great importance [17]–but how meaningful are they? This can be approximated by citation analysis. Total numbers of citations largely follow the total numbers of publications. More detailed information can be obtained by assessing the citation rates and country-specific h-indices. Here, the highest citation rate is present for Norway, followed by Canada and New Zealand. A similar pattern with Scandinavian countries in the leading position has also been discovered in other studies [18]. To the same extend, Scandinavian countries such as Sweden seem to have a very high degree of collaborative research–we found that 94.3% of all Swedish Cochrane articles were published as international collaboration. It is noteworthy that the US also have a relatively high proportion of international collaborations (69%).
Since systematic reviews constitute the backbone of evidence-based medicine, highly industrialized countries should allocate reasonable resources to the advancement of science in this field. Our calculations show that the Commonwealth countries New Zealand (calculated 2.151 reviews per 1000 billion US$ GDP), the UK (1.528 reviews per 1000 billion US$ GDP). and Australia (1.342 reviews per 1000 billion US$ GDP) are very active with the US just publishes about 71 systematic reviews per 1000 billion US$ GDP. Again, Japan has a very poor rate with 16.42 systematic reviews per 1000 billion US$ GDP. The UK also lead when the global Cochrane database systematic review activity is related to the countries´ number of researchers per capita. With India (n = 208) and Colombia (n = 73), two countries that have no high-income status are ranked next. The most collaborations of the Indian reviews were worked out with the UK. Colombia mostly worked together with Spain. Both connections are certainly caused by the historical connection and the common language.
What is the reason for this imbalance within highly industrialized countries such as the USA and highly industrialized Commonwealth countries such as the UK or Australia? One reason might be found in the history of the Cochrane database of systematic reviews. Cochrane is named in honor of Professor Archibald Cochrane, a Scottish physician who pioneered the field of evidence-based medicine. His visions were the founding principles that led to the opening of the first Cochrane Centre in Oxford, UK in 1992 and the founding of the Cochrane Collaboration in 1993 [19]. A second reason might be the geographical distribution of Cochrane Review Group editorial bases and the British origin of most of the editors. However, countries without review group editorial bases in countries such as China, Switzerland, Brazil, Ireland, or India are much more active than countries with bases in countries such as France or Portugal.
Conclusions
The established global maps in this study offer intriguing insights into the world of Cochrane systematic reviews that form the basis of evidence-based medicine. They clearly show that the UK and Commonwealth countries take the lead position in this extremely important field of medicine. Patients, care providers and health systems all over the world benefit from this commitment. With regard to the unusual non-leading role of the US and the great potential of US institutions, clinicians and scientists from the US should foster their activities in this field. Future studies using novel approaches might also assess gender distribution of Cochrane authors [20, 21].
References
- 1. Starr M, Chalmers I, Clarke M, Oxman AD. The origins, evolution, and future of The Cochrane Database of Systematic Reviews. International journal of technology assessment in health care. 2009;25 Suppl 1:182–95. Epub 2009/06/19. pmid:19534840.
- 2. Chevret S, Ferguson ND, Bellomo R. Are systematic reviews and meta-analyses still useful research? No. Intensive Care Med. 2018;44(4):515–7. Epub 2018/04/18. pmid:29663047.
- 3. Groneberg-Kloft B, Fischer TC, Quarcoo D, Scutaru C. New quality and quantity indices in science (NewQIS): the study protocol of an international project. J Occup Med Toxicol. 2009;4:16. Epub 2009/06/27. pmid:19555514; PubMed Central PMCID: PMC2708171.
- 4. Groneberg-Kloft B, Quarcoo D, Scutaru C. Quality and quantity indices in science: use of visualization tools. EMBO reports. 2009;10(8):800–3. Epub 2009/08/04. pmid:19648952; PubMed Central PMCID: PMC2726685.
- 5. Quarcoo D, Bruggmann D, Klingelhofer D, Groneberg DA. Ebola and Its Global Research Architecture—Need for an Improvement. PLoS neglected tropical diseases. 2015;9(9):e0004083. Epub 2015/09/26. pmid:26406894; PubMed Central PMCID: PMC4583251.
- 6. Bruggmann D, Pulch K, Klingelhofer D, Pearce CL, Groneberg DA. Ovarian cancer: density equalizing mapping of the global research architecture. International journal of health geographics. 2017;16(1):3. Epub 2017/01/15. pmid:28086974; PubMed Central PMCID: PMC5237222.
- 7. Schreiber M, Klingelhofer D, Groneberg DA, Bruggmann D. Patient safety: the landscape of the global research output and gender distribution. BMJ open. 2016;6(2):e008322. Epub 2016/02/14. pmid:26873042; PubMed Central PMCID: PMC4762108.
- 8. Gastner MT, Newman ME. Diffusion-based method for producing density-equalizing maps. Proceedings of the National Academy of Sciences of the United States of America. 2004;101(20):7499–504. Epub 2004/05/12. pmid:15136719; PubMed Central PMCID: PMC419634.
- 9. Groneberg-Kloft B, Scutaru C, Kreiter C, Kolzow S, Fischer A, Quarcoo D. Institutional operating figures in basic and applied sciences: scientometric analysis of quantitative output benchmarking. Health research policy and systems. 2008;6:6. Epub 2008/06/17. pmid:18554379; PubMed Central PMCID: PMC2459159.
- 10.
CIA. The World Factbook, URL: https://www.cia.gov/library/publications/the-world-factbook/ (accessed Jan 2019). 2017.
- 11.
UIS.stat. Science, technology and innovation, URL: http://data.uis.unesco.org (accessed Aug 2019). 2013–18.
- 12. Glynn RW, Scutaru C, Kerin MJ, Sweeney KJ. Breast cancer research output, 1945–2008: a bibliometric and density-equalizing analysis. Breast cancer research: BCR. 2010;12(6):R108. Epub 2010/12/24. pmid:21176219; PubMed Central PMCID: PMC3046453.
- 13. Schoffel N, Krempel M, Bundschuh M, Bendels MH, Bruggmann D, Groneberg DA. Pancreatic Cancer-Critical Examination of the Global Research Architecture and Recent Scientific Developments. Pancreas. 2016;45(10):1378–85. Epub 2016/10/18. pmid:27518470.
- 14. Glynn RW, Lowery AJ, Scutaru C, O'Dwyer T, Keogh I. Laryngeal cancer: quantitative and qualitative assessment of research output, 1945–2010. The Laryngoscope. 2012;122(9):1967–73. Epub 2012/06/01. pmid:22648552.
- 15. Phillips N. A close look at China's rise. Nature. 2017;545(7655):S39. Epub 2017/05/26. pmid:28538710.
- 16. Phillips N. Nature Index 2016 Japan. Nature. 2016;531(7594):S97. Epub 2016/03/18. pmid:26982025.
- 17. Djulbegovic B, Guyatt GH. Progress in evidence-based medicine: a quarter century on. Lancet. 2017;390(10092):415–23. WOS:000405911300030. pmid:28215660
- 18. Bruggmann D, Lohlein LK, Louwen F, Quarcoo D, Jaque J, Klingelhofer D, et al. Caesarean Section—A Density-Equalizing Mapping Study to Depict Its Global Research Architecture. International journal of environmental research and public health. 2015;12(11):14690–708. Epub 2015/11/26. pmid:26593932; PubMed Central PMCID: PMC4661674.
- 19. Dickersin K, Manheimer E. The Cochrane Collaboration: evaluation of health care and services using systematic reviews of the results of randomized controlled trials. Clinical obstetrics and gynecology. 1998;41(2):315–31. Epub 1998/07/01. pmid:9646964.
- 20. Bendels MH, Bruggmann D, Schoffel N, Groneberg DA. Gendermetrics.NET: a novel software for analyzing the gender representation in scientific authoring. J Occup Med Toxicol. 2016;11:43. Epub 2016/09/22. pmid:27651823; PubMed Central PMCID: PMC5024497.
- 21. Bendels MHK, Wanke E, Schoffel N, Bauer J, Quarcoo D, Groneberg DA. Gender equality in academic research on epilepsy-a study on scientific authorships. Epilepsia. 2017;58(10):1794–802. Epub 2017/09/01. pmid:28857143.