https://orcid.org/0000-0003-0196-7224
Figures
Abstract
Aims
Over the last two decades, the existence of an open access citation advantage (OACA)—increased citation of articles made available open access (OA)—has been the topic of much discussion. While there has been substantial research to address this question, findings have been contradictory and inconclusive. We conducted a systematic review to compare studies of citations to OA and non-OA articles.
Methods
A systematic search of 17 databases attempted to capture all relevant studies authored since 2001. The protocol was registered in Open Science Framework. We included studies with a direct comparison between OA and non-OA items and reported article-level citation as an outcome. Both randomized and non-randomized studies were included. No limitations were placed on study design, language, or publication type.
Results
A total of 5,744 items were retrieved. Ultimately, 134 items were identified for inclusion. 64 studies (47.8%) confirmed the existence of OACA, while 37 (27.6%) found that it did not exist, 32 (23.9%) found OACA only in subsets of their sample, and 1 study (0.8%) was inconclusive. Studies with a focus on multiple disciplines were significantly positively associated with finding that OACA exists in subsets, and are less associated with finding that OACA did not exist. In the critical appraisal of the included studies, 3 were found to have an overall low risk of bias. Of these, one found that an OACA existed, one found that it did not, and one found that an OACA occurred in subsets.
Citation: Langham-Putrow A, Bakker C, Riegelman A (2021) Is the open access citation advantage real? A systematic review of the citation of open access and subscription-based articles. PLoS ONE 16(6): e0253129. https://doi.org/10.1371/journal.pone.0253129
Editor: Sergi Lozano, Universitat de Barcelona, SPAIN
Received: February 19, 2021; Accepted: June 1, 2021; Published: June 23, 2021
Copyright: © 2021 Langham-Putrow et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data are available from https://doi.org/10.13020/357e-ek33.
Funding: The authors received no funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Scholarly publishing relies on the gift economy of academia in which authors provide content for free and editors and peer reviewers donate their time to review that content [1, 2]. Prior to the development of the internet, the costs of printing, shipping, and other operations were offset by user and institutional subscriptions. As publishing became more commercialized in the second half of the 20th century, journal subscription prices began increasing, outpacing both inflation and university budgets [3–7]. Despite the shift from print to electronic, prices have only risen.
Various modes of open access (OA) publishing have emerged over the last 30 years to replace or augment subscription-based (“toll-access”) publishing. Green OA refers to materials that are made openly available through archiving in an OA repository or other legally permissible venue. Depending on journal policies and the decision-making of the depositor, a green OA version could be an early version of a manuscript or a copy-edited, post-peer review version. Historically, Gold OA was used to refer to “OA delivered by journals, regardless of the journal’s business model” [8 p. 52]. Gold OA can be further broken into gold journals (fully OA journals, which may or may not charge article processing charges (APCs)) and hybrid journals (which charge authors an APC to make their article open in a subscription journal).
Over the last 20 years, researchers and publishers have introduced a multitude of categories of OA beyond green and gold. These include "diamond" or "platinum" OA (i.e., OA in the place of publication but without an APC charged [9, 10]), “bronze” OA (i.e., articles that publishers make available to readers for free, but do not have an OA license [11]), and "grey" OA (i.e., authors make their content available via upload to an academic social network or personal website, potentially without consideration of previously signed copyright transfer agreements). Others have used the term “black OA” to mean access facilitated by the posting of materials to pirate sites, such as SciHub [12]. It is important to note that this type of access does not necessarily involve the consent or participation of the author or publisher; as such, it is questionable whether this is truly a mode of OA. Because terminology has developed over time, we refer only to green OA versus gold OA, as these are the most broadly adopted categories of OA.
The potential for OA publication to increase citations was first articulated in an empirical study of computer science in 2001 [13]. Since 2001, there have been many studies both supporting and refuting the existence of an open access citation advantage (OACA). This nomenclature is consistently used in the literature, even in those studies reporting negative or null findings.
Although many relevant primary studies and some secondary studies exist on this topic, the influence of OA on citation remains unclear, particularly when considering the range of disciplines, data sources, publishing models, and other contextual factors. In a critical review, Davis and Walters [14] found a notable increase in the number of downloads for articles published OA but no clear evidence of OACA. Additionally, Davis and Walters found that studies finding citation advantages for OA failed to “adequately control for confounding variables” [14 p. 208]. In a narrative review, Turk [15] noted that OACA studies varied greatly in terms of data collection platforms used (e.g., Scopus, Google Scholar), differing publication types (e.g., journal articles, conference papers), and different disciplines analyzed, which potentially influenced the variety of findings.
In an update to the literature review by Davis and Walters [14], Lewis [16] concluded that only a “few of the authors…actually claim causation”, and instead pointed to the insight gained by examining correlation between OA and a citation advantage (CA). Lewis argued that more research is needed "to prove a causal relationship between OA and CA" [16 p. 59]. Like Turk [15] and others, Lewis noted that differences existed in studies targeting certain disciplines, and future research across all disciplines would "provide a wider array of evidence for the occurrence of field-specific OACA and therefore of a more widespread OACA" [16 p. 59].
A 2007 critical review by Craig et al. looked at three non-exclusive postulates when examining citation differences between both OA and non-OA articles: that the advantage is due to 1) OA status of the article; 2) selection bias (i.e., authors select their best works to share openly), and 3) early view effects (i.e., the extra time that an article is available) [17]. They pointed to studies in which controlling for one postulate (e.g., early view) revealed that citation counts can be explained by another postulate (e.g., selection bias) and advised that more rigorous methods were needed in future studies to look at causation [17].
Hua et al. [18] published a narrative review specific to OA concepts in dentistry in which they go beyond OACA and venture into other topics such as research waste of inaccessible research. Regarding OACA, they looked at nine studies across many disciplines and found variation of methods and materials, but ultimately the citation advantage ranged from -5% to 83% [18]. Turk [19] published an overview on OA, pertaining only to medical articles, looking at OA factors in which studies of possible OACA mostly found a citation advantage, but articles reporting randomized controlled trial (RCT) results revealed no OACA. Turk [19] indicated that other factors may influence a citation advantage for OA such as discipline, impact factor of journals studied, and the early view effect and selection bias Craig noted in 2007 [17].
Although these reviews provide valuable context, they were not as comprehensive and the methods were not transparently reported to the level of rigor needed to ensure reproducibility or replicability. The present study is unique in that there are no other existing systematic reviews or meta-analyses comparing citations of OA and non-OA articles. We address this need and critically appraise the existing studies to better understand the totality of the evidence while also identifying future areas of research. These findings could be of value to researchers when making evidence-based decisions about which mode of publishing—OA or otherwise—may most closely align with some of their objectives when choosing a publication venue.
Methods
In accordance with best practices for systematic reviews, a combination of controlled vocabulary and natural language searching was used to comprehensively capture relevant studies (S1 Checklist). The search included iterations of terms for OA and citation advantage. A primary database search strategy was developed in Ovid Medline and then translated to the following databases reflecting a range of disciplines: Library & Information Science Source (LISS), Library Information Science & Technology Abstracts (LISTA), ERIC, Academic Search Premier, and Business Source Premier via EBSCO; PubMed; Embase, CAB Abstracts and PsycINFO via Ovid; Scopus; Web of Science Core Collection; Compendex; Sociological Abstracts, EconLit, and Dissertations & Theses Global via ProQuest; and SHARE. A full search strategy of our primary database is available in S1 Appendix.
As indicated in our inclusion and exclusion criteria, this project targeted only studies published after 2001, which is the date of the earliest known article on citation advantage of OA [13]. By study, we refer to those items that were retrieved, screened, and included in this project. By articles, we refer to the subject of those studies: the OA and non-OA materials for which citation counts were being gathered. The searches were initially conducted in July 2019 and were rerun in November 2020. We did not include any filters to limit language, study design, or publication type. To ensure that no potentially relevant items were overlooked, hand searching of reference lists was undertaken. In cases where limited or unclear data were available, authors were contacted for clarification and to seek additional information. Any non-English language content was translated into English for screening and, if applicable, extraction and assessment. The protocol was registered in Open Science Framework (osf.io/p2a7q).
Two screeners independently reviewed every title and abstract, applying previously determined inclusion and exclusion criteria. In order to be included, studies needed to contain a direct comparison between a sample of OA articles and non-OA articles, with citation counts as an outcome, and have been published since January 2001. We excluded studies that did not report citation count data, such as narrative reviews, editorials, or opinion pieces; did not include citation counts as an outcome; and/or did not directly compare OA versus non-OA articles. For example, a study that reported on citation counts for a set of OA articles against a different set of OA articles would be excluded; a study that compared citations to gold OA articles with citations to subscription articles would be included even if the authors did not take into account the possible overlap of green OA articles within either set.
Screening was completed using Rayyan, a web application that facilitates independent screening [20]. Where there were discrepancies, the conflicts were resolved via discussion or by a third person tie-breaker when necessary. Full-text screening was also completed by two independent screeners, and reasons for exclusion were recorded and are reported in Fig 1.
PRISMA flowchart showing number of studies at each step of the selection process.
The data extraction form was developed and piloted by three researchers. Data elements extracted included information about the study, including the publication type of the study, as well as the type of metrics used to measure citations, mode of OA (green, gold, or a combination), and overall findings of the study. With respect to categorization of the mode of OA, we relied on the study authors’ reporting of OA status. As noted in the introduction, more nuanced definitions of OA exist. However, these broader categories are more commonly used and have been in effect for longer, thus making them more applicable for the purposes of this study. As hybrid and gold OA were sometimes conflated within the included studies, these categories are combined in this project. Similarly, some authors used “green OA” to describe whether a version of the article was findable through a search engine (i.e., available in an OA repository or academic social network but not through a pirate site).
We extracted data elements describing the metrics used and the findings of the study. These elements were categorized into larger groupings for further analysis. Metrics were grouped into mean, median, total, and other, representing the most common methods of measurement, while overall findings were grouped as OACA exists, OACA does not exist, or OACA exists in subsets of the data. We extracted the disciplinary subject of each study, and subsequently mapped these subjects to the six major subject codes outlined by the Organisation for Economic Cooperation and Development (OECD)’s Field of Science and Technology Classification [21]. Use of OECD overcomes challenges in using database or publisher-specific resources, such as taxonomies from Elsevier or Clarivate, while providing a framework applicable to a broad range of disciplines and document types. The disciplinary subject of each study was defined as the disciplinary subject of the articles included in the study, rather than the discipline of the journal that published the study. Where the subject matter reflected more than one classification, the item was marked as “multiple disciplines.” Data extraction was completed independently by two researchers, as was risk of bias assessment. The Evidence-based Librarianship (EBL) critical appraisal tool developed by Glynn [22] was used, as this tool was developed for library and information science literature and therefore would be the most broadly applicable to the literature included in this study. Where there were any discrepancies in the data extracted or the risk of bias assessment, these discrepancies were resolved through discussion or by a third party when necessary.
Following data extraction, heterogeneity of the studies was assessed using the I2 statistic to determine if quantitative synthesis would be appropriate. I2 was calculated using RevMan 5.3 [23]. An I2 of 90% was found among studies with sufficient data for pooling. As such, the studies were found to be significantly heterogeneous and quantitative synthesis was deemed inappropriate. Although we were not able to pool the data in a meta-analysis, chi-squared tests were conducted to analyze the associations between findings, as categorical variables, and their associations with characteristics of studies. These tests were performed using R 3.6.0 [24].
Results
5,697 results were retrieved through database searching, and 47 additional items were retrieved through handsearching. Once duplicates were removed, 4,019 items were subject to title-abstract screening. Of these, 3,677 were excluded, leaving 342 items for full-text assessment. 208 items were excluded at the full-text screening phase, resulting in 134 studies that were ultimately included in this analysis [11, 25–157].
Of the 134 items, more fully described in Table 1, 132 were non-randomized studies while 2 were randomized. The studies most frequently addressed multiple disciplines (n = 45, 33.6%), followed by Medical and Health Sciences (n = 36, 26.9%), Natural Sciences (n = 22, 16.4%), and Social Sciences (n = 21, 15.7%). Detailed information on included studies is available in S1 Dataset.
The studies used a range of data sources. Most frequently, data were retrieved from Clarivate/Web of Science (n = 82, 61.2%), followed by Scopus (n = 38, 28.4%) and Google Scholar (n = 29, 21.6%), including Google Scholar data accessed through Publish or Perish [158]. 19 (14.2%) studies used other sources of data, and 24 (17.9%) studies used more than one source of data.
The majority of the studies (n = 64, 47.8%) found that there was OACA while 27.6% (n = 37) found that it does not exist. 23.9% (n = 32) of studies found that, while there was no overall OACA, there was an advantage in subsets, such as for specific journals, certain periods of time, or subdisciplines. One study (0.8%) was inconclusive in its findings. These studies based their conclusions on a range of measures.
The characteristics of the 134 studies and their associated findings are described in Table 2.
Mode of OA
There was no statistically significant relationship between mode of OA and findings. Findings that OACA existed were most common regardless of mode of OA: 72% (18/25) of studies focused on green OA, 39.6% (21/53) of those focused on gold OA, 39.4% (13/33) of those focused on both green and gold OA, and 52.2% (12/23) of items that did not specify mode found that OACA existed. While 39.6% of studies focused on gold OA found OACA, gold studies account for 51.4% (19/37) of all studies that found no OACA existed.
Discipline
We found a statistically significant relationship between discipline and OACA finding (X2(18,134) = 42.763,p < 0.001). Studies addressing multiple disciplines were positively associated with finding that OACA sometimes exists, and are less associated with finding that OACA did not exist. While not statistically significant associations, findings of the existence of OACA were more common in Social Sciences (66.7%, 14/21), Medical and Health Sciences (52.8%, 19/36), and Natural Sciences (45.5%, 10/22). In both Agricultural Sciences and studies that did not specify discipline, 25% of studies found that OACA existed, while OACA was not found in 75% of studies. Of the 32 studies that found OACA was present in subsets, 20 (62.5%) addressed multiple disciplines.
Risk of bias
The EBL critical appraisal tool considers the possibility of bias in four domains: (1) population, (2) data collection, (3) study design, and (4) results [22]. Of the 134 studies assessed, 3 were found to have an overall low risk of bias. Of the three studies that had low overall risk of bias, one found that an OACA existed, one found that it did not, and one found that an OACA occurred in subsets.
All studies were found to have high risk of bias in the domain of population, 118 (88.1%) studies were found to have high risk of bias in the results domain, 107 (79.9%) were high risk of bias in data collection, and 91 (67.9%) had high risk of bias in study design. A summary of risk of bias assessment can be found in Fig 2, while an itemized risk of bias assessment for every component study can be found in S1 Dataset. There were no statistically significant associations between overall risk of bias and findings.
Summary of overall risk of bias findings (low or high) and in the population, data collection, study design, and results domains.
Publication types
The majority of studies were published as journal articles (107/134), regardless of findings. Of those studies published as journal articles, the majority were published in subscription or hybrid journals (75/107). The remaining studies were made available through conference presentations (15/134), as working papers or preprints (8/134), or as theses or dissertations (4/134). There were no statistically significant relationships between publication type and findings of the studies.
Discussion
The concept of OACA has been discussed since before the various declarations and definitions of OA [159–161]. In 2001, Lawrence reported that “free online availability substantially increases a paper’s impact.” [13 p. 521] As seen in our study, the topic has been the subject of numerous studies over the last 20 years.
Due to the heterogeneity of the sample of studies, a meta-analysis was not possible. We noted variations in the mode of OA, the types of publications, and the disciplines studied. There was also variation in the methods used and metrics reported. This suggests the need for reporting guidelines for bibliometric studies. At minimum, studies should report clearly on the mode of OA, the number of OA and non-OA articles, comparability of the data used, the time frame (citation window) and a justification for this choice, a common metric (e.g., mean and median citations per article), and confounding factors and how they are addressed.
With respect to the OA mode, studies should provide their definition of the mode of OA being considered. This may become even more necessary as researchers continue to create and implement new OA definitions. Additionally, if studying green OA, the study should assure that the articles in the non-OA population remain closed throughout the study and specify their exact working definition of “green” (e.g., whether articles found on academic social networks or author web pages are included). Similarly, studies exploring gold OA should consider whether articles in the OA sample are open throughout the entire citation window and the possibility that “non-OA” articles may have been openly available through other modes. The first consideration is complicated not only by potential delays in archiving an open copy, but also by new modes of OA. For example, with the introduction of bronze OA, publishers are free to move articles without an open license behind or in front of a paywall at any time and the status of an article at the time of data collection may differ from its previous status.
Although many studies reported metrics of mean citations per article and median citations per article, we found a wide variety of metrics studies reported. These included time-based metrics (e.g., citations per month since publication) and proportional metrics (e.g., difference in citations per paper, ratio of cited to non-cited articles). Other studies explored OACA by building linear regression models. The variations in methodology are not surprising, given the wide range of authors’ fields of research, including library and information studies, health sciences, computer science, and physics. OACA studies are published in bibliometric-focused journals, but also in discipline-specific journals. Disciplinary differences may explain some of the variation of methods employed. While a range of outcome measures can prove challenging, the consistent sharing of underlying data can help to overcome these barriers.
Very few (1.5%) of the studies in our sample were RCTs. There are notable challenges to conducting a randomized study on this topic. Beyond issues of funding, it may prove difficult to obtain consent from authors to randomize their articles to either condition. Instead, studies look retrospectively at populations of previously published articles. Nonrandomized studies face the challenge of ensuring that studies are truly comparable at baseline, allowing researchers to control for confounding factors including the potential "novelty" of the study, interest in the study, author reputation, and journal prestige [162]. Studies may also need to address the possibility of surreptitious cointerventions, such as toll access articles being made available through venues like ResearchGate, regardless of the journal’s OA policy. Although the prevalence of nonrandomized studies is understandable, these potential confounders pose challenges for identifying the implications of publishing decisions.
Out of the 134 studies included, 40 acknowledged the possibility of confounders, although not all 40 subsequently controlled for confounders in their analyses. Confounders noted included years since publication, Journal Impact Factor, number of authors, length of article, type of study (e.g., empirical or otherwise), prominence in search engine results, and the alphabetical position of the first author [77, 91, 136, 140, 142, 148, 149]. While examination of confounders is an important component of analysis, we found that this was inconsistently done and the confounders being considered varied significantly between studies.
We found high risk of bias in nearly 98% of the included studies (Fig 2). High risk of bias/low validity in the population domain was often due to a poorly described sample or use of too narrow a sample to support the conclusions drawn. Few studies provided justification for their sample size. We saw high risk of bias/low validity in data collection, often due to lack of justification for why a particular time frame was used (i.e., length of the citation window). Citations accrue over time, at rates that can vary across disciplines. OACA studies should take into account the citation patterns for the discipline being studied. High risk of bias/low validity in study design was often due to poor reporting of outcomes in relation to data collection.
Because of the limitations of the quality of the studies in our review, it is not possible to draw definitive conclusions and recommendations for authors deciding whether to make their work OA. We also recognize that venue choice is complex, and rarely driven by a single factor. Authors may be required to make work public or OA by funding agency mandates, or they may wish to do so to reach certain audiences. These decisions may be complicated, and arrived at through collaboration and discussion, and may be influenced by external factors such as career stage, departmental and organizational requirements, and disciplinary norms. In a survey of over 2,100 researchers at R1 institutions, respondents named journal reputation and quality, alignment between the article and journal scope, and the journal’s readership as the most important factors when selecting a journal, while the OA status of the journal was rated to be the least important consideration in the journal selection process, with 12% of respondents considering it to be very important and 18% of respondents considering it to be important [163]. Although the OA status of the journal seems to be relatively unimportant to many researchers, the potential impact of OA on citations remains an ongoing discussion.
One reason there is so much interest in whether OACA exists is due to the emphasis placed on citation metrics in retention, promotion, and tenure (RPT) decisions. Aksnes et al. [164] provide an overview of the use of citation metrics to evaluate the quality of research and whether citations accurately reflect quality or impact. The Scholarly Communications Lab (https://www.scholcommlab.ca/) has explored the use of citation metrics in RPT. Alperin et al. [165] analyzed 129 RPT documents from US and Canadian universities and found references to metrics in more than 70% of documents from research-intensive universities and nearly half of master’s colleges and universities.
However, citation metrics only measure the use of a study in the academic world. The goal of OA is to enable broader access to research; these uses may not be captured through citations. Scholars wish to publish in the venues that reach their audiences, for example, to reach practitioners who do not have institutional access to subscription resources. These uses may not result in subsequent citation in scholarly work, but are valuable nonetheless. Conversations around the need to and methods of acknowledging impact outside of an academic space have been ongoing, with organizations such as the European Commission noting that “[t]he exclusive use of bibliometric parameters as proxies for excellence in assessment by most funding agencies and universities/research organisations does not facilitate Open Science” [166 p. 8].
Limitations and future work
There are limitations to our study. Most notably, we were unable to conduct a quantitative meta-analysis due to the heterogeneity of and the high risk of bias in our pool of studies. Our findings of high risk of bias for nearly all of the studies may be reflective of the lack of reporting guidelines. Reporting guidelines are “[a] checklist, flow diagram, or structured text to guide authors in reporting a specific type of research, developed using explicit methodology,” providing a list of the minimum information to be shared in each section of a paper [167]. Establishing reporting guidelines for bibliometrics studies and meta-research would be valuable in improving the completeness, clarity, and quality of studies in this area. Scholars would continue to have freedom regarding study design but have guidance on what details would need to be transparently reported. We also note that the EBL critical appraisal tool was designed for library and information research, but not specifically for bibliometric studies [22]. [To the best of our knowledge, there is no risk of bias assessment specifically designed for bibliometric studies, and the unique features of those studies. Development of such a tool, or modification of an existing tool for this purpose, could be of value. Although we did conduct searches across 17 databases, it is possible that relevant resources were not included. Non-English language content and publishers from outside of North America, the UK, and western Europe are underrepresented in scholarly databases, which may have resulted in potential omission in our study [105, 168].
The understanding and use of alternative metrics ("altmetrics”) has increased over the last 10 years [169]. Altmetrics measure the attention a work receives through metrics such as number of downloads, shares, or tweets on Twitter. These metrics have the potential to reflect use of research that would not be captured through scholarly citations, such as articles that are downloaded by healthcare providers and used in clinical care. Some of the studies in our sample addressed altmetrics (alternative metrics, such as number of downloads or views of an article). Studies included in our analysis occasionally considered altmetrics along with traditional citations (15%). An “open access altmetric advantage” could be evaluated through a review like this one.
As seen through the large number of studies identified for this review, OACA is a topic of high and continuing interest. In a now discontinued service, SPARC Europe previously tracked findings of studies measuring OACA [170]. Although this was a worthy effort, data collection ceased in 2015, the methods of identifying the included studies are not transparent or reproducible, and no risk of bias assessments were conducted. Scholars and institutions continue to wrestle with difficult decisions regarding OA publishing amidst a variety of funding arrangements. More rigorous and robustly reported primary research and follow-up syntheses are needed to equip stakeholders with evidence to make informed choices.
OACA studies could be continuously tracked and reviewed qualitatively, if not quantitatively, in the mode of an emerging method, Living Systematic Reviews (LSRs). The purpose of a LSR is to incorporate new studies as they are published [171]. Further, a LSR could be used to regularly communicate the review status updates with stakeholders as the new evidence is incorporated. Whether or not there is an OACA is a topic of continuing interest and a LSR could support rapidly incorporating new evidence.
Supporting information
S1 Checklist. PRISMA Checklist.
This file the PRISMA 2009 checklist with page numbers.
https://doi.org/10.1371/journal.pone.0253129.s001
(DOC)
S1 Appendix. Search strategy.
This file contains the primary database search strategy that was developed in Ovid Medline.
https://doi.org/10.1371/journal.pone.0253129.s002
(DOCX)
S1 Dataset. Data.
This table contains data for all of the included studies.
https://doi.org/10.1371/journal.pone.0253129.s003
(CSV)
Acknowledgments
We would like to thank Jonathan Koffel and Emma Molls for their critical reading of the manuscript.
References
- 1.
Wellen R. Open access scholarly publishing and the problem of networks and intermediaries in the academic commons. 5th Frankfurt Scientific Symposium; Frankfurt am Main, DE. 2005.
- 2. Fitzpatrick K. Giving it away: Sharing and the future of scholarly communication. Journal of Scholarly Publishing. 2012;43(4):347–62.
- 3. Creaser C, White S. Trends in journal prices: an analysis of selected journals, 2000–2006. Learn Publ. 2008;21(3):214–24.
- 4. Dingley B. U.S. Periodical Price Index 2005. 2005 [cited 6 Feb 2021]. Available from: www.ala.org/alcts/files/resources/collect/serials/ppi/05usppi.pdf.
- 5. Greco A. Academic libraries and the economics of scholarly publishing in the twenty-first century: portfolio theory, product differentiation, economic rent, perfect price discrimination, and the cost of prestige. Journal of Scholarly Publishing. 2015;47(1):1–43.
- 6. Okerson A. Report on the ARL Serials Project. Ser Libr. 1989;17(3–4):111–9.
- 7.
Taylor ME. The high price of scholarly journals: Igniting a SPARC [cited 6 Feb 2021]. Available from: library.brown.edu/publications/Bibliofile/Biblio31/high.html.
- 8.
Suber P. Open access. Cambridge, MA: MIT Press; 2012.
- 9. Baric H, Polsek D, Andrijasevic L, Gajovic S. Open access—is this the future of medical publishing? Croat Med J. 2013;54(4):315–8. pmid:23986271
- 10. Fuchs C, Sandoval M. The diamond model of open access publishing: Why policy makers, scholars, universities, libraries, labour unions and the publishing world need to take non-commercial, non-profit open access serious. Triple C: Communication, Capitalism & Critique. 2013;11(2):428–43.
- 11. Piwowar H, Priem J, Lariviere V, Alperin JP, Matthias L, Norlander B, et al. The state of OA: A large-scale analysis of the prevalence and impact of open access articles. PeerJ. 2018;6. Epub 2018/02/20. pmid:29456894
- 12. Björk B-C. Gold, green, and black open access. Learn Publ. 2017;30(2):173–5.
- 13. Lawrence S. Free online availability substantially increases a paper’s impact. Nature. 2001;411(6837):521. pmid:11385534
- 14. Davis PM, Walters WH. The impact of free access to the scientific literature: a review of recent research. Journal of the Medical Library Association. 2011;99(3):208–17. pmid:21753913
- 15. Turk N. Citation impact of open access journals. New Library World. 2008;109(1/2):65–74.
- 16. Lewis CL. The open access citation advantage: Does it exist and what does it mean for libraries? Information Technology and Libraries. 2018;37(3):50–65.
- 17. Craig I, Plume A, McVeigh M, Pringle J, Amin M. Do open access articles have greater citation impact? A critical review of the literature. J Informetr. 2007;1(3):239–48.
- 18. Hua F, Shen C, Walsh T, Glenny AM, Worthington H. Open access: Concepts, findings, and recommendations for stakeholders in dentistry. J Dent. 2017;64:13–22. Epub 2017/07/04. pmid:28669693
- 19. Turk N. The impact of open access on the medical literature: A review of current literature. Slovenian Medical Journal. 2016;85(11–12):640–63.
- 20. Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan—a web and mobile app for systematic reviews. Syst Rev. 2016;5(1):1–10. pmid:27919275
- 21. OECD. Revised field of science and technology (FOS) classification in the Frascati Manual. 2007.
- 22. Glynn L, Cleyle S. A critical appraisal tool for library and information research. Library Hi Tech. 2006;24(3):387–99.
- 23.
The Cochrane Collaboration. Review Manager (RevMan). Version 5.3 ed. Copenhagen: The Nordic Cochrane Centre 2014.
- 24.
R Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing. Vienna, AT. 2017.
- 25. Abbasi Z, Shekofteh M, Shahbodaghi A, Kazemi E. Citation indicators’ comparison of lis open access and subscription publications based on scopus. Global Knowledge, Memory and Communication. 2019;68(4):288–99.
- 26. Abt A, Monteith J, Loss J. Open access status and number of citations in the peer-reviewed hemophilia literature. Curr Med Res Opin. 2017;33:16.
- 27. Alkhawtani RHM, Kwee TC, Kwee RM. Citation advantage for open access articles in European Radiology. Eur Radiol. 2020;30(1):482–6. Epub 2019/08/21. pmid:31428826
- 28.
Aman V. The potential of preprints to accelerate scholarly communication-A bibliometric analysis based on selected journals. M.A Thesis, Humboldt University of Berlin. 2013. Available from: https://arxiv.org/abs/1306.4856.
- 29. Anderson K, Sack J, Krauss L, O’Keefe L. Publishing online-only peer-reviewed biomedical literature: Three years of citation, author perception, and usage experience. J Electron Publ. 2001;6(3).
- 30. Antelman K. Do open-access articles have a greater research impact? College & Research Libraries. 2004;65(5):372–82.
- 31. Antoniou GA, Antoniou SA, Georgakarakos EI, Sfyroeras GS, Georgiadis GS. Bibliometric analysis of factors predicting increased citations in the vascular and endovascular literature. Ann Vasc Surg. 2015;29(2):286–92. pmid:25462541
- 32. Archambault É, Amyot D, Deschamps P, Nicol A, Provencher F, Rebout L, et al. Proportion of open access papers published in peer-reviewed journals at the European and world levels—1996–2013. 2014. Available from: https://digitalcommons.unl.edu/scholcom/8/.
- 33. Archambault É, Côté G, Struck B, Voorons M. Research impact of paywalled versus open access 2016. Available from: https://digitalcommons.unl.edu/scholcom/29/.
- 34. Arendt J, Peacemaker B, Miller H. Same question, different world: replicating an open access research impact study. College & Research Libraries. 2019;80(3):303–18.
- 35. Atchison A, Bull J. Will open access get me cited? An analysis of the efficacy of open access publishing in political science. PS Polit Sci Polit. 2014;48(1):129–37.
- 36. Atchison AL. Negating the gender citation advantage in political science. PS Polit Sci Polit. 2017;50(2):448–55.
- 37. Beatty JR. Revisiting the open access citation advantage for legal scholarship. Law Library Journal. 2019;111(4):573–90.
- 38. Bjork BC, Solomon D. Open access versus subscription journals: a comparison of scientific impact. BMC Med. 2012;10(1):73. pmid:22805105
- 39. Bongiovani P, Miguel S, Gómez ND. Open access, scientific impact and the scientific production in two Argentine universities in the field of medicine. Revista Cubana de Informacion en Ciencias de la Salud. 2013;24(2):118–32. pmid:26935617
- 40. Bordignon F, Andro M. Impact de l’Open Access sur les citations: une étude de cas. I2D–Information, données & documents. 2016;53(3):70–9.
- 41. Breugelmans JG, Roberge G, Tippett C, Durning M, Struck DB, Makanga MM. Scientific impact increases when researchers publish in open access and international collaboration: A bibliometric analysis on poverty-related disease papers. PLOS ONE. 2018;13(9). pmid:30231044
- 42.
Brody T. The effect of open access on citation impact. Fourth International Conference on Cooperation and Promotion of Information Resources in Science and Technology; Southampton, UK. 2009.
- 43.
Brody TD. Evaluating research impact through open access to scholarly communication. Ph.D. Thesis, University of Southampton. 2006. Available from: eprints.soton.ac.uk/id/eprint/263313.
- 44. Calver MC, Bradley JS. Patterns of citations of open access and non-open access conservation biology journal papers and book chapters. Conserv Biol. 2010;24(3):872–80. Epub 2010/05/12. pmid:20455909
- 45. Campos J, Sobral A, Silva J, Araújo T, Ferreira-Júnior W, Santoro F, et al. Insularity and citation behavior of scientific articles in young fields: the case of ethnobiology. Scientometrics. 2016;109(2):1037–55.
- 46. Caulley L, Cheng W, Catala-Lopez F, Whelan J, Khoury M, Ferraro J, et al. Citation impact was highly variable for reporting guidelines of health research: A citation analysis. J Clin Epidemiol. 2020;127:96–104. pmid:32712175
- 47.
Chen Y, Wang Z, Tan J, Liu Z, Issi, Atanassova I, et al. The position of preprint in scholarly communication: A bibliometric and empirical study of arXiv. 16th International Conference on Scientometrics and Informetrics; Wuhan, CN. 2017. p. 799–809.
- 48. Cho J. Analysis of open access status of domestic author’s papers published in international journals: based on highly cited papers. Journal of the Korean Library and Information Science Society. 2020;54(1):325–41.
- 49. Cintra PR, Furnival AC, Milanez DH. The impact of open access citation and social media on leading top Information Science journals. Investigación bibliotecológica. 2018;32(77):117–32.
- 50. Clements JC. Open access articles receive more citations in hybrid marine ecology journals. Facets. 2017;2:1–14.
- 51.
Creaser C. Evaluation of open access journal experiment: Stage 2 interim data. Oxford University Press. Assessing The Impact Of Open Access Preliminary Findings From Oxford Journals. Oxford, UK: Oxford University Press; 2006.
- 52. Davis PM. Author-choice open-access publishing in the biological and medical literature: a citation analysis. J Am Soc Inf Sci Technol. 2009;60(1):3–8.
- 53. Davis PM. Does open access lead to increased readership and citations? A randomized controlled trial of articles published in aps journals. Physiologist. 2010;53(6):197, 200–1. pmid:21473414
- 54. Davis PM, Fromerth MJ. Does the arXiv lead to higher citations and reduced publisher downloads for mathematics articles? Scientometrics. 2007;71(2):203–15.
- 55. Davis PM, Lewenstein BV, Simon DH, Booth JG, Connolly MJ. Open access publishing, article downloads, and citations: Randomised controlled trial. BMJ. 2008;337(7665):a568. pmid:18669565
- 56. De Groote SL, Shultz M, Smalheiser NR. Examining the impact of the National Institutes of Health Public Access Policy on the citation rates of journal articles. PLOS ONE. 2015;10(10). pmid:26448551
- 57. Ding Z, Li F. Publications in integrative and complementary medicine: A ten-year bibliometric survey in the field of ICM. Evidence-Based Complementary and Alternative Medicine. 2020;2020. pmid:33082824
- 58. Donovan JM, Watson CA. Citation advantage of open access legal scholarship. Law Libr J. 2011;103(4):553–73.
- 59.
Donovan JM, Watson CA, Osborne C. The open access advantage for American law reviews. Edison Law & Technology. 2015.
- 60. Dorta-Gonzalez P, Gonzalez-Betancor SM, Dorta-Gonzalez MI. Reconsidering the gold open access citation advantage postulate in a multidisciplinary context: an analysis of the subject categories in the Web of Science database 2009–2014. Scientometrics. 2017;112(2):877–901.
- 61. Dorta-Gonzalez P, Santana-Jimenez Y. Prevalence and citation advantage of gold open access in the subject areas of the Scopus database. Res Eval. 2018;27(1):1–15.
- 62. Eysenbach G. Citation advantage of open access articles. PLOS Bio. 2006;4(5):e157. pmid:16683865
- 63. Ezema IJ, Onyancha OB. Citation impact of health and medical journals in Africa: does open accessibility matter? Electronic Library. 2017;35(5):934–52.
- 64. Fiebig A, Blanke M. Influence of language, open access option and topic on reading and citing articles in Erwerbs-Obstbau. Erwerbs-Obstbau. 2019;61(1):1–8.
- 65. Frisch NK, Nathan R, Ahmed YK, Shidham VB. Authors attain comparable or slightly higher rates of citation publishing in an open access journal (CytoJournal) compared to traditional cytopathology journals—A five year (2007–2011) experience. Cytojournal. 2014;11(10). pmid:24987441
- 66. Gargouri Y, Hajjem C, Lariviere V, Gingras Y, Carr L, Brody T, et al. Self-selected or mandated, open access increases citation impact for higher quality research. PLOS One. 2010;5(10). pmid:20976155
- 67. Gaule P, Maystre N. Getting cited: Does open access help? Res Policy. 2011;40(10):1332–8.
- 68. Gentil-Beccot A, Mele S, Brooks T. Citing and reading behaviours in high-energy physics. Scientometrics. 2010;84(2):345–55.
- 69. Gonzalez-Betancor SM, Dorta-Gonzalez P. Publication modalities ’article in press’ and ’open access’ in relation to journal average citation. Scientometrics. 2019;120(3):1209–23.
- 70. Greyson D, Morgan S, Hanley G, Wahyuni D. Open access archiving and article citations within health services and policy research. J Can Health Libr Assoc. 2009;30(2):51–8.
- 71. Hafeez DM, Jalal S, Khosa F. Bibliometric analysis of manuscript characteristics that influence citations: A comparison of six major psychiatry journals. J Psychiatr Res. 2019;108:90–4. pmid:30049529
- 72. Hajjem C, Gingras Y, Brody T, Carr L, Harnad S. Open access to research increases citation impact. 2005. [cited 6 Feb 2021]. Available from: https://eprints.soton.ac.uk/261687/.
- 73. Hajjem C, Harnad S, Gingras Y. Ten-year cross-disciplinary comparison of the growth of open access and how it increases research citation impact. arXiv:cs/0606079 [Internet]. 2006. [cited 6 Feb 2021]. Available from: http://arxiv.org/abs/cs/0606079.
- 74. Hassan A, Aggarwal S, Kistler J, Hazzard A. Citation rates for open-access versus pay-to-access peer-reviewed articles in clinical oncology journals. Curr Med Res Opin. 2018;34:14–5.
- 75. Henneken EA, Kurtz MJ, Eichhorn G. Effect of e-printing on citation rates in astronomy and physics. J Electron Publ. 2006;9(2).
- 76. Holmberg K, Hedman J, Bowman TD, Didegah F, Laakso M. Do articles in open access journals have more frequent altmetric activity than articles in subscription-based journals? An investigation of the research output of Finnish universities. Scientometrics. 2020;122(1):645–59.
- 77. Hua F, Sun H, Walsh T, Glenny AM, Worthington H. Open access to journal articles in oncology: Current situation and citation impact. Annals of Oncology. 2017;28(10):2612–7. pmid:28961842
- 78. Hua F, Sun H, Walsh T, Worthington H, Glenny AM. Open access to journal articles in dentistry: Prevalence and citation impact. J Dent. 2016;47:41–8. pmid:26875610
- 79. Hubbard DE. Open access citation advantage? A local study at a large research university. Proc Assoc Inf Sci Technol. 2017;54(1):712–3.
- 80. Hudson MA, Heerssen HM, Curristin SM, Storey A. Citation rates for open-access versus pay-to-access articles in clinical journals specializing in paediatric medicine. Curr Med Res Opin. 2019;35:34.
- 81.
Ingwersen P, Elleby A, Noyons E, Ngulube P, Leta J. Do open access working papers attract more citations compared to printed journal articles from the same research unit? 13th International Conference of the International Society for Scientometrics and Informetrics; Durban, ZA 2011.
- 82. Kamat P. Impact of open access papers in hybrid journals. ACS Energy Lett. 2018;3(2):410–1.
- 83. Kim HE, Jiang X, Kim J, Ohno-Machado L. Trends in biomedical informatics: Most cited topics from recent years. JAMIA Open. 2011;18:166–70. pmid:22180873
- 84.
Kim HH. The effect of free access on the diffusion of scholarly ideas. Thirty-Third International Conference on Information Systems; Orlando, FL2012.
- 85. Koler-Povh T, Juznic P, Turk G. Impact of open access on citation of scholarly publications in the field of civil engineering. Scientometrics. 2014;98(2):1033–45.
- 86. Kortlever JTP, Tran TTH, Ring D, Menendez ME. The growth of poorly cited articles in peer-reviewed orthopaedic journals. Clinical Orthopaedics and Related Research. 2019;477(7):1727–35. Epub 2019/05/29. pmid:31135548
- 87. Kousha K, Mahshid A. The citation impact of open access agricultural research: A comparison between OA and non-OA publications. Online Information Review. 2010;34(5):772–85.
- 88.
Kullman L. The effect of open access on citation rates of self-archived articles at Chalmers. IATUL 2014 – 35th Annual Conference Espoo, FI. 2014. p. 1–5.
- 89. Kurtz MJ, Eichhorn G, Accomazzi A. The effect of use and access on citations. Information Processing & Management. 2005;41(6):1395–402.
- 90. Kurtz MJ, Henneken EA. Open access does not increase citations for research articles from The Astrophysical Journal. arXiv:07090896 [Internet]. 2007. [cited 6 Feb 2021]. Available from: arxiv.org/abs/0709.0896.
- 91. Lansingh VC, Carter MJ. Does open access in ophthalmology affect how articles are subsequently cited in research? Ophthalmology. 2009;116(8):1425–31. pmid:19545905
- 92.
Lewison G, editor Open access papers: their growth over time and from different countries, and their citations. 16th International Conference on Scientometrics and Informetrics; 2017; Wuhan, CN
- 93. Lin S-K. Non-open access and its adverse impact on international journal of molecular sciences. Int J Mol Sci. 2007;8(7):686–7.
- 94. Makeenko M, Trishchenko N. The impact of open access on citations and alternative metrics of scientific articles in media and communication studies. Vestnik Moskovskogo Universiteta. 2018;10(5):3–26.
- 95. McCabe MJ, Snyder CM. Identifying the effect of open access on citations using a panel of science journals. Econ Inq. 2014;52(4):1284–300.
- 96. Metcalfe TS. The rise and citation impact of astro-ph in major journals. Bulletin of the American Astronomical Society. 2005;37:555–7.
- 97. Metcalfe TS. The citation impact of digital preprint archives for solar physics papers. Sol Phys. 2006;239(1):549–53.
- 98. Mikki S. Scholarly publications beyond pay-walls: Increased citation advantage for open publishing. Scientometrics. 2017;113(3):1529–38.
- 99. Mikki S, Gjesdal OL, Stromme TE. Grades of openness: open and closed articles in Norway. Publications. 2018;6(4):46.
- 100. Mikki S, Ruwehy HAA, Gjesdal ØL, Zygmuntowska M. Filter bubbles in interdisciplinary research: a case study on climate and society. Library Hi Tech. 2018;36(2):225–36.
- 101. Miller GW, Stoner R, Boissaud-Cooke MA, Lim J, Furness H, Putt O, et al. Open access availability of anatomy papers presented at meetings of the American and British Associations of Clinical Anatomists. Clin Anat. 2020. pmid:32065677
- 102. Mimouni M, Barak Y, Shapira Y, Beiran I, Blumenthal EZ, Zayit-Soudry S. Citation rates of open-access and non-open access clinical studies in leading ophthalmology journals. Invest Ophthalmol Vis Sci. 2017;58(8).
- 103. Moed HF. The effect of "open access" on citation impact: An analysis of ArXiv’s condensed matter section. J Am Soc Inf Sci Technol. 2007;58(13):2047–54.
- 104.
Momeni F, Mayr P, Fraser N, Peters I, Catalano G, Daraio C, et al. From closed to open access: A case study of flipped journals. 17th International Conference on Scientometrics & Informetrics; Rome, IT. 2019. p. 1270–5.
- 105.
Moskaleva O, Akoev M. Non-English language publications in citation indexes–quantity and quality. 17th International Conference on Scientometrics and Informetrics; Rome, IT. 2019.
- 106. Mueller-Langer F, Watt R. How many more cites Is a $3,000 open access fee buying you? Empirical evidence from a natural experiment. Econ Inq. 2018;56(2):931–54.
- 107. Müller-Langer F, Watt R. The hybrid open access citation advantage: how many more cites is a $3,000 fee buying you? Munich Personal RePEc Archive. 2014.
- 108. Nazim M, Ashar M. Do open access journals have greater citation impact? A study of journals in health and medical sciences in India. Journal of Electronic Resources in Medical Libraries. 2018;15(3):149–64.
- 109. Nelson GM, Eggett DL. Citations, mandates, and money: author motivations to publish in chemistry hybrid open access journals. J Am Soc Inf Sci Technol. 2017;68(10):2501–10.
- 110.
Norris M. The citation advantage of open access articles. Ph.D Thesis, Loughborough University. 2008. Available from: https://dspace.lboro.ac.uk/2134/4089.
- 111. O’Kelly F, Fernandez N, Koyle M. Predatory publishing or a lack of peer review transparency?—a contemporary analysis of indexed open and non-open access articles in paediatric urology. J Pediatr Urol. 2019;15(2):159.e1–.e7. pmid:30867116
- 112. Ottaviani J. The post-embargo open access citation advantage: it exists (probably), its modest (usually), and the rich get richer (of course). PLOS ONE. 2016;11(8). pmid:27548723
- 113.
Pacheco A, Sousa A, re, Yanai A, Lopes S, Machado L, editors. Metrics and altmetrics: an exploratory study of a possible correlation between the most cited papers in open and restricted access in 2016–2018. Sixth International Conference on Technological Ecosystems for Enhancing Multiculturality; 2018 2018; Salamanca, ES: Association for Computing Machinery.
- 114. Patel RB, Vaduganathan M, Mosarla RC, Venkateswaran RV, Bhatt DL, Bonow RO. Open access publishing and subsequent citations among articles in major cardiovascular journals. Am J Med. 2019;132(9):1103–5. pmid:30851265
- 115. Peidu CH. An empirical examination of citation in life science. Journal of Scientometric Research. 2020;9(1):70–6.
- 116. Pitol SP, De Groote S, L r. Google Scholar versions: Do more versions of an article mean greater impact? Library Hi Tech. 2014;32(4):594–611.
- 117. Povh TK. Vpliv odprtega dostopa na citiranost znanstvenih objav s področja gradbeništva. Revija za področje bibliotekarstva in informacijske znanosti. 2016;60(4):125–47.
- 118. Razumova IK, Kuznetsov A. Impact of open access models on citation metrics. Journal of Information Science Theory and Practice. 2019;7(2):23–31.
- 119. Riera M, Aibar E. Does open access publishing increase the impact of scientific articles? An empirical study in the field of intensive care medicine. Med Intensiva. 2013;37(4):232–40. pmid:22683044
- 120. Sababi M, Marashi SA, Pourmajidian M, Darki SSPF, Sadrzadeh MR, Dehghani M, et al. How accessibility influences citation counts: The case of citations to the full text articles available from ResearchGate. RT A Journal on Research Policy and Evaluation. 2017;5(1).
- 121.
Sahu DK, Gogtay NJ, Bavdekar SB. Effect of open access on citation rates for a small biomedical journal. International Congress on Peer Review and Biomedical Publication; 2005; Chicago, US. 2005.
- 122. Salisbury L, Chowdhury AR, Smith JJ. Faculty publications from a research university: The scholarly impact of open access versus non-open access. Sci Technol Libr. 2017;36(2):187–99.
- 123. Salisbury L, Smith JJ, Omotola Omolewu A. Analyzing food science research for publishing characteristics as a model for understanding the needs of researchers. Collection Management. 2020;45(4):335–53.
- 124.
Sanders DS, Menic N, Nyamali I, Punnamkuzhy JM, Whelpdale P, Kurien M. PTU-002 Does open access influence the citation metrics in gastroenterology journals? British Society of Gastroenterology Annual General Meeting 2017; Manchester, UK. 2017.
- 125.
Schrader BP. Cross-language citation analysis of traditional and open access journals, M.Sc Thesis, University of North Carolina Chapel Hill. 2016. Available from: 10.17615/djpr-1k06.
- 126. Shekhani HN, Shariff S, Bhulani N, Khosa F, Hanna TN. Bibliometric analysis of manuscript characteristics that influence citations: A comparison of six major radiology journals. American Journal of Roentgenology. 2017;209(6):1191–6. pmid:29064759
- 127. Sood S, Chua SK, Qureshi AM, Krishnan V, Pai DR, Kamal LB, et al. The impact factor of an open access journal does not contribute to an article’s citations. F1000Research. 2017;6(208). pmid:28649365
- 128. Sotudeh H. Does open access citation advantage depend on paper topics? J Inf Sci. 2020;46(5):696–709.
- 129. Sotudeh H, Abbas H. Great expectations: The role of open access in improving countries’ recognition. Scientometrics. 2008;76(1):69–93.
- 130. Sotudeh H, Arabzadeh H, Mirzabeigi M. How do self-archiving and author-pays models associate and contribute to OA citation advantage within hybrid journals. Journal of Academic Librarianship. 2019;45(4):377–85.
- 131. Sotudeh H, Estakhr Z. Sustainability of open access citation advantage: the case of Elsevier’s author-pays hybrid open access journals. Scientometrics. 2018;115(1):563–76.
- 132. Sotudeh H, Ghasempour Z, Yaghtin M. The citation advantage of author-pays model: The case of Springer and Elsevier OA journals. Scientometrics. 2015;104(2):581–608.
- 133. Stankus T. Is PLOS ONE attracting highly cited papers in the food sciences? Comparing authors’ most cited work to their PLOS ONE articles published 2006–2016. Journal of Agricultural & Food Information. 2018;19(1):97–100.
- 134.
Sygocki W, Korzeniewska E, editors. What kind of impact is more important?. role of altmetrics indicators. 2018 Applications of Electromagnetics in Modern Techniques and Medicine (PTZE); 2018; Raclawice, PL.: Institute of Electrical and Electronics Engineers Inc.
- 135. Tahim A, Bansal H, Goodson AM, Payne KF, Sabharwal S. Open access publishing: a study of current practice in oral and maxillofacial surgery research. Journal of Oral and Maxillofacial Surgery. 2016;15(4):517–20. pmid:27833346
- 136. Tang M, Bever JD, Yu F-H. Open access increases citations of papers in ecology. Ecosphere. 2017;8(7).
- 137. Tonia T, Van Oyen H, Berger A, Schindler C, Kuenzli N. If I tweet will you cite? The effect of social media exposure of articles on downloads and citations. Int J Public Health. 2016;61(4):513–20. pmid:27193574
- 138. Valderrama-Zurián JC, Aguilar-Moya R, Gorraiz J. On the bibliometric nature of a foreseeable relationship: Open access and education. Scientometrics. 2019;120:1031–105.
- 139.
van Leeuwen TN, Tatum C, Wouters P. Open access publishing and citation impact-an international study. 15th International Conference of the International Society for Scientometrics and Informetrics; Istanbul, TR. 2015.
- 140. Vanclay JK. Factors affecting citation rates in environmental science. J Informetr. 2013;7(2):265–71.
- 141. Vaughan L, Shaw D. A new look at evidence of scholarly citation in citation indexes and from web sources. Scientometrics. 2008;74(2):317–30.
- 142. Wakeling S, Willett P, Creaser C, Fry J, Pinfield S, Spezi V. Transitioning from a conventional to a ’mega’ journal: A bibliometric case study of the journal Medicine. Publications. 2017;5(2).
- 143.
Wang M-L. The impact of open access journals on library and information scientists’ research in Taiwan. Proceedings of the Asia-Pacific Conference On Library & Information Education & Practice 2011; Putrajaya, MY. 2011.
- 144. Wang X, Liu C, Mao W, Fang Z. The open access advantage considering citation, article usage and social media attention. Scientometrics. 2015;103(2):555–64.
- 145. Wang Z, Chen Y, Glänzel W. Preprints as accelerator of scholarly communication: An empirical analysis in Mathematics. J Informetr. 2020;14(4). pmid:33072184
- 146. Wang Z, Glanzel W, Chen Y. The impact of preprints in Library and Information Science: an analysis of citations, usage and social attention indicators. Scientometrics. 2020;125(2):1403–23.
- 147. Wen YL, Hsieh JL. Informetric analysis on open-access high productivity authors in biomedical area. Journal of Educational Media and Library Science. 2014;51(2):219–24.
- 148. Wohlrabe K, Birkmeier D. Do open access articles in economics have a citation advantage? 2014 [cited 6 Feb 2021]. Available from: mpra.ub.uni-muenchen.de/56842/1/MPRA_paper_56842.pdf.
- 149. Xia J, Lynette Myers R, Kay Wilhoite S. Multiple open access availability and citation impact. J Inf Sci. 2011;37(1):19–28.
- 150. Xia J, Nakanishi K. Self-selection and the citation advantage of open access articles. Online Information Review. 2012;36(1):40–51.
- 151.
Xu L, Liu J, Fang Q, editors. Analysis on open access citation advantage: An empirical study based on Oxford open journals. Proceedings of the 2011 iConference; 2011; Seattle, WA.
- 152. Young F, Wiehn J, Gothard D, Southam E. Does publication with open access enhance article impact? Curr Med Res Opin. 2018;34:18.
- 153. Young JS, Brandes PM. Green and gold open access citation and interdisciplinary advantage: A bibliometric study of two science journals. Journal of Academic Librarianship. 2020;46(2).
- 154. Zawacki-Richter O, Anderson T, Tuncay N. The growing impact of open access distance education journals: A bibliometric analysis. International Journal of E-Learning & Distance Education. 2010;24(3).
- 155. Zhang L, Watson E. The prevalence of green and grey open access: Where do physical science researchers archive their publications? Scientometrics. 2018;117(3):2021–35.
- 156. Zhang L, Watson EM. Measuring the impact of gold and green open access. Journal of Academic Librarianship. 2017;43(4):337–45.
- 157. Zhang Y. The effect of open access on citation impact: a comparison study based on web citation analysis. Libri: International Journal of Libraries & Information Services. 2006;56(3):145–56.
- 158. Harzing AW. Publish or Perish. 2007. Available from https://harzing.com/resources/publish-or-perish.
- 159.
Berlin Declaration on Open Access to Knowledge in the Sciences and Humanities [Internet] 2003 [cited 6 Feb 2021]. Available from: https://openaccess.mpg.de/Berlin-Declaration.
- 160.
L C, D C, M E, F F, Y G, J-C G, et al. Budapest Open Access Initiative [Internet] February 14, 2002 [cited 6 Feb 2021]. Available from: https://www.budapestopenaccessinitiative.org/read.
- 161.
PO B, D C, A C, B C, T D, M E, et al. Bethesda Statement on Open Access Publishing [Internet] 2003 [cited 6 Feb 2021]. Available from: http://www.earlham.edu/~peters/fos/bethesda.htm.
- 162. Tahamtan I, Afshar AS, Ahamdzadeh K. Factors affecting number of citations: a comprehensive review of the literature. Scientometrics. 2016;107(3):1195–225.
- 163. Smith M, Anderson I, Bjork B-C, McCabe M, Solomon D, Tananbaum G, et al. Pay it forward: Investigating a sustainable model of open access article processing charges for large North American research institutions. 2016. Available from: escholarship.org/uc/item/8326n305.
- 164. Aksnes DW, Langfeldt L, Wouters P. Citations, citation indicators, and research quality: An overview of basic concepts and theories. Sage Open. 2019;9(1).
- 165. Alperin J, Nieves C, Schimanski L, Fischman G, Niles M, McKiernan E. How significant are the public dimensions of faculty work in review, promotion, and tenure documents? [preprint]. Humanities Commons 2018.
- 166.
The Working Group on Rewards under Open Science. Evaluation of research careers and fully acknowledging open science practices. Luxembourg: Publications Office of the European Union. 2017. https://doi.org/10.2777/75255
- 167.
Equator Network. What is a reporting guideline? [cited 4 May 2021]. Available from: https://www.equator-network.org/about-us/what-is-a-reporting-guideline/.
- 168. Mongeon P, Paul-Hus A. The journal coverage of Web of Science and Scopus: a comparative analysis. Scientometrics. 2015;106(1):213–28.
- 169. Engineering A, Konkiel S, Priem J, Adie E, Derrick G, Didegah F, et al. The state of altmetrics: A tenth anniversary celebration. 2020.
- 170. SPARC Europe. The open access citation advantage service 2016 [cited 6 Feb 2021]. Available from: sparceurope.org/what-we-do/open-access/sparc-europe-open-access-resources/open-access-citation-advantage-service-oaca/.
- 171. Cochrane Community. Guidance for the production and publication of Cochrane living systematic reviews: Cochrane Reviews in living mode 2019. Available from: community.cochrane.org/sites/default/files/uploads/inline-files/Transform/201912_LSR_Revised_Guidance.pdf.