Why Do We Publish Clinical Trial Results?

Publication (and non-publication) of trial results can be motivated by several factors. Research reports may affect clinical practice, inform patients, influence future research, and prevent duplication of effort. Many believe there is a moral obligation to report results of all trials that involve patients [2].

Publication also has important secondary effects. These may explain existing patterns of publication and resistance to change—despite evidence that current systems are not the best. Authorship of papers in journals can establish reputations and enhance career prospects. Institutional or corporate reputations may also benefit. The productivity of academic departments is judged on their publication output, which also affects their chances of obtaining future funding. The peer-reviewed (and indexed) journal has thus become part of the process of academic appointments and promotions, which, in turn, has spawned the discipline of bibliometrics [3]. And, of course, there are financial interests: drug companies use publications to increase sales while publishers make money both directly from journal sales and from spin-offs such as reprints and advertising [4].

Problems with Existing Publishing Models

The traditional system of publishing trials in medical journals reflects many of the limitations of paper-based systems despite the advent of electronic publishing. In general, new media and the Internet have been used to modify existing systems rather than for dramatic redesign of the architecture of the medical evidence base. For example, papers may now be accessed from journal Web sites but the format of research articles and the ways in which findings are presented remain the same. While some journals allow additional material to be displayed electronically, most still impose strict size limits on research reports and simply use Web sites as a means of accessing documents that are identical to the print version. The result is that, while there have been some major changes such as the development of “open access” journals, information remains hard (or expensive) to access, scattered across many places, and difficult to synthesize. Producing and disseminating printed material can be slow, and journal peer-review often contributes further delays [5].

Another problem with the existing system is that non-publication of negative trials and non-reporting of negative outcomes, coupled with redundant publication of positive findings, has led to systematic publication bias, which can undermine the reliability of medical evidence [2,6,7].

Benefits of Alternative Models

One of the most important initiatives aimed at reducing publication bias is trial registration (i.e., making details of study designs publicly available at the start of a study) [8]. One potential advantage of electronic publication coupled with trial registration is that study results can be linked to the original protocol, or at least to a summary of its main features. Clear study identification, e.g., by including a trial register number, should highlight redundant publications. Secondary analyses will be easy to identify, and the risk of inadvertently including the same results more than once (which can bias systematic reviews and meta-analyses) will thus be reduced [6]. Reviewer (and reader) access to information about trial designs and planned outcomes, which was recorded at the start of each trial, will permit detection of selective reporting since reports of results can be compared with the original protocol [9,10].

Suppression of unfavourable findings will be apparent if all studies are registered at inception. But trial registration can prevent publication bias only if it is coupled with a commitment to make results from all studies available and a mechanism for implementing and policing this. Guidelines for pharmaceutical companies [11] and investigators [12] have long emphasized the importance (and some would argue moral obligation) of publishing results from all studies, regardless of their outcome. But if such guidelines are to be enforced, we need a clear definition of what is meant by publication.

What Do We Mean by Publication?

In the era when print technology was dominant, publication was easy to define. The need for expensive equipment and expertise restricted entry into large-scale publishing activities. This was the golden age of the publisher. Within science and medicine, peer review also became established (despite limited evidence of its effectiveness and lack of agreement about its primary purpose) [13]. Until recently, publication therefore meant being printed in a peer-reviewed journal. However, a combination of new technology and increasing public calls for transparency means that different modes of publication are now being explored. A comparison of some publication methods is shown in Table 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1.

Comparison of Various Methods of Results Publication

https://doi.org/10.1371/journal.pctr.0010031.t001

Looking Further into the Future

While free text may serve readers well, it is not an efficient electronic storage medium. The most powerful, and some would argue the most efficient and transparent, method of making results available would be to provide raw (but anonymized) datasets for public scrutiny. It is now technically possible to store results in a computable databank as in the Trial Bank Project (http://rctbank.ucsf.edu/). Based on such research, a Global Trial Bank has recently been established and aims to be “the most advanced computable repository of trial protocol and results information to promote biomedical discovery as well as transparency and accountability in clinical trial research” [16]. Companies already provide such data to the US Food and Drug Administration, proving that this is technically possible and that standard data transfer protocols are available.

However, adoption of such an initiative requires not only technical feasibility but also political and commercial acceptability. Individuals, academic institutions, and commercial companies usually consider data as a precious possession, not something to be shared with potential rivals. Many researchers are concerned that public access to raw data could encourage inappropriate analyses and misinterpretation. However, Senn has suggested that providing computable data might become “the ultimate test and promoter of statistical quality” [25]. He argues that “every statistician will have to prepare reports knowing that they will be scrutinised in the finest detail by a frequently hostile posterity”. This, surely, is the ultimate form of peer review. However, although databanks offer exciting possibilities, researchers' reluctance to expose their raw data to the world may hold back change.

Problems Ahead

Publicly accessible, searchable electronic results dissemination coupled with trial registers could solve many of the problems that limit the current system of research reporting via peer-reviewed journals. However, such a new system could also create problems of its own.

For a start, trial registers and publication vehicles need to be funded. Currently, some registers and Web sites charge an administration fee for entries. This may be a barrier for unfunded research and projects from the developing world. It is therefore vital that sustainable funding systems are devised, and these must allow inclusive policies for trial registration and results dissemination.

Researchers will need to be equipped to produce reports or data in the required format. This might be facilitated by open-access software and protocols for data transfer, but human effort will also be required for education and support.

The proposed system of posting results is largely self-policing and relies to a greater extent on internal systems of quality control (within companies or academic institutions) than traditional peer-reviewed publication. Performing trials and handling data according to Good Clinical Practice requires considerable expertise and resources. This is currently reinforced by oversight from regulatory bodies (e.g., random audits) in the case of the pharmaceutical industry. The new model of publication might require similar policing systems, especially for non-industry-funded studies, and such systems would require considerable resources.

Editors and publishers may be concerned about what will happen to their journals if they are no longer the primary vehicle for publishing clinical trial results. Some medical editors have already considered and even prophesied this scenario [26–28]. Under the new system, successful journals will be those brave enough to focus on other functions. These could include secondary publications providing synthesis, commentary, opinion, and interpretation, along with other roles such as stimulating debate, campaigning for change, educating, and even entertaining their readers [26]. Freed from the responsibility of publishing clinical trial results, journals might also develop new and more effective models of peer review.

The current system of academic credits and research assessment based on authorship of peer-reviewed articles would need to be changed. Entries on trial registers and results databases are currently anonymous. If electronic posting becomes the primary (or, for some studies, the sole) method of publication, a fair and transparent system for acknowledging contributors will need to be devised. This should be viewed as an opportunity to replace traditional journal authorship systems, which are open to abuse [29]. Better methods of assessing academic performance than relying on journal impact factors should also be developed [30]. Use of study identifiers presents possibilities to develop more accurate citation measures relating to the original study rather than its publication in a journal—for example, by measuring the number of times that a study is re-analysed, synthesized, or referred to in journals.

Conclusions

We are at the threshold of a new era of reporting clinical trial results. We need to develop an evidence-based system to ensure that data storage and dissemination technology are harnessed to produce the best outcome for all users. Given the huge sums spent on medical research, it is a scandal that we have invested so little in determining the most effective and efficient ways of disseminating findings. I urge funders, especially non-commercial ones such as the Wellcome Trust, Medical Research Council (in the UK), and National Institutes for Health (in the US), to rise to this challenge. We also need to consider the best funding models for such systems that would optimize access to findings without creating barriers to non-commercially funded research. Changes in research reporting systems will have implications for the future of medical journals and for current systems of measuring research productivity. Given the established position of peer-reviewed journals, there is likely to be a gradual transition from the current model. This transition may be accelerated by influencers such as journal editors (if they are brave enough to embrace the change rather than resisting it), regulators, and legislators. With wide consultation and cooperation from all key players (including clinicians, drug companies, and bioinformatics experts), we could produce a system that greatly enhances the efficiency, transparency, and power of the medical knowledge base. This is an opportunity we must not miss.