Introduction

The importance of understanding and using science for public policy-making has long been recognised [1], but recent years have seen a growing debate over how this is best achieved [2]–[4]. Still more recently, ‘evidence-based policy’ has become the desired norm in many fields (even if its meaning is still disputed), and this has led to a greater embedding of scientists, both natural and social, alongside other specialists in public policy–making processes. In many governments, scientists are engaged at a senior level. The US, for example, has the President's Council of Advisors on Science and Technology, while the UK has Chief Scientific Adviser posts in all government departments, in addition to a Government Chief Scientific Adviser with a place in some Cabinet Committees.

In spite of their acknowledged importance however, relations between science and policy are sometimes troubled [5], and periodically erupt into controversy. Prominent examples include the acrimonious debate over scientific understandings of climate change [6], further inflamed by the ‘Climategate’ email controversy, disputes over the use of genetically modified crops and foods in Europe, the failure to acknowledge the risk of possible BSE transmission to humans [7], and conflict over stem cell research, which is particularly acute in the United States. In 2009, the public sacking of the Chair of the UK Advisory Council on the Misuse of Drugs began a row not only about appropriate policy (in this case for drugs classification), but also about the proper place of independent scientific advice in the policy-making process. Such troubles are symptomatic of the complexity of science-policy interactions, and suggest that there is still much to understand about the nature of scientific authority and processes of policy formation and change [8]–[10].

Against this backdrop, this paper reports the results of an exercise that sought to identify the most important outstanding questions in this domain. Precedents for attempts to identify ‘key questions’ go back to the learned civic societies of enlightenment England and France. For example, the Royal Society for the Encouragement of Arts, Manufactures, and Commerce (founded 1752) and the French National Institute (1795–1983) identified specific policy-relevant questions for which they offered prizes to promote commercial and social applications of science [11]. Other examples include Hilbert's famous set of mathematical questions [12], Paul Erdös' posing of mathematical questions with cash prizes for those who solved them [13] and Steffen et al's [14] listing of questions in the environmental sciences. Contemporary ‘top down’ examples include the US National Research Council, in its assessment of strategic directions for the geographical sciences [15], and the International Council for Science, with its Grand Challenges in Global Sustainability Research [16].

We have adopted a rather different, bottom-up, approach, bringing together researchers, policy makers and practitioners with interests in relations between science and policy to identify priority, researchable questions in this field. The method is similar to that used in conservation biology [17]–[21] and agricultural science [22]. Previous exercises have been remarkably influential [23]. For example, two of the resulting papers [17], [22] were the most downloaded ever from their respective journals, and one [17] was explicitly cited as the basis for the priority research questions identified within the UK Marine Science Strategy [24]. Our aim has been to identify key questions which, if addressed through focused research and enquiry, might not only help resolve important theoretical challenges but might also improve the mutual understanding and effectiveness of those who work at the interface of science and policy.

The questions presented below were generated through a democratic, transparent and collaborative process similar to those used in previous exercises [23]. There are interesting differences in this case, however, because the existence of a pre-determined research and policy community is much less evident. Participants were therefore selected to cover a wide range of academic disciplines (including the biological, environmental, medical, physical, and social sciences) as well as governmental and non-governmental organisations, consultancies and industry. Initially, each participant was invited to produce a list of questions, consulting widely if they wished to do so (see the Materials and Methods section below). The 239 questions submitted at this first stage are presented in the Material S1. A process of voting, deliberation and further voting (the final stages of which took place at a meeting of participants over two days) subsequently reduced the initial list to a final set of 40 questions. During this process the questions were also redrafted and grouped thematically. They are presented in the following section, ordered by theme but not in rank order.

The outcomes of an exercise such as this are inevitably influenced by the composition of the set of participants, as well as by the process. Clearly, therefore, the results are not ‘reproducible’ (in the sense that a re-run with different people could be expected to produce exactly the same set of questions). Nevertheless, if the exercise were to involve a similarly large and diverse group of participants, and were to be conducted, like this one, through several rounds of voting, deliberation and editing, we consider it highly likely that broadly similar general themes would emerge. This is, of course, an empirically testable proposition.

Results

Discussion

Although it may seem self-evident that policy should be informed by scientific understanding, and should therefore be evidence-based, this normative assumption is itself based on surprisingly weak evidence. Debates continue, for example, about what exactly constitutes good evidence, where and how such evidence should be sought, and at what stage in the policy process different forms of evidence might be more or less appropriate. That such debate persists reflects the fact that there are many open questions about the nature of science-policy interactions, as this exercise has revealed. In short therefore, we need to ask not just how science can best inform policy, but also how policy and political processes affect what counts as authoritative evidence in the first place.

Jasanoff's [2] seminal study of science advisers showed that the value of science in policy stemmed in part from its capacity for detailed engagement with practical policy problems. At the same time, the authority of science was seen to depend on maintaining its independence from politics through separation, in what has been referred to as ‘boundary-work’ [25]–[26]. Rhetorical commitments in the policy world to a clear distinction between facts and values were ever-present. Since then, however, experience in many different contexts, both national and issue-based, has brought about a much greater awareness of the processes of interconnection among science, politics, policy-making and publics [27]–[28]. As Bijker et al. [8] note, an appreciation of the limits of science as an impartial arbiter among policy options comes at exactly the moment when demands for scientific input to policy are increasing. This tension is reflected and articulated in many of the questions generated by the interdisciplinary exercise reported here.

The six broad themes around which the questions have been organized constitute a potential framework for formulating research priorities, if we seek to develop better understandings of how science-policy interactions occur, and of evidence-based policy in practice. Beginning with a set of questions that consider the formal role that science might be expected to play in policy-making, we move on to two sets of more empirical questions about the ways in which science is selected and evaluated within the policy process, and how advisory processes actually work as an established system of governance; both sets of questions bear on the issues of expertise and authority. The following two themes then consider some of the limits to scientific knowledge, specifically in relation to inherent uncertainty and pervasive interdisciplinarity, and the roles of democratic participation and accountability in science-policy interactions. Taken together, these first five themes suggest a maturing appreciation of complexity and mutual interdependence in these relations; of the value and ubiquity of science in contemporary policy making; of the limits of ‘speaking truth to power’; and of the considerable effort that goes into the routine tasks of managing science policy.

Perhaps most interestingly, the final theme opens up a series of questions about how reflection on, and better understanding of, the nature of science-policy relations might help to improving the ways in which scientific evidence and advice is commissioned, constructed and transmitted when developing forms of evidence-based policy. The exercise reported here may therefore be seen as a contribution to developing a broad research agenda for investigating this critical, complex and contested relationship, perhaps in ways that could enhance its capacity to bring the best available knowledge effectively to bear on twenty-first century problems.

Materials and Methods

The methods used in this exercise are similar to those described in Sutherland et al. [23] based on the experience of a series of attempts to identify priority questions [17]–[19], [21]–[22], [29]. The 52 participants were selected to cover a wide range of approaches to science and policy across government, non-governmental organisations, academia and industry. All participants are authors of this paper; the address list indicates their affiliations.

Each participant was permitted to consult widely among their own colleagues in obtaining an initial list of questions. We asked participants how many people they had actively consulted (for example, in workshops, meetings or email discussions, but not including those who were sent details and did not respond). From the responses we know at least an additional 83 beyond the participants were involved in devising questions. In total, 239 questions were submitted. These questions were collated into twelve themes. They were then sent to all participants, who were asked to select around fifty that they considered to be the most important. 29 voted. 11 questions obtained no votes. Participants were also invited to suggest alternative wording.

The final screening took place at a two-day workshop held in Cambridge in April 2011. On the first evening the process was discussed and potential misunderstandings and problems resolved. Prior to the meeting all participants had been provided with the number of votes for each question and any suggested rephrasing. On the following day, the workshop was divided into three 105 minute sessions, each with four groups meeting in parallel – twelve discussion groups in total, one discussing each question theme. Each group was charged with reducing one of the twelve-question themes to three priority questions plus a ranked list of three reserves. A rapporteur (from outside the team) was assigned to each session to incorporate changes to questions and capture the shortlist of the emergent top six; participants observed the editing process (projected onto a screen) as it was being carried out.

Each group had a different, pre-allocated chair (three of whom had previous experience of chairing sessions in similar exercises). A guidance note for chairs suggested that early decisions could be made to drop questions with zero or very few votes from the initial voting round; and also that groups of questions that clearly addressed similar issues could be identified. This process was designed to assist the group in identifying priorities, removing redundancy, and rewording questions to eliminate overlap and improve clarity. The group then voted on the remaining questions in order to select those considered the most important. Chairs also needed to maintain structure and direction in what were invariably vigorous and challenging deliberations.

In a final plenary session chaired by WJS, the top 36 questions (three from each of the twelve groups) were presented as a printed list to each participant to identify overlaps, problem questions and potential clarifications. Editing was again projected onto a screen and so was visible to all. When disagreement could not be resolved by discussion, decisions about inclusion or exclusion of questions, and about specific wording, were made by majority voting. Seven questions were removed by this process. The 12 top-ranking second-level questions were examined and the top 6 of these selected by voting (each participant having 6 votes). They were then discussed further to resolve any overlaps. The next 12 secondary questions were examined along with the remaining top ranked questions and the final five questions selected with each participant having five votes.

Selected questions were then clustered into 6 categories by placing related questions together, and edited by the entire group to produce the questions set out in this paper. During this process, after discussion and another round of voting, one question was removed and one short-listed question was added. As with previous exercises [23] most questions changed considerably from initial submission to final product. Forty-three participants made comments on or edits to the 64 successive versions of the paper that were circulated to all participants.

We did not obtain ethics approval for this exercise, as it was agreed from the outset that all those participating in the voting and selection of questions were to become authors of the resulting paper. However, all submitted questions were treated anonymously; and an agreement was made to publish in an open-access journal, if possible, in order to facilitate general accessibility for those in policy communities.

Supporting Information

Acknowledgments

This project was organised as an activity of the Centre for Science and Policy, University of Cambridge. We thank J. Ouchikh for efficiently dealing with the administration, L. Luheshi, I. Naicker, J. Palmer and E. Rough for transcribing the discussions, Miranda Gomperts for constructive help and three referees for useful and constructive comments.