Figures
Abstract
Background
Zoonotic diseases account for over 60% of all communicable diseases causing illness in humans and 75% of recently emerging infectious diseases. As limited resources are available for the control and prevention of zoonotic diseases, it is necessary to prioritize diseases in order to direct resources into those with the greatest needs. The selection of criteria for prioritization has traditionally been on the basis of expert opinion; however, details of the methods used to identify criteria from expert opinion often are not published and a full range of criteria may not be captured by expert opinion.
Methodology/Principal Findings
This study used six focus groups to identify criteria for the prioritization of zoonotic diseases in Canada. Focus groups included people from the public, animal health professionals and human health professionals. A total of 59 criteria were identified for prioritizing zoonotic diseases. Human-related criteria accounted for the highest proportion of criteria identified (55%), followed by animal-related criteria (26%) then pathogen/disease-related criteria (19%).
Similarities and differences were observed in the identification and scoring of criteria for disease prioritization between groups; the public groups were strongly influenced by the individual-level of disease burden, the responsibility of the scientific community in disease prioritization and the experiences of recent events while the professional groups were influenced by the societal- and population-level of disease burden and political and public pressure.
Conclusions/Significance
This was the first study to describe a mixed semi-quantitative and qualitative approach to deriving criteria for disease prioritization. This was also the first study to involve the opinion of the general public regarding disease prioritization. The number of criteria identified highlights the difficulty in prioritizing zoonotic diseases. The method presented in this paper has formulated a comprehensive list of criteria that can be used to inform future disease prioritization studies.
Citation: Ng V, Sargeant JM (2012) A Stakeholder-Informed Approach to the Identification of Criteria for the Prioritization of Zoonoses in Canada. PLoS ONE 7(1): e29752. https://doi.org/10.1371/journal.pone.0029752
Editor: Ben Adler, Monash University, Australia
Received: June 14, 2011; Accepted: December 5, 2011; Published: January 6, 2012
Copyright: © 2012 Ng, Sargeant. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: VN is supported by the Canadian Institute of Health Research/Public Health Agency of Canada, Applied Public Health Research Chair Program. JMS holds the Canadian Institute of Health Research/Public Health Agency of Canada, Applied Public Health Research Chair. The funding agency had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Zoonotic diseases are defined by the World Health Organization (WHO) as those that are naturally transmitted between vertebrate animals and humans. Zoonotic diseases account for over 60% of all communicable diseases causing illness in humans and 75% of recently emerging infectious diseases [1], [2]; each disease posing a varying degree of threat to public health. As limited resources are available for research, surveillance, control and prevention of zoonotic diseases, it is necessary to prioritize diseases in order to direct resources into those with the greatest needs.
A number of studies have attempted to methodically prioritize communicable diseases and pathogens of national and international public health concern 3,4,5,6,7,8,9,10. More recently, studies have focused on the prioritization of zoonotic diseases and pathogens [11], [12], [13], [14]. Although methodological approaches differ, priority-setting exercises typically follow a series of steps, these include (i) selecting a group of diseases/pathogens for prioritization; (ii) identifying a list of appropriate and measurable criteria to assess diseases/pathogens; (iii) defining a range of levels for each criterion; (iv) determining the relative importance by means of a weight or score for each level within each criterion; (v) assigning weights and/or scores on the selected criteria and levels and aggregating to produce an overall score for each disease/pathogen; and (vi) ranking diseases/pathogens by their overall score to derive a recommended list for prioritization. A cut-off score may apply for the inclusion or exclusion of diseases/pathogens from the priority list.
In a review of prioritization studies conducted between 1997 and 2011 (Tables 1 and 2), 11 studies identified a list of criteria to assess diseases/pathogens (step (ii)). The number of criteria ranged from 5 to 12 and was primarily selected on the basis of expert opinion; however, details of the methods used to identify those criteria from expert opinion were not published.
As the success of a prioritization exercise is largely determined by having an appropriate list of criteria to assess diseases and pathogens, the methods for deriving the list of criteria should be transparent and scientifically driven. Although expert opinion was the preferred method in the selection of criteria for prioritization (Tables 1 and 2), many of these studies questioned whether all potential criteria were considered and whether the entire range of criteria can be captured impartially by expert opinion [4], [11], [12], [13], [15]. Expert groups acknowledge their priorities may not reflect the priorities concerning the general public or decision makers, particularly under social or political pressure [11], [12], [13]. It is therefore rational to include these stakeholders in developing a list of criteria for disease prioritization. Identifying an approach, that is transparent, reproducible and engages the collective opinion of content experts, decision makers and the general public, may lead to the selection of more appropriate criteria for a scientifically valid and robust prioritization exercise.
The first objective of this paper was to identify criteria for the prioritization of zoonotic diseases in Canada. Multiple focus groups comprising people from the public, animal health professionals and human health professionals were conducted to achieve this objective. The second objective was to explore the similarities and differences in responses between the public and the professional focus groups.
Methods
Focus groups were used to identify a list of criteria to prioritize zoonoses [16]. Six focus groups were conducted over a five-week period in February and March of 2010, each group comprising eight to ten individuals (Table 3). A total of 54 individuals participated in the focus groups. Written informed consent was obtained from each participating individual. The groups were selected to reflect a range of demographic and professional characteristics. The number of groups were selected with the goal of reaching theoretical saturation [17], [18].
Three groups comprised of individuals from the general public. These individuals were screened and disqualified if they were employed in any of the following fields: medical science, veterinary science, epidemiology and public health sciences, infectious diseases research including laboratory work on infectious diseases, nurses, dentists and animal health technicians. Two recruitment agencies were enlisted to recruit individuals by telephone in the three public focus groups (one group was conducted in Guelph, Canada while the remaining two were conducted in Toronto, Canada). Recruitment involved the random selection of individuals from an in-house database with specifications to include a range of demographic characteristics (gender, age and educational background) in each group. The response rate during the screening process was not requested or available for these focus groups. A financial incentive (C$50) was given to public group participants. The remaining three groups comprised of individuals employed exclusively in these professional fields; these included infectious disease epidemiologists, academic and practicing physicians and veterinarians, human and animal health laboratory microbiologists, pathologists and technicians, public health practitioners and policymakers at the local, provincial and national level with at least five years of work experience. Individuals in the professional groups were recruited by one of the authors (VN) by email invitation. Recruitment involved targeting selected individuals representing the range of professions listed above in the local areas (Toronto and Guelph) rather than targeting all relevant healthcare professionals in the local region. The response rate was 86% (31 of 36 individuals approached), of these, 28 (78% of the total individuals approached) were available at the set times and dates of the focus groups. The professional group individuals were divided into one group of animal health professionals, one group of human health professionals and the last group comprising a mix of the two. No financial incentive was given to professional group participants.
A total of 28 people participated in the public focus groups and 26 professionals participated in the professional focus groups (Table 3). There were slightly more males than females across all groups (57% to 43%) and within public groups (54% to 46%) and professional groups (62% to 38%). The age range was 20 to 70 years in the public groups and 30 to 59 years in the professional groups. The professional groups encompassed a narrower age range because only individuals in active employment with at least 5 years of work experience in their profession were selected to participate. The public groups were more likely to be high school and college graduates or hold a Bachelor's degree in an unrelated field (these fields were medical and veterinary sciences, epidemiology and public health sciences, infectious diseases research including laboratory work on infectious diseases, nursing and dentistry) while the professional groups were more likely to hold a PhD in the related fields and/or a professional degree (DVM, DVSc or MD).
An experienced moderator conducted all six focus groups using a prepared script (available as Information S1) to ensure consistency between groups. The group discussions were audio taped and transcribed for validation of the group exercises. A nominal group technique [19] was used to structure the group discussion. Participants were presented with the research question - ‘What are important characteristics of zoonotic diseases that should be considered in disease prioritization?’ and were informed that the objective of the session was to formulate a list of criteria, in order of importance, that could be used to prioritize zoonotic diseases for their control and prevention in Canada. The exact mode of control and prevention was not specified but could include regulation, management, vaccination, laboratory diagnosis, research and surveillance. A zoonosis was defined as a disease that is naturally transmitted between humans and animals, including vector-borne and enteric diseases, as well as diseases of animal-origin but primarily circulating amongst humans (for example, SARS and H1N1).
The first half of the focus group session required participants to identify a list of criteria to prioritize zoonoses; participants were presented with a comparison of two zoonoses (H1N1 and H5N1) as a prompt for identifying a list of criteria. Group discussion at this stage was discouraged and participants were asked to identify a list of criteria without consultation from other group members. Participants were given 20 minutes for this step. Once completed, participants were invited to share one criterion from their list in a round-robin format until every criterion on each participant's list had been shared. To avoid duplication, participants were asked to only share one criterion that the group had not previously shared. A flip chart was used to document the group's combined list of criteria. On completion, participants were given the opportunity to explain why certain criteria were identified and a group discussion was encouraged to clarify and discuss the list. The group was then asked to review the list and to remove or merge criteria that appeared to overlap.
The second half of the focus group session was used to apply scores to the list of criteria identified by the group. Participants were asked to score on a scale of 1 (least important) to 9 (most important) each criterion identified. As with the formulation of the list of criteria, participants were asked to score each criterion on their own. Scores from each participant were tabulated and the mean score per criterion was used to create a ranked list of criteria; this list was presented back to the group for discussion. Participants were encouraged to discuss why some criteria ranked low while others ranked high. For the three public focus groups, a second round of scoring was undertaken giving participants the opportunity to re-rank each criterion in light of group discussion. Due to time constraint, the three professional groups only participated in one round of scoring.
The two authors (VN and JMS) independently merged and condensed the lists of criteria from each focus group into one overall list by combining similar criteria together; this process was informed by reviewing the transcripts from the focus group discussions. The decisive factors for combining similar criteria together were that they described the same characteristic (for example, “ability or potential to control the disease” and “efficacy of current control methods”) and/or that they overlapped in their definitions (for example, “case-fatality rate in humans” and “mortality caused by the disease in people” were merged into the criterion mortality characteristics in humans). Although case-fatality and mortality have distinct epidemiological definitions, both include the total number of deaths caused by the disease in their derivation, which would result in duplicating the characteristic ‘total number of deaths caused by the disease’ in the final list of criteria had both been included.
On completion, the two authors examined their condensed lists together and reached consensus on the number of unique criteria identified across the six focus groups. A score for each unique criterion was calculated by averaging the mean scores across duplicate criteria within each unique criterion. The authors also divided the unique criteria into nine themes for analysis by consensus. These themes were the burden of illness, disease epidemiology, control measures, socioeconomic impact, diagnosis, disease knowledge, public awareness and concerns, international considerations and local considerations. Although the burden of illness can be considered a theme within disease epidemiology, it encompasses the range of individual-level physical burden of the disease; these include mortality, severity, and the duration of illness. Disease epidemiology was considered to be aspects of the force of infection driving disease burden at the population-level; these include incidence, prevalence, transmission potential, speed of transmission within individuals, speed of transmission between individuals, immunogenicity, endemicity and high-risk groups.
Results
Six separate lists of criteria were obtained from the focus groups; each group identified between 24 and 33 criteria during the exercise, a total of 164 criteria were identified across all groups (Table 3). As expected, criteria duplication was observed across groups. A final list of 59 unique criteria for prioritizing zoonotic diseases elicited from six focus groups using the nominal group technique [19] are presented in Tables 4, 5 and 6. The criteria are presented by the nine themes that are further divided into human-related, animal-related or pathogen/disease-related categories. Criteria are presented in order of highest mean score by theme, then by human-, animal- or pathogen/disease-related categories. The theme with the most number of criteria identified was disease epidemiology (43%), followed by control measures (17%), socioeconomic impact (12%), disease burden (10%), and diagnosis (7%). Human-related criteria accounted for the highest proportion of criteria identified (55%), followed by animal-related criteria (26%) then pathogen and disease-related criteria (19%).
The public groups identified fewer criteria (35) than the professional groups (46); of these, 22 were common, 13 were unique to the public groups and 24 were unique to the professional groups. Criteria relating to disease epidemiology, socioeconomic impact, diagnosis, public awareness and concerns and local considerations were identified more often in the professional groups (Tables 4, 5 and 6). Control measures, burden of illness, international considerations and disease knowledge were identified more often in the public groups. Human-related criteria accounted for the highest proportion of listed criteria in both the public and professional groups. The professional groups identified more animal-related criteria than the public groups. Due to the qualitative approach, the terms used to describe differences between groups, for example, more/less, higher/lower, are only numeric as no statistical comparisons were performed.
A mean score was calculated for each criterion; a score closer to 9 indicated the criterion was most important while a score closer to 1 indicated the criterion was least important. The mean score for all criteria across all groups was 6.2 ranging from 3.5 (seasonality of the disease) to 8.6 (mortality characteristics in humans and combination of disease risk and probability of infection) (Tables 4, 5 and 6). The mean score for the public groups was 6.2 ranging from 3.6 to 8.6. The mean score for the professional groups was also 6.2 ranging from 3.3 to 8.7.
The top ten criteria ranked by mean score by public and professional groups are presented in Table 7. Mortality characteristics in humans was the highest scored criteria in both the public and professional groups. Other shared criteria in the top ten between the two groups included mode of transmission and potential for human-to-human transmission. The bottom ten criteria ranked by mean score by public and professional groups are presented in Table 8. Seasonality of the disease was the lowest scored criteria in both the public and the professional groups. Other shared criteria in the bottom ten between the two groups included geographic distribution of the disease and immunogenicity in humans.
A list of criteria most frequently identified by all groups is presented in Table 9. Five criteria were identified by all six focus groups - severity of illness in humans, treatment and prevention in humans, high-risk groups in humans, socioeconomic burden of disease in humans and mode of transmission. For the public group, the criteria speed of disease spread in individuals (incubation period), speed of transmission between humans and visual cues to avoid the disease in humans were identified by all three public groups but not in any professional groups. Conversely, the criteria potential for human to animal transmission and public perception were identified by all three professional groups but not in any public groups.
Discussion
We present on a stakeholder-informed, mixed semi-quantitative and qualitative approach to identify a list of criteria that can be used to prioritize zoonotic diseases in Canada. A total of 59 unique criteria were identified in this exercise across six focus groups. Comparing the list of 59 unique criteria with the criteria used in previous prioritization studies (Tables 1 and 2), a much larger number of criteria were identified in this study than have been previously used, although many of these studies involved diseases exclusively in humans hence animal-related criteria would not be expected to be included in these studies. Further, previously published works focused on specific aspects of disease prioritization or a specialized group of diseases, for example prioritizing solely for surveillance [7], [8] or prioritizing emerging diseases only [13]. Finally, two or more criteria from previous studies could be combined into one of our unique criterion, for example, the severity of illness and duration of illness in humans were often combined into morbidity in humans while disease incidence could be combined with severity and mortality [3], [4], [5], [11], [12], [13], [14], thus reducing the number of criteria considered.
Similarities and differences were observed between the public and the professional focus groups. Amongst the similarities were the number of criteria identified within groups, the mean score across all identified criteria and the range in scores across all identified criteria (Tables 4, 5 and 6). The scores given to the most important criteria and least important criteria by groups were also near identical in score and range (Tables 7 and 8). The concordance in the mean score and range across the most salient and least salient criteria by groups indicate an overall agreement with the use of the scoring system and the nominal group technique for the rating of criteria importance.
Similarities were also observed in the overlap of three identified criteria each in the top ten and bottom ten lists suggesting some agreement between the public and professional groups in the criteria that can inform disease prioritization. The agreement on the high scoring of mortality characteristics in humans, mode of transmission and potential for human-to-human transmission suggest these criteria are of greatest importance for both public and professional stakeholders. This was substantiated by the focus group discussions in which the implications for human mortality and the transmissibility of the disease in humans dominated the discussion on what should be considered the most influential characteristics for disease prioritization. Conversely, low scoring criteria - seasonality of the disease, geographic distribution of the disease and immunogenicity in humans did not dominate in the focus group discussions, this was not because these criteria were unimportant (in fact, one could argue that all 59 unique criterion identified in this study were important given their identification in the focus groups), but rather, in comparison to all other identified competing criteria, these criteria were deemed less important by both public and professional stakeholders. The arguments for reduced importance varied and included that the cooler climate in Canada was unlikely to support disease persistence throughout the year (seasonality of the disease), that diseases do not respect political boundaries and with global trade and travel, a disease in one country can spread across the world within days (geographic distribution of the disease), and that immunity status can be modified by vaccination or is only relevant for diseases where reoccurrence is of high concern (immunogenicity in humans).
More differences were observed than similarities between the public and the professional groups. Although the number of criteria identified within each group was similar, the professional groups collectively identified more criteria with over half of these unique to the group (Tables 4, 5 and 6). Further, the professional groups described criteria definitions in more detail than the public group. For example, for the criterion socioeconomic burden of the disease in humans, the definition for this criterion in the professional groups encompassed a range of direct and indirect costs to the burden of illness including “the societal impact, the commerce impact, the psychological impact beyond the impact of the illness”, “cost of treating and managing people”, “impact on daily living” and “it's broader, indirect things, isolation, psychological impact, lost labour … it encompasses things that are more than just the direct cost of illness”. In comparison, the public group's definition of socioeconomic burden was primarily the “direct cost of treatment” (identified by two groups), “cost of prevention” and the “cost to control disease spread”, although one group expanded the definition further to include the “effect on the health care system including cost, hospital bed and logistics”. The larger number of unique criteria identified and the broader criteria definition in the professional groups was not surprising due to the educational background and professional experiences relating to the prioritization of diseases in this group. The differences in criterion definition therefore influenced its scoring with the professional groups scoring socioeconomic burden of the disease in humans high (Table 7); largely on the basis of the impact of this criterion at the societal- and population-level while the public group scored this criterion low (Table 8); primarily on the basis of the impact of this criterion at the individual-level.
Differences could also arise from emphasis placed on who was responsible for addressing the criterion. For example, the criterion scientific knowledge of the disease scored high in the public groups (Table 7) but low in the professional groups (Table 8). Although the reasoning given when identifying the criterion were similar between the public (“if it is a brand new disease and nobody knows anything about it, say in the 1980s and AIDS popped up and nobody knew anything about it, then it's a high priority for me”) and the professionals (“there is a need for research … how much is known generally, for example, if it is a brand new disease, like SARS was when it first came out, then I think it's going to be high priority”), the scoring differed between the public and professional groups because the public group placed more emphasis on the responsibility of scientists in addressing the scientific knowledge of the disease. Some of the comments from the public groups included “Scientists are the ones that are going to fix this problem if they came up with a cure”, “Because they [scientists] are the ones that are doing the research” and “If nothing is known, then it is a high priority for scientists in terms of finding funding for prioritization”. In comparison, the professional groups made comments relating only to the criterion rather than who was responsible for addressing the criterion, in fact, responsibility was assumed for this group - “if we know a lot versus if we don't know much, then maybe we need to prioritize it to know more” and “from a research point of view, how much do you need to know about a disease, so in other words, how much money needs to be spent on research”.
The focus groups were conducted less than one year after the initial outbreak of the pandemic influenza H1N1 virus in April 2009 [20] and the subsequent rapid spread across the globe [21], [22]. The awareness of the rapid spread of the H1N1 pandemic was evident in two criteria identified in the public groups – speed of disease spread in individuals (incubation period) and speed of transmission between humans, these were also criteria that were only identified by all three public groups and not in any professional groups. In reference to the speed of transmission, one participant noted ‘with speed of transmission I mean it originates from Central America and it's already in North America a week later’. Direct reference to H1N1 was also made for the criteria ability to mutate – ‘I was just thinking if the swine flu kind of mutated [then it would be a problem].’ The criteria how quickly can the vaccine or antiviral be manufactured, although not made in direct reference to H1N1, likely reflected on the vaccine shortages throughout Canada during the H1N1 pandemic [23]. It was evident that the criteria identified by the public groups were influenced by recent events while this was not as apparent for the professional groups.
While recent events may influence the public groups, the professional groups were strongly influenced by public perception. The criterion public perception was identified by all three professional groups but not in any public groups. Although public perception was a central theme for the professional group discussions, it was often argued that public perception should not be a factor in disease prioritization, thus a low score. However, the professional group acknowledged that disease prioritization in practice was often driven by public and political pressure. This is best summed up by one of the professional participants: “I think generally, politically, that is what happens [priority is given to diseases of media attention]. A disease may not be that important really in terms of morbidity and mortality, but if the public is panicking, then the government will do something”.
There are limitations relating to the method described in this paper, first and foremost, there may have been a selection bias relating to how individuals were chosen to participate in the study; the use of an in-house recruitment agency database to recruit individuals from the public and targeting selected professionals in the local region rather than randomly selecting individuals from the general public and targeting all relevant healthcare professionals across Canada. Our study therefore makes the assumption that the local public and professional groups selected to participate are in fact representative of the Canadian public and Canadian healthcare professionals. There may have also been a response bias relating to those who agreed to participate in the study. The effect of these methodological biases on the study results is unknown. Second, the professional groups undertook one round of scoring while the public groups were asked to re-score their criteria after a group discussion on the first round of scoring. While this may influence the final scores within groups, the concordance in the mean score and range between the public and the professional groups indicate an overall agreement with the use of the scoring system regardless of the number of rounds of scoring that was undertaken. Additionally, the professional groups were asked whether their scores would change after their group discussion and it was unanimously agreed upon that their scores would not change. Conversely, the public groups all agreed on the value of having a group discussion to come up with a final round of scoring, the latter of which did change from the initial round of scoring.
The advantages and disadvantages of using the nominal group technique in focus groups have already been described by other papers [16], [19]. The major strengths in the nominal group technique are the ability to clarify and minimize the differences of opinion between multiple participants and ensuring equal participation in all members. The main weaknesses are that a certain amount of agreement must already exist in participating members and the amount of time necessary to prepare and execute the technique in each focus group. The method described in this paper has been used to develop criteria for consideration in other health care studies [24], [25], [26].
We presented on the use of focus groups to derive a list of criteria for the prioritization of zoonotic diseases in Canada. This was the first study to describe a semi-quantitative and qualitative mixed approach to deriving criteria for disease prioritization. This study was also the first to collectively engage public and professional stakeholders in disease prioritization. Similarities and differences were observed in the identification and scoring of criteria for disease prioritization between groups; the public groups were strongly influenced by the individual-level of disease burden, the responsibility of the scientific community in disease prioritization and the experiences of recent events while the professional groups were influenced by the societal- and population-level of disease burden and political and public pressure.
The number of unique criteria identified in this study highlights the number of factors that need to be considered jointly, and thus, the difficulty in prioritizing zoonotic diseases. The study showed that relying on expert opinion alone may limit the range of suitable criteria considered for disease prioritization. The method presented in this paper has formulated a comprehensive list of criteria that can be used to inform other disease prioritization studies.
Author Contributions
Conceived and designed the experiments: VN JMS. Performed the experiments: VN. Analyzed the data: VN JMS. Contributed reagents/materials/analysis tools: VN JMS. Wrote the paper: VN.
References
- 1.
Heymann DL (2008) Control of Communicable Diseases Manual;. In: Heymann DL, editor. Washington D.C: American Public Health Association.
- 2. Taylor LH, Latham SM, Woolhouse ME (2001) Risk factors for human disease emergence. Phil Trans R Soc Lond B 356: 983–989.
- 3.
WHO (2003) The Dubrovnik plege on surveillance and prioritization of infectious diseases. Report on a WHO meeting in Bucharest, Romania 21–23 November, 2003. Copenhagen, WHO Regional Office for Europe: World Health Organisation.
- 4. Horby P, Rushdy A, Graham C, O'Mahony M (2001) PHLS overview of Communicable Diseases 1999. Communicable Disease and Public Health 4: 8–17.
- 5. Rushdy A, O'Mahony M (1998) PHLS overview of communicable diseases 1997: results of a priority setting exercise. Communicable Disease Report Supplement 8: S1–12.
- 6. Weinberg J, Grimaud O, Newton L (1999) Establishing priorities for European collaboration in communicable disease surveillance. European Journal of Public Health 9: 236–240.
- 7. Doherty J (2006) Final Report and Recommendations from the National Notifiable Diseases Working Group. Canadian Communicable Disease Report 32: 211–225.
- 8. Doherty J (2000) Establishing priorities for national communicable disease surveillance. Canadian Journal of Infectious Diseases 11: 21–24.
- 9. Krause G, Alpers K, Benzler J, Bremer V, Claus H, et al. (2008) How can infectious diseases be prioritized in public health. EMBO Reports. European Molecular Biology Organization 9: S22–27.
- 10. Balabanova Y, Gilsdorf A, Buda S, Burger R, Eckmanns T, et al. (2011) Communicable Diseases Prioritized for Surveillance and Epidemiological Research: Results of a Standardized Prioritization Procedure in Germany, 2011. PLoS ONE 6:
- 11.
Institut de Veille Sanitaire (2002) Définition des priorités dans le domaine des zoonoses non alimentaires 2000–2002. Paris, France: Institut de Veille Sanitaire.
- 12. Cardoen S, van Huffel X, Berkvens D, Quoilin S, Ducoffre G, et al. (2009) Evidence-based semiquantitative methodology for prioritization of foodborne zoonoses. Foodborne pathogens and disease 6: 1083–1096.
- 13. Havelarr AH, van Rosse F, Bucura C, Toetenel MA, Haagsma JA, et al. (2010) Prioritizing Emerging Zoonoses in The Netherlands. PLoS ONE 5:
- 14.
Institut de Veille Sanitaire (2010) Définition des priorités dans le domaine des zoonoses non alimentaires 2008–2009. Paris, France: Institut de Veille Sanitaire.
- 15. Krause G, Alpers K, Benzler J, Bremer V, Claus H, et al. (2008) Prioritisation of infectious diseases in public health - call for comments. Eurosurveillance 13: 1–6.
- 16. Kitzinger J (1995) Introducing focus groups. British Medical Journal 311: 299–302.
- 17.
Lewis-Beck M, Bryman A, Liao T (2004) The Sage encyclopedia of social science research methods, Volume 1. London, UK: Sage Publications.
- 18.
Krueger R, Casey M (2009) Focus groups: a practical guide for applied research. London, UK: Sage Publications. 240 p.
- 19. Jones J, Hunter D (1995) Consensus methods for medical and health services research. British Medical Journal 311: 376–380.
- 20. CDC (2009) Outbreak of swine-origin influenza A (H1N1) virus infection - Mexico, March–April 2009. Morbidity Mortality Weekly Report 58: 467–470.
- 21. Darwood F, Jain S, Finelli L, Shaw M, Lindstrom S, et al. (2009) Emergence of a novel swine-origin influenza A (H1N1) virus in humans. The New England Journal of Medicine 360: 2605–2615.
- 22. Butler D (2009) Swine flu goes global. Nature 458: 1082–1083.
- 23.
PHAC (2010) Lessons Learned Review: Public Health Agency of Canada and Health Canada response to the 2009 H1N1 Pandemic. Ottawa: Public Health Agency of Canada and Health Canada. 106 p.
- 24. Sampietro-Colom L, Espallargues M, Rodríguez E, Comas M, Alonso J, et al. (2008) Wide social participation in prioritizing patients on waiting list for joint replacement: a conjoint analysis. Medical Decision Making 28: 554–566.
- 25. Leach M, Segal L (2011) Patient attributes warranting consideration in clinical practice guidelines, health workforce planning and policy. BMC Health Services Research 19: 221.
- 26. McCance T, Telford L, Wilson J, Macleod O, Dowd A (2011) Identifying key performance indicators for nursing and midwifery care using a consensus approach. Journal of Clinical Nursing.
- 27. Gilsdorf A, Krause G (2011) Prioritisation of infectious diseases in public health: feedback on the prioritisation methodology, 15 July 2008 to 15 January 2009. Euro Surveillance 16: pii:19861.