http://orcid.org/0000-0003-1680-5535
Figures
Abstract
Background
Current immunisation levels in England currently fall slightly below the threshold recommended by the World Health Organization, and the three-year trend for vaccination uptake is downwards. Attitudes towards vaccination can affect future decisions on whether or not to vaccinate, and this can have significant public health implications. Interventions can impact future vaccination decisions, and these interventions can take several forms. Relatively little work has been reported on the use of vaccination interventions in young people, who form the next generation of individuals likely to make vaccination decisions.
Method
We investigated the impact of two different types of educational intervention on attitudes towards vaccination in young people in England. A cohort of young people (n = 63) was recruited via a local school. This group was divided into three sub-groups; one (n = 21) received a presentation-based intervention, one (n = 26) received an interactive simulation-based intervention, and the third (n = 16) received no intervention. Participants supplied information on (1) their attitudes towards vaccination, and (2) their information needs and views on personal choice concerning vaccination, at three time points: immediately before and after the intervention, and after six months.
Results
Neither intervention had a significant effect on participants’ attitudes towards vaccination. However, the group receiving the presentation-based intervention saw a sustained uplift in confidence about information needs, which was not observed in the simulation-based intervention group.
Discussion
Our findings with young people are consistent with previous work on vaccination interventions aimed at adults, which have shown limited effectiveness, and which can actually reduce intention to vaccinate. Our findings on the most effective mode of delivery for the intervention should inform future discussion in the growing “games for health” domain, which proposes the use of interactive digital resources in healthcare education.
Citation: Carolan K, Verran J, Crossley M, Redfern J, Whitton N, Amos M (2018) Impact of educational interventions on adolescent attitudes and knowledge regarding vaccination: A pilot study. PLoS ONE 13(1): e0190984. https://doi.org/10.1371/journal.pone.0190984
Editor: Albert Lee, The Chinese University of Hong Kong, HONG KONG
Received: July 27, 2017; Accepted: December 22, 2017; Published: January 19, 2018
Copyright: © 2018 Carolan et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: KC was supported by a Ph.D. studentship from the Faculty of Science and Engineering, Manchester Metropolitan University. The development of SimFection was supported by a grant to JV from the Society for Applied Microbiology. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Vaccination [1] is a process whereby an individual may become artificially immunised against an infectious disease. Deliberate introduction of antigens (i.e., the vaccine) stimulates the body to produce antibodies, which allow it to fight off future exposure to a disease. Herd immunity is the effect produced by a significant proportion of a population being immunised against an infectious disease [2]. Immunocompromised individuals cannot receive vaccines containing live or attenuated cells, and receiving the vaccine could lead to their contracting an active infection. Herd immunity is therefore important to both the eradication and containment of serious infectious diseases, and to the protection of those who cannot be vaccinated, by creating a “barrier” of immunised people. However, herd immunity can be compromised if the proportion of vaccinated individuals in a population drops below a critical threshold.
Current guidance from the European Region of the World Health Organization (WHO) recommends that at least 95% of children be immunised against specific diseases such as diphtheria, tetanus, pertussis, polio, Hib, measles, mumps and rubella [3]. The latest available National Health Service statistics for England show that in 2015-16 93.6% of children reaching their first birthday had completed their primary immunisation courses against diptheria, tetanus, pertussis, polio, and Hib (compared with 94.2% in 2014-15, and 94.3% in 2013-14), and coverage of the first dose of the measles, mumps and rubella (MMR) vaccine for children reaching their second birthday stood at 91.9% in the same period (compared with 92.3% in 2014-15, and 92.7% in 2013-14) [4].
Current immunisation levels in England therefore fall slightly below the WHO threshold, and the three-year trend for vaccination is downwards. The issue of vaccination resistance/refusal [5–7] in parents is complex and multi-factorial [8–10], and falls outside the scope of our study. However, recent arguments [11, 12] suggest that public health efforts to address issues of vaccine hesitancy [6, 13, 14] (as opposed to active resistance) may prove beneficial in terms of maintaining coverage. Moreover, some researchers have argued [15] that efforts in addressing vaccination hesitancy should become more focussed on children and young people, for two main reasons: (1) there exists recent evidence that interventions in adults aimed at improving vaccination rates or correcting myths about vaccines can actually be counter-productive, and lead to further entrenchment of anti-vaccination positions (the so-called “backfire effect”) [16, 17]; (2) Given that attitudes towards vaccination seem to be firmly-held by adulthood, if we assume that beliefs are often formed during childhood and early adolescence, then an opportunity exists to strengthen positive messages about vaccination through school-based educational programmes, which will hopefully influence young people’s future vaccination decisions about their own children.
Some community medicine researchers advocate the use of games and other digital resources in school curricula dealing with vaccination [15], but no evaluation studies have yet been performed to assess their effectiveness in terms of improving either educational or attitudinal outcomes in young people. Our study addresses precisely this gap in the literature. We assessed both digital simulation-based and traditional educational interventions with young people to assess whether or not these can affect their attitudes towards vaccination or their level of confidence in their knowledge of vaccination.
Methodology
We took a mixed quantitative/qualitative data collection approach. This was necessary because attitudes are inherently multi-faceted [10, 18, 19]. The Health Belief Model (HBM) [20] underpinned the development of the interview schedule and discussion questions, as this has proved particularly effective in establishing the impact of attitudes and beliefs on behavioural intentions (and in the context of vaccination) [21]. The HBM focuses on understanding attitudes towards a health topic, by investigating the impact of “concepts” on health beliefs, including perceived benefits of an action such as vaccination, perceived barriers to the action, perceived susceptibility and severity (e.g. to a disease), and cues to action (e.g. a letter from a doctor) [20]. Several studies have used the HBM in an exploratory way [22–25]; this study used the HBM to explore the attitudes of teenagers towards vaccination during the initial research stages, using interviews (n = 14). In addition, The HBM underpinned the development of the initial interview schedule and discussion questions, as this has proved particularly effective in establishing the impact of attitudes and beliefs on behavioural intentions (and in the context of vaccination) [21]. In the next Section, we describe, specifically, how the study was influenced by the HBM.
Study design
Prior to the main study, we performed a literature search for questionnaires exploring attitudes towards vaccination. However, none of these were found to be suitable for our purposes, for two reasons: (1) they focussed solely on adults, and/or (2) they focussed on a specific vaccine (e.g., HPV, MMR) [26–28]. For these reasons, we developed our own questionnaire (the design of which was guided by those found in the literature).
The development of the attitudinal survey proceeded over several stages. These encompassed in-depth interviews, selection of survey items, selection of a scale, validation of the questionnaire, and the use of statistical analysis to refine the survey into an eight-item questionnaire. We conducted in-depth interviews conducted with local teenagers (n = 14) to explore the range of attitudes towards vaccination (these individuals were not part of the main trial). We designed an interview schedule (S1 Text) to explore the full range of attitudes towards vaccination in teenagers. The interview schedule used open questions, and was semi-structured, with prompts for each question. The interview schedule was reviewed by experts in Microbiology and Education research to ensure that the questions were not leading, and used introductory questions to “settle” participants and ensure they were at ease before the main body of the interview. The interview schedule was designed around the following concepts, which are related to the Health Belief Model:
- Perceived susceptibility to infectious diseases included in the immunisation schedule
- Perceived seriousness of vaccine-preventable infectious diseases
- Perceived benefits of vaccination
- Perceived barriers to vaccination
- Sources of information in vaccination decisions.
An initial discussion task incorporated “role-play” and decision-making; participants were asked to imagine that they needed to decide whether or not to vaccinate their child against measles. They were provided with the “pros and cons” of vaccinating, using a “doctor’s” opinion and a “friend’s” opinion. The following discussion questions were supplied:
- Are there any advantages of vaccination? If so, what are they? (Perceived benefits)
- Are there any disadvantages of vaccination? If so, what are they? (Perceived barriers)
- Why do you think some people don’t want to vaccinate? (Perceived barriers)
- How serious do you think infectious diseases like measles are? (Perceived severity)
- How likely do you think it is that someone could catch measles? (Perceived susceptibility)
- Should people be encouraged to vaccinate by their doctors? (Personal choice)
- What would make you more likely to vaccinate? (Cues to action)
- Do you think that doctors or parents should have the most say about children’s vaccinations? (Personal choice)
- Can you think of any other issues surrounding vaccination? (General discussion)
Interviews were then conducted to data saturation [29], and yielded six main themes that were considered to be important to the participants when considering issues surrounding vaccination: (1) trust, (2) effectiveness of vaccination, (3) safety of vaccination, (4) risk of infectious disease, (5) information needs, and (6) personal choice. The prevalence of these themes is consistent with previous research on attitudes towards vaccination [10, 18, 30], including a qualitative study of Scottish teenagers’ understanding towards and views of vaccination [19].
These themes informed the design of the attitudinal survey, with five questions designed for each of the six themes (in order to ensure a representative range). We used a Likert scale for the questionnaire (with responses coded 1-5), allowing each participant to receive a score corresponding to their overall attitude towards vaccination. Face validity [31] of questions was assessed by microbiology and education experts, and we made changes based on their feedback (for example, the questions were re-worded to make them more suitable for the age group.)
A Flesch Reading Ease analysis [32] of the draft questionnaire gave a score of 79.5, suggesting the questionnaire was suitable for 13-15 year olds. We then presented it to a focus group made up of subjects from the target age group (n = 9). Participants were asked to give feedback on the terminology used in the survey, as well as general opinions and thoughts about the survey. Four participants said that the questionnaire was readable as it was. Two questions were re-written based on feedback provided. In order to further refine the survey, we presented it to anonymous participants using online forums. We collected 46 responses, and used discriminant analysis, correlation analysis and Cronbach’s alpha to eliminate questions that gave similar types of response. The final attitudinal survey included eight items from the original 30 statements, covering four themes: trust (of doctors and healthcare professionals), risk of infectious diseases, safety of vaccination, and effectiveness of vaccination. The final set of additional questions included in the questionnaire also included six questions about information needs and personal choice. This gave a total of 14 questions in the survey.
The target age range for the study was 14-18; however, due to a lack of availability of appropriate participants, and the need for the trials to have some educational alignment with taught material (so that school teaching time was not “wasted”), only students aged 14-15 participated in the intervention study. However, the preliminary stages (interviews, focus groups, pilot trials) featured participants across the full 14-18 age range.
Participants in the intervention study were drawn from a Secondary school in North West England. Most of the participants were fifteen years old, just over half of the participants were male, the majority of participants were White British, and participants largely reported as being either Christian or non-religious (see Table 1 for a full breakdown). Participants were provided with a detailed information sheet and consent form prior to participation, parental permission was requested, and the researcher had a full DBS (Disclosure and Barring Service) check performed, allowing her to work unsupervised with children. Participants were recruited through a “gatekeeper” senior school staff member. Consent was obtained from participants prior to their involvement in the study (additionally, parental consent was obtained via the school), and participants could choose to not answer any question. Ethical approval for interviews and trials was granted through Manchester Metropolitan University’s Ethical Approval Procedure (application number SE141521).
The number of participants in our study (n = 63) is comparable to to that seen in similar recent studies, including one (n = 58) that looked at the effect of an educational intervention on human papillomavirus vaccine uptake in female students [33], and another (n = 54) that evaluated the impact of an educational intervention on students attitudes towards mental health [34]. Participants were each assigned a unique ID code to allow pre- and post-trial responses to be recorded consistently. Participants were assigned, according to their class, to either one of the two intervention groups, or to the control group.
We performed an initial survey of all participants using a questionnaire (S1 Questionnaire) in order to establish (1) their attitudes towards vaccination, (2) their confidence in their knowledge of vaccination, (3) their information needs, and (4) their views on personal choice concerning vaccination. This established baseline scores for each individual in order to assess the impact of the interventions.
Both intervention groups received material with the same learning objectives. Students should:
- Know what a pathogen is
- Understand the process of vaccination and how it leads to immunity
- Know what herd immunity is and how it is beneficial to a population
- Understand the background to measles and how it can be prevented by vaccination
One group (Group A) received the digital game-based resource (n = 26), and the other group (Group B) received a traditional PowerPoint lesson (n = 21). We denote the control group (n = 16) as Group C. Both groups A and B then participated in a short session (one per group), where the advantages and disadvantages of vaccination were discussed (both groups discussed the same questions), and participants completed a worksheet (S1 Worksheet) and a feedback survey. The control group attended their usual lessons while Groups A and B were receiving the interventions, and did not participate in a discussion. The questionnaire was then completed again by all participants. To reduce the possibility of data contamination, trial sessions were held on the same day, one after another (meaning students would not encounter each other in between trial sessions). In addition, students receiving the digital intervention were unable to share the trial material with other students as it was pre-loaded onto the school’s laptops, which did not leave the classroom in which the trial was held. Initial data collection was conducted in January 2016, and six-month follow up assessments were conducted in July 2016, when the same questionnaire was again filled out by all participants.
Experimental interventions
We trialled two different interventions; with Group A we trialled an interactive software package called SimFection, which uses computer simulations to illustrate concepts such as herd immunity, infectivity, mortality rates, the effect of migration, and ring vaccination (which are all covered in the current GCSE and A-Level Biology curricula). Diseases covered by SimFection include mumps, influenza, mumps and smallpox. SimFection is based on the SimZombie package [35], which has been successfully used by us for teaching and public engagement. This approach is based on the “health games” model [36, 37], which uses software [38], board games [39, 40] or other activities [41] to develop understanding of health-related issues.
The full package (including both Powerpoint presentations and the software) is freely available at http://www.simfection.org.uk [42]. With Group B we delivered a “traditional” Powerpoint-based presentation on infectious diseases (S1 Presentation).
For Group A, we used the measles simulation to illustrate the impact of different levels of vaccination coverage. At a low level of vaccination coverage, outbreaks occur and spread quickly through the population, and some agents in the simulation die, demonstrating the risk of infectious diseases to non-immunised people. When the vaccination coverage is set to a high value (above 95%), outbreaks are prevented, demonstrating to the user the effectiveness of vaccination at preventing the spread of infectious disease. Participants worked individually on the task, which was to find the minimum level of immunisation coverage needed to prevent a measles epidemic. This required the participants to use either trial-and-error or their previous knowledge of vaccination in order to establish that 95% is the minimum coverage level needed.
Outcome measures
The questionnaire delivered before and after the interventions, and after a six month period, comprised two sections: an attitudinal survey (8 questions, using a Likert scale), and questions on information needs and personal choice (6 questions). The engagement survey, completed only after each intervention, comprised 6 questions.
The attitudinal score for each participant was generated from their responses to the first eight questions, which were scored as shown in Table 2. Each point on the scale was allocated a value in the range 1-5, and responses for each participant were summed to give an overall score. Higher scores signify a more positive attitude towards vaccination and the importance of protecting against infectious diseases, while low scores indicate mistrust of vaccination and/or negative views towards the need to protect populations. The maximum possible score was 40 (corresponding to someone having the most positive view of vaccination), and the lowest (most negative) possible score was 8 (assuming all questions were answered).
The questions concerning information needs and personal choice (all scored on a Likert scale) are shown in Table 3. Engagement with each intervention was measured by the questions shown in Table 4 (also scored on a Likert scale).
Results
The initial group sizes were as follows: Group A (digital intervention), n = 26; Group B (presentation intervention), n = 21; Group C (control), n = 16. Subsequent absences or failure to engage with the follow-up questionnaire meant that we analysed complete results for 19, 17 and 16 individuals for Groups A, B and C respectively). The full data set of attitudinal scores is available in S1 Dataset, and the information/personal choice dataset in S2 Dataset.
The attitudinal analysis was performed as follows: for each participant, we compared their baseline (initial) attitudinal survey score with their post-intervention score, and their post-intervention score with their six-month follow-up score (that is, two comparisons per participant). Our summarised results are shown in Table 5; note that we only record the direction of attitudinal shift (or no shift), as we are not interested in absolute values. This allows us to sum over each comparison column in order to find the number of individuals who have changed attitudes.
For each participant within a group, we denote a positive attitudinal shift with “+”, a negative shift with “-”, and no shift with “0” (“x” denotes the fact that no response was recorded.
Chi-squared analysis revealed no statistically significant difference between the three groups after the intervention was delivered (p-0.115, df = 4). In addition, there was no statistically significant difference between the groups after the 6-month follow-up (p = 0.116, df = 4).
In Fig 1 we show the net attitudinal shift per group over time (from after the intervention to the 6-month follow-up). For each group at each time point, we calculate the net attitudinal score by subtracting the number of negative shifts from the number of positive shifts. We see that both intervention groups have actually shifted to a less enthusiastic attitude towards vaccination, while the control group has moved to a more sympathetic position. We discuss the implications of this finding in the next Section.
We now consider the responses given to the questions concerning information needs and personal choice. Using Kruskal-Wallis analysis, we saw no statistical difference in responses across the trial groups to Q1, Q3, Q4, Q5 or Q6. We did, however, see a statistically significant difference between the three groups for Q2: “I know all I need to know about vaccination and how it works” (p = 0.004, df-8). Post-hoc analysis showed a significant difference between the simulation intervention group and the control group after six months (p = 0.044, df = 8), with fewer participants in the digital group agreeing with the statement after six months.
In terms of engagement with the interventions, using Mann-Whitney analysis we observed no statistically significant difference in responses to questions on this subject across the intervention groups, apart from Q1: “I found the session informative”, where more participants from the digital group agreed with the statement than in the presentation group (p = 0.04, df = 2). Qualitative written feedback received from participants focussed on a desire for more information about vaccination and its possible side-effects, and a need to use a wider range of example diseases. Several participants in Group A expressed a desire for more interactivity in the software. Although this feedback illuminates the design of the sessions, it does not fundamentally affect the findings with regard to attitudes.
Discussion
Several initiatives have attempted to improve public attitudes towards vaccination. These have previously focused on adults (e.g., [43–45]), but a recent meta-analysis of previous vaccination interventions aimed at adults found that (a) they tend to have limited success, and (b) that they can, in some cases, actually decrease intent to vaccinate [16]. For this reason, and because of the gap in the literature covering young people, our research sought to assess the effectiveness of an educational intervention on attitudes in teenagers towards vaccination. This was deemed an appropriate age group to target, because it would reach a generally pre-parenthood group, members of which have expressed interest in receiving more information about vaccination [46].
When considering the format the intervention should take, the literature provided numerous examples of successful digital-based interventions for health. Notable examples include the “Re-mission” game, a digital health intervention, which has been shown to improve adherence to medical treatments and knowledge and understanding of cancer in young adults and adolescents with cancer [47], and a game (“PR:EPARe”) to be used in the classroom for Relationship and Sex Education [48].
Interviews were conducted with fourteen teenagers from the local area in order to explore teenagers’ attitudes towards vaccination. This provided a wealth of qualitative data that provided general themes that were significant to teenagers’ attitudes towards vaccination. These themes were: effectiveness of vaccination, safety of vaccination, risk of infectious disease, trust of healthcare professionals, information needs and personal choice. These were used to develop a series of statements about vaccination, which were refined into an eight-statement attitudinal survey. A focus group was used to test suitability for use with the target group.
We performed a full trial with three groups; one control, one receiving a digital resource-based intervention, and one receiving a traditional presentation-based intervention. We observed no significant differences between the three groups immediately after intervention, or after six months. For this reason, the main conclusion of our research is that educational interventions focussed on vaccination do not have a significant effect on the attitudes of young people. This conclusion is consistent with several recent studies; Nyhan, et al. showed that vaccination interventions aimed at adults have limited effectiveness [16], Dube, et al. showed that no available interventions could usefully address vaccine-hesitancy [12], and Fu, et al. could not find evidence to recommend any specific educational intervention to improve HPV vaccine acceptance [49]. However, a commentary on this latter paper [50] highlights the study of Marek, et al. [51], which demonstrated a positive impact on attitudes and intentions concerning HPV vaccination in young Hungarian adolescents. Although the the transferability of HPV-specific findings to a more general domain of vaccination remains an open question, it does suggest that, in certain contexts, educational interventions can have a positive effect on attitudes and behaviour.
Pre-intervention attitudinal scores were generally positive (simulation-based intervention group: 31.4/40; presentation-based intervention group: 32.5/40; Control group: 31.5/40), suggesting that this group was already well-disposed towards vaccination. This may account for the fact that there was no significant difference in attitudes after receiving the intervention.
Interestingly, there was no statistically significant difference between the digital resource group and the presentation-based intervention group in terms of engagement. This has significance for the current debate about the value of so-called “games for health” [15, 52]. The results from both test groups suggested that, in this particular study, the format of the intervention did not affect engagement levels.
The main limitation of this study was the sample size. For this reason, it is entirely possible that the study is underpowered and that some difference may have been observed between the trial groups had a larger sample size been used. All participants included in the trial were GCSE Biology students from the same school, with a focus on science education, so different results may have been obtained from a less engaged group of students. In addition, several ethnic and religious groups were under-represented; full trials conducted with a wider range of demographics would provide a clearer picture of the impact (if any) of different demographics. Additionally, this study bases its results on a one-time intervention, and it is possible that repeated interventions or a longer-term study would be more effective.
Some questions have emerged from this work:
- Would vaccination interventions have a more significant effect on participants with more negative initial attitudes towards vaccination?
- What, if any, effects do ethnicity and religious background have on the effectiveness of educational vaccination interventions?
- If teenagers’ attitudes towards vaccination are generally positive, but vaccination uptake is lower than the recommended level set by the World Health Organisation, what other factors are negatively influencing attitudes towards vaccination between adolescence and parenthood?
In terms of the wider area of research, this project has demonstrated the difficulty of changing attitudes when using short-timescale interventions. This might suggest that more in-depth and/or longer-term interventions are needed to change complex attitudes such as attitudes towards vaccination.
Supporting information
S1 Worksheet. Worksheet used with intervention groups.
https://doi.org/10.1371/journal.pone.0190984.s003
(DOCX)
S1 Presentation. PowerPoint presentation used with Group B.
https://doi.org/10.1371/journal.pone.0190984.s004
(PPTX)
S2 Dataset. Information needs/personal choice dataset.
https://doi.org/10.1371/journal.pone.0190984.s006
(DOCX)
References
- 1.
Plotkin SL, Plotkin SA. A short history of vaccination. In: Vaccines. 5th ed. Saunders Philadelphia; 2004. p. 1–16.
- 2. Fine PE. Herd immunity: history, theory, practice. Epidemiologic Reviews. 1993;15(2):265–302. pmid:8174658
- 3.
WorldHealthOrganization. Health21: the health for all policy framework for the WHO European Region; 1999.
- 4.
NHSDigital. NHS Immunisation Statistics, England, 2015-16; 2016.
- 5. Omer SB, Salmon DA, Orenstein WA, Dehart MP, Halsey N. Vaccine refusal, mandatory immunization, and the risks of vaccine-preventable diseases. New England Journal of Medicine. 2009;360(19):1981–1988. pmid:19420367
- 6. Sadaf A, Richards JL, Glanz J, Salmon DA, Omer SB. A systematic review of interventions for reducing parental vaccine refusal and vaccine hesitancy. Vaccine. 2013;31(40):4293–4304. pmid:23859839
- 7. Schwartz JL, Caplan AL. Vaccination refusal: ethics, individual rights, and the common good. Primary Care: Clinics in Office Practice. 2011;38(4):717–728. pmid:22094142
- 8. Streefland PH. Public doubts about vaccination safety and resistance against vaccination. Health Policy. 2001;55(3):159–172. pmid:11164965
- 9. Hobson-West P. Understanding vaccination resistance: moving beyond risk. Health, Risk & Society. 2003;5(3):273–283.
- 10. Brown KF, Long SJ, Ramsay M, Hudson MJ, Green J, Vincent CA, et al. UK parents’ decision-making about measles–mumps–rubella (MMR) vaccine 10 years after the MMR-autism controversy: A qualitative analysis. Vaccine. 2012;30(10):1855–1864. pmid:22230590
- 11. Dubé E, Laberge C, Guay M, Bramadat P, Roy R, Bettinger JA. Vaccine hesitancy: an overview. Human Vaccines & Immunotherapeutics. 2013;9(8):1763–1773.
- 12. Dubé E, Gagnon D, MacDonald NE, et al. Strategies intended to address vaccine hesitancy: Review of published reviews. Vaccine. 2015;33(34):4191–4203. pmid:25896385
- 13. Peretti-Watel P, Larson HJ, Ward JK, Schulz WS, Verger P. Vaccine hesitancy: clarifying a theoretical framework for an ambiguous notion. PLoS Currents. 2015;7. pmid:25789201
- 14. Salmon DA, Dudley MZ, Glanz JM, Omer SB. Vaccine hesitancy: causes, consequences, and a call to action. Vaccine. 2015;33:D66–D71. pmid:26615171
- 15.
Wilson K, Atkinson K, Crowcroft N. Teaching children about immunization in a digital age. Human Vaccines & Immunotherapeutics. 2017; p. 1–3.
- 16. Nyhan B, Reifler J, Richey S, Freed GL. Effective messages in vaccine promotion: a randomized trial. Pediatrics. 2014;133(4):e835–e842. pmid:24590751
- 17. Nyhan B, Reifler J. Does correcting myths about the flu vaccine work? An experimental evaluation of the effects of corrective information. Vaccine. 2015;33(3):459–464. pmid:25499651
- 18. Gardner B, Davies A, McAteer J, Michie S. Beliefs underlying UK parents’ views towards MMR promotion interventions: a qualitative study. Psychology, Health & Medicine. 2010;15(2):220–230.
- 19. Hilton S, Patterson C, Smith E, Bedford H, Hunt K. Teenagers’ understandings of and attitudes towards vaccines and vaccine-preventable diseases: A qualitative study. Vaccine. 2013;31(22):2543–2550. pmid:23602536
- 20. Janz NK, Becker MH. The health belief model: A decade later. Health Education Quarterly. 1984;11(1):1–47. pmid:6392204
- 21. Smith PJ, Humiston SG, Marcuse EK, Zhao Z, Dorell CG, Howes C, et al. Parental delay or refusal of vaccine doses, childhood vaccination coverage at 24 months of age, and the Health Belief Model. Public Health Reports. 2011;126(2_suppl):135–146. pmid:21812176
- 22. Brown W, Ottney A, Nguyen S. Breaking the barrier: the Health Belief Model and patient perceptions regarding contraception. Contraception. 2011;83(5):453–458. pmid:21477689
- 23. Hanson JA, Benedict JA. Use of the Health Belief Model to examine older adults’ food-handling behaviors. Journal of Nutrition Education and Behavior. 2002;34:S25–S30. pmid:12047826
- 24. Kennedy AM, Brown CJ, Gust DA. Vaccine beliefs of parents who oppose compulsory vaccination. Public Health Reports. 2005;120(3):252–258. pmid:16134564
- 25. Reiter PL, Stubbs B, Panozzo CA, Whitesell D, Brewer NT. HPV and HPV vaccine education intervention: effects on parents, healthcare staff, and school staff. Cancer Epidemiology and Prevention Biomarkers. 2011;20(11):2354–2361.
- 26. Bolton-Maggs D, Conrad D, Keenan A, Lamden K, Ghebrehewet S, Vivancos R. Perceptions of mumps and MMR vaccination among university students in England: An online survey. Vaccine. 2012;30(34):5081–5085. pmid:22705081
- 27. Dannetun E, Tegnell A, Hermansson G, Giesecke J. Parents’ reported reasons for avoiding MMR vaccination: a telephone survey. Scandinavian Journal of Primary Health Care. 2005;23(3):149–153. pmid:16162466
- 28. Reiter PL, Brewer NT, Gottlieb SL, McRee AL, Smith JS. Parents’ health beliefs and HPV vaccination of their adolescent daughters. Social Science & Medicine. 2009;69(3):475–480.
- 29. Guest G, Bunce A, Johnson L. How many interviews are enough? An experiment with data saturation and variability. Field Methods. 2006;18(1):59–82.
- 30. Bond L, Nolan T. Making sense of perceptions of risk of diseases and vaccinations: a qualitative study combining models of health beliefs, decision-making and risk perception. BMC Public Health. 2011;11(1):943. pmid:22182354
- 31.
Robson C, McCartan K. Real World Research. John Wiley & Sons; 2016.
- 32. Flesch R. A new readability yardstick. Journal of Applied Psychology. 1948;32(3):221. pmid:18867058
- 33. Gross MS, Tran CH, Sutherland KH, Castagno JC, Amdur RJ. Pilot Study: Can an Educational Intervention Increase Human Papillomavirus Vaccination in Female College Students?. Obstetrics & Gynecology. 2014;123:114S–115S.
- 34. Dale J, Richards F, Bradburn J, Tadros G, Salama R. Student filmmakers’ attitudes towards mental illness and its cinematic representation-an evaluation of a training intervention for film students. Journal of Mental Health. 2014;23(1):4–8. pmid:24484185
- 35. Verran J, Crossley M, Carolan K, Jacobs N, Amos M. Monsters, microbiology and mathematics: the epidemiology of a zombie apocalypse. Journal of Biological Education. 2014;48(2):98–104.
- 36. McCallum S. Gamification and serious games for personalized health. Stud Health Technol Inform. 2012;177(2012):85–96. pmid:22942036
- 37. Thompson D, Baranowski T, Buday R, Baranowski J, Thompson V, Jago R, et al. Serious video games for health: How behavioral science guided the development of a serious video game. Simulation & Gaming. 2010;41(4):587–606.
- 38. Baranowski T, Blumberg F, Buday R, DeSmet A, Fiellin LE, Green CS, et al. Games for health for children—Current status and needed research. Games for Health Journal. 2016;5(1):1. pmid:26262772
- 39. Coil DA, Ettinger CL, Eisen JA. Gut Check: The evolution of an educational board game. PLoS Biology. 2017;15(4):e2001984. pmid:28453514
- 40. Viggiano A, Viggiano E, Di Costanzo A, Viggiano A, Andreozzi E, Romano V, et al. Kaledo, a board game for nutrition education of children and adolescents at school: cluster randomized controlled trial of healthy lifestyle promotion. European Journal of Pediatrics. 2015;174(2):217–228. pmid:25048788
- 41. Van Scoy LJ, Reading JM, Scott AM, Green MJ, Levi BH. Conversation game effectively engages groups of individuals in discussions about death and dying. Journal of Palliative Medicine. 2016;19(6):661–667. pmid:27022862
- 42.
Carolan K, Crossley M, Redfern J, Whitton N, Loutitt D, Amos M, et al.. SimFection: A digital resource for vaccination education; Submitted, 2017.
- 43. Gowda C, Schaffer SE, Kopec K, Markel A, Dempsey AF. A pilot study on the effects of individually tailored education for MMR vaccine-hesitant parents on MMR vaccination intention. Human Vaccines & Immunotherapeutics. 2013;9(2):437–445.
- 44. Porter-Jones G, Williams S, Powell C, Pusey L, Roberts R. Impact of a novel way to communicate information about MMR on uptake of MMR vaccine: a randomized controlled trial. Public Health. 2009;123(1):78–80. pmid:19081119
- 45. Shourie S, Jackson C, Cheater F, Bekker H, Edlin R, Tubeuf S, et al. A cluster randomised controlled trial of a web based decision aid to support parents’ decisions about their child’s Measles Mumps and Rubella (MMR) vaccination. Vaccine. 2013;31(50):6003–6010. pmid:24148574
- 46. Gowda C, Schaffer SE, Dombkowski KJ, Dempsey AF. Understanding attitudes toward adolescent vaccination and the decision-making dynamic among adolescents, parents and providers. BMC Public Health. 2012;12(1):509. pmid:22768870
- 47. Kato PM, Cole SW, Bradlyn AS, Pollock BH. A video game improves behavioral outcomes in adolescents and young adults with cancer: a randomized trial. Pediatrics. 2008;122(2):e305–e317. pmid:18676516
- 48. Arnab S, Brown K, Clarke S, Dunwell I, Lim T, Suttie N, et al. The development approach of a pedagogically-driven serious game to support Relationship and Sex Education (RSE) within a classroom setting. Computers & Education. 2013;69:15–30.
- 49. Fu LY, Bonhomme LA, Cooper SC, Joseph JG, Zimet GD. Educational interventions to increase HPV vaccination acceptance: a systematic review. Vaccine. 2014;32(17):1901–1920. pmid:24530401
- 50. Shapiro GK, Joyal-Desmarais K, Perez S, Rosberger Z. A response to Fu et al.’s “Educational interventions to increase HPV vaccination acceptance”. Vaccine. 2014;32(48):6342–6344. pmid:25306910
- 51. Marek E, Dergez T, Rebek-Nagy G, Szilard I, Kiss I, Ember I, et al. Effect of an educational intervention on Hungarian adolescents’ awareness, beliefs and attitudes on the prevention of cervical cancer. Vaccine. 2012;30(48):6824–6832. pmid:23000124
- 52.
Smeddinck JD. Games for Health. In: Entertainment Computing and Serious Games. Springer; 2016. p. 212–264.