http://orcid.org/0000-0003-4691-7802
Figures
Abstract
Background
Vaccine hesitancy has been little studied in low- and middle-income countries but is a potential concern because vaccine refusal may increase the burden of infectious diseases and impede control efforts. The aim of this study was to compare vaccine hesitancy between locals, long-time city residents, and non-locals, who have more recently moved to the city from either other urban or rural areas, in Shanghai, China.
Methods
Parents of infants ≤3 months of age were surveyed at immunization clinics in Shanghai, China. Participants completed a paper questionnaire utilizing the 10-item Vaccine Hesitancy Scale, which was developed by the World Health Organization Strategic Advisory Group of Experts on Immunization. Items were grouped based on internal consistency, and regressed onto demographic variables using a negative binomial model.
Results
In total, 1,188 (92.5%) individuals participated. For most items on the scale, parents expressed positive beliefs about vaccines. However, about half of parents somewhat or strongly agreed that new vaccines carried more risks than older vaccines, and 71.6% somewhat or strongly agreed that they were concerned about serious adverse effects. Seven items from the Vaccine Hesitancy Scale were highly correlated and mapped onto “lack of confidence”; the other three items were analysed separately. Compared to mothers, fathers had less lack of confidence (β: -0.06, 95% CI: -0.12, -0.01), and individuals living in the outer suburbs (β: 0.13, 95% CI: 0.01, 0.25) and rural non-locals (β: 0.10, 95% CI: 0.02, 0.18) had greater lack of confidence in vaccines compared to their urban or local counterparts, respectively.
Discussion
Shanghai parents professed confidence in certain vaccine benefits, but vaccine messaging could focus on addressing misconceptions about vaccines for diseases no longer common, newer vaccines, and adverse effects associated with vaccination. These messages may need to be separately tailored to locals and non-locals, who have differing concerns.
Citation: Ren J, Wagner AL, Zheng A, Sun X, Boulton ML, Huang Z, et al. (2018) The demographics of vaccine hesitancy in Shanghai, China. PLoS ONE 13(12): e0209117. https://doi.org/10.1371/journal.pone.0209117
Editor: Italo Francesco Angelillo, University of Campania, ITALY
Received: September 27, 2018; Accepted: November 29, 2018; Published: December 13, 2018
Copyright: © 2018 Ren et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The questionnaire, dataset, and code to create derived variables are publicly available at a figshare repository (https://doi.org/10.6084/m9.figshare.6463304).
Funding: This work was supported by The Fourth Round of Three-Year Public Health Action Plan of Shanghai, China (No. 15GWZK0101). This study was funded through an MCubed grant from the University of Michigan (https://mcubed.umich.edu/). AZ received an international internship funded by the Office of Global Public Health at the University of Michigan School of Public Health (https://sph.umich.edu/global/funding.html), and ALW’s salary was funded by a Health Outcomes Post Doctoral Fellowship from the Pharmaceutical Research and Manufacturers of America Foundation (PhRMA Foundation) (http://www.phrmafoundation.org/). The funders did not play any role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
The provision of vaccines through the government-funded Expanded Program on Immunization (EPI) has been a successful means of controlling many infectious diseases globally [1,2]. China’s EPI was started in 1978 and initially included vaccines for tuberculosis, polio, measles, pertussis, diphtheria, and tetanus, and was later expanded to include several additional vaccines in 2002 and again in 2007 [1].
Progress in the control of vaccine-preventable diseases can by stymied by vaccine hesitancy and resultant low uptake of vaccines. A World Health Organization (WHO) Strategic Advisory Group of Experts on Immunization (SAGE) working group defines vaccine hesitancy as a “delay in acceptance or refusal of vaccines despite availability of vaccination services. Vaccine hesitancy is complex and context specific varying across time, place and vaccines. It includes factors such as complacency, convenience and confidence” [3].
The relationship between individual decisions not to vaccinate and population-level outbreaks is particularly relevant for highly infectious diseases like measles and pertussis [4,5]. Vaccine hesitancy has been predominantly studied in high income countries [6,7], although some limited information from low- and middle-income countries (LMICs) is available [8]. One review found that, in 2016, national assessments of vaccine hesitancy were undertaken in 47% of low income countries, 45% of lower middle income countries, 22% of upper middle income countries, and 53% of high income countries [9]. Clearly, more information is needed from upper middle income countries like China.
Vaccine hesitancy is likely to vary by demographic conditions, educational background, and experiences with diseases and vaccines. Residency is one such grouping: non-locals, sometimes referred to as the “floating population,” are migrants from rural areas to urban cities (“rural non-locals”) or from other urban areas in China to another city (“urban non-local”) [10], and typically have less access to governmental entitlement programs than locals [10,11]. However, non-local children, like locals, receive all EPI vaccines for free. Of China’s 1.3 billion residents, 260 million are non-locals; and in Shanghai, 9 of 23 million residents are non-locals. Non-locals often cluster within neighborhoods in suburban districts of cities because of affordability and location of jobs: urban districts represent historical business areas, whereas suburban districts are more industrial. Regardless of residency, health statuses in suburban areas are expected to be poorer than urban areas because of lower individual socioeconomic status and because there is less access to public services in these areas [12,13]. Because of their socioeconomic and legal circumstances, it is highly plausible that medical decision-making, including decisions about vaccines, differs between locals and non-locals [14].
More information is needed about the development of vaccine hesitancy in LMICs like China, and within special populations, like non-locals or migrants. The aim of this study was to describe the features of vaccine hesitancy among parents of young infants and to compare vaccine hesitancy across different demographic groups, in particular between locals and non-locals in Shanghai, China.
Methods
Study population
This cross-sectional study utilized a stratified, two-stage cluster sample in Shanghai, China. The first stage was at the township level with city districts forming a sampling strata, such that at least one township was selected from 15 districts (i.e., all districts in the municipality with the exception of Chongming district, which is an insular, less densely-populated district). We selected 40 townships based on the population of children in the township according to the 2010 China Census. Within each township, study staff visited an immunization clinic and selected 30 caregivers (preferentially mothers or fathers) based on a convenience sample of who was visiting the clinic that day. Every township has an immunization clinic, and, therefore, the catchment area for the immunization clinic is coterminous with the township. The inclusion criterion was a parent or grandparent of a child <3 months of age. All participants had to be at least 18 years.
The sample size calculation was estimated based on another aim of the study: to evaluate longitudinal relationships between vaccine hesitancy and vaccine uptake. Based on a previous survey looking at uptake of pneumococcal vaccination—a non-EPI vaccine which requires payment (data unpublished), we found that those who thought that the vaccine was more necessary had an uptake of 13.90%, whereas those who perceived the vaccine as unnecessary had an uptake of 5.49%. A sample size of 191 in each group (i.e. those who think necessary and those who think unnecessary) would be required to see a statistically significant relationship of this size given 80% power and a 95% confidence level. Assuming 50% in each category, this would mean we would need 382 individuals. Given an intra-cluster coefficient of 0.02389 (estimated from this previous data source), that meant 647 individuals needed to be enrolled.
Data collection occurred between May and September 2017.
Vaccine Hesitancy Scale
The 10-item Vaccine Hesitancy Scale was developed by the WHO SAGE Working Group to address Vaccine Hesitancy [15]. Each item was assessed on a 5-point Likert scale; to make directionality uniform across all items, some items’ response values were flipped, such that the highest number represented more hesitant beliefs. Based on internal consistency of scale items and from considerations of previous research [16], seven items (L1-L4, L6-L8) were grouped into the component “Lack of confidence,” and the other three items were analyzed separately. To create a scale for the component “lack of confidence,” the average of all seven items was taken in order to preserve the original 1–5 scale. The other three items (concerns about side effects, newer vaccines, and the continued use of vaccines for relatively controlled diseases) were analyzed individually. Fig 1 contains an English translation of all questions.
Statistical analysis
The distribution of demographic characteristics is displayed using descriptive statistics. We assessed the differences in the proportion of various demographic characteristics between locals and non-locals through a Rao-Scott chi-square test. Internal consistency of the scale was analyzed using Crohnbach’s alpha. We regressed components of the Vaccine Hesitancy Scale onto demographic characteristics using generalized estimating equations with a negative binomial distribution and identity link and using an exchangeable correlation matrix. This regression model output β estimates that represent the increase in hesitancy for a given change in the dependent variable.
Analyses used weighting and survey procedures when possible. The sampling weights derived from the complex sampling scheme and the inverse probability of selection for each participant. The weights are designed to make the study population generalizable to the population of parents of young infants in Shanghai. The analyses also clustered observations by township. We assessed significance of results an alpha level of 0.05, and precision of results through 95% confidence intervals (CI).
Ethical approval
The study protocol was approved by the University of Michigan Health Sciences and Behavioral Sciences Institutional Review Board (#HUM00125379) and an ethical review committee from the Shanghai Centers for Disease Control and Prevention (#2017–2). Participants gave written informed consistent prior to data collection.
Results
In total, 1,285 individuals were approached to participate and 1,188 (92.5%) consented and were enrolled in the study. Table 1 shows the demographic distribution of the study population. Locals were 48.3% of the population; non-locals formed the remainder, with 15.8% originally from urban areas and 35.9% originally from rural areas., Some characteristics varied by residency status with urban and rural non-locals comprising a larger proportion of the population in the inner and outer suburbs than in the urban districts (P<0.0001). Additionally, rural non-locals reported less personal experience with vaccine-preventable diseases (22.8%) compared to locals (24.8%) and urban non-locals (32.9%) who had similar proportions (P = 0.0021).
Fig 1 shows the distribution of responses for each item on the scale. A large majority of parents (69.7%) strongly agreed that vaccines are important for their child’s health (item L1), and about half strongly agreed that vaccines are a good way to protect their child from disease (item L7) and that vaccinating their child is important for the health of others (item L3). For most other items, parents expressed positive beliefs about vaccines. However, about half of parents somewhat or strongly agreed that new vaccines carried more risks than older vaccines (item L5), and 71.6% somewhat or strongly agreed that they were concerned about serious adverse effects (item L9).
Internal consistency was acceptable for the entire 10-item scale (Crohnbach’s alpha = 0.73). It was improved by removing items L5, L9, and L10. The other 7 items had good internal consistency (Crohnbach’s alpha = 0.89) and mapped onto the component “lack of confidence.” The other three items had poor internal consistency (Crohnbach’s alpha = 0.48 for L5, L9, and L10, or 0.37 for L5 and L9 without L10). Accordingly, these three other items were analyzed separately. Details of the factor loadings from an exploratory and confirmatory factor analysis are in S1 File.
Multivariable regression model results for the “lack of confidence” component and three other items are shown in Table 2. For each model, positive β estimates represent greater degrees of hesitancy and negative β estimates represent less hesitancy, relative to reference group. We observed significant socioeconomic differentials for all four measures of vaccine hesitancy. However, the item “concerns about side effects” had the least amount of variation, while also having the highest intercept, indicating that most individuals expressed a relatively large amount of concern, regardless of their background.
Residency was a significant predictor for all vaccine hesitancy measures, except for “concerns about side effects.” There was no significant difference in the lack of confidence between locals and urban non-locals, but rural non-locals had less confidence than locals (β: 0.10, 95% CI: 0.02, 0.18). However, compared to locals, both urban and rural non-locals had less concerns about new vaccines being risk or about some vaccines no longer being needed.
There were also significant differences by parent type, education, and income. Compared to mothers, fathers had more confidence (β: -0.06, 95% CI: -0.12, -0.01) and were less concerned about side effects (β: -0.13, 95% CI: -0.25, -0.01). However, they also expressed greater beliefs that new vaccines were riskier than older ones (β: 0.21, 95% CI: 0.07, 0.36). Education and income was related to beliefs about new vaccines being risky, but not the other vaccine hesitancy measures. Individuals with lower levels of education or lower income levels had greater degrees of belief that new vaccines were riskier. For example, compared to those with a graduate school education, those with a high school education were 0.53 points higher on the 5-point Likert scale (95% CI: 0.22, 0.83).
Discussion
Vaccine hesitancy is a globalized phenomenon with the potential consequence of impeding efforts to control or eradicate preventable but potentially debilitating diseases. This study, based in Shanghai, a populous area undergoing rapid economic development and is, consequently, a destination for many internal migrants, found that parents expressed positive beliefs about certain aspects of vaccination. However, those positive perceptions were tempered among most parents by concerns about adverse events of vaccination and doubts about both newer vaccines and vaccines for diseases that were no longer common.
We found that residency was a strong predictor of vaccination beliefs. Individuals with a rural, non-local residency permit had less belief in vaccine benefits but were also less concerned about side effects and vaccines for uncommon diseases. These beliefs were not influenced by experience with vaccine-preventable diseases. Interestingly, the proportion of individuals with personal experience having a vaccine-preventable disease varied significantly by residency, with rural non-locals much less likely to have such experience.
In summary, the group of individuals who had more experience with vaccine-preventable diseases (locals) also had more confidence in vaccines while also expressing more concerns in the riskiness of new vaccines and in the utility of certain vaccines. In a previous study in Shanghai, we also did not observe a relationship between experience with disease and perceptions of vaccine necessity [17]. Although we had initially hypothesized that exposure to vaccine-preventable disease would predispose an individual to wanting that vaccine, other research has shown that the opposite association can exist. Researchers in Arizona, for example, found that parents who obtained a nonmedical vaccine exemption for their child (and who could therefore be labeled as relatively vaccine hesitant) were much more likely to report knowing someone who had a vaccine-preventable disease compared to those who did not obtain an exemption [18].
Our findings had both similarities and differences to a previously published paper analyzing differences across demographic groups using this scale [16]. We have found the opposite association compared to the Canadian study [16], in that our study showed fathers to have more confidence in vaccines but heightened concern about certain vaccine risks compared to mothers. However, similar to the Canadian study, we found that individuals with less education had more concerns about newer vaccines. Although there is rising concern globally that higher educated groups express more concerns about vaccines and more frequently seek exemptions [19], this is not a uniform pattern [18]. In past research, we have also found that some higher educated groups in Shanghai have more concerns about vaccine scheduling [20], but in this study, we did not find many significant differences by education, except that lower educated groups had more concerns about the riskiness of new vaccines. One systematic review of studies from LMICs posits that maternal education directly impacts vaccination awareness and attitudes [21]. And results from our study suggest that vaccination providers may be more effective if they tailor their discussion of vaccines to individuals’ educational levels with special consideration given to discussing the importance of new vaccines and their associated risks with individuals who are less educated.
Public health significance
The three items that did not map onto the component “lack of confidence” represent important areas for public health officials to promote. Caregivers show clear concerns about side effects, newer vaccines, and the continued use of vaccines for relatively controlled diseases. These concerns are not exclusive to the general population; qualitative interviews with doctors in Alberta, Canada, found that health care professionals, while overall holding very positive beliefs (or attitudes) about vaccines also expressed reservations about the newness of certain vaccines, in addition to distrust of pharmaceutical companies and of vaccine schedules [22]. Therefore, from a societal perspective, limiting the diffusion of vaccine hesitancy will rely on education of vaccination providers on both the benefits of vaccines and on best practices to communicate with parents and other caregivers. Doctors are trusted sources of health information and are therefore pivotal in promoting the use of vaccines [23,24]. For example, in a series of studies from Italy, information from physicians was associated with greater knowledge of human papillomavirus vaccination in pregnant women [25], adolescent and young adult men [26], and parents of young men [27]. However, in order to avoid exacerbating intransigency from caregivers, vaccination providers should be able to briefly provide caregivers with facts about the safety of vaccines, and be willing to ask open-ended questions and listen to the concerns of parents [28].
Caregivers in this study believed vaccines were important for both their own child’s health and for the health of others. It follows that another tactic—to increase confidence in vaccines among caregivers and promote their decision to obtain a vaccine for a child—would be for vaccination providers to talk with them about how some children have a contraindication to vaccination, and so vaccinating one’s own child can have a positive impact on other community members.
Use of vaccine Hesitancy Scale
Our future research will focus on the relationship between vaccine hesitancy, as quantified through baseline results in this Vaccine Hesitancy Scale, and parents’ decision to obtain mandatory and non-mandatory vaccines. In the absence of this predictive validity, we advise the use of the scale with the following observations: Similar to studies in Canada [16] and Guatemala [29], we found that the Vaccine Hesitancy Scale, as developed by Larson et al. [15], had adequate internal consistency, which was improved by removing items L5, L9, and L10. Unlike the Canadian study, we did not think that items L5 and L9 were correlated highly enough to combine into one construct. We also remark that these three items (L5, L9, and L10) are worded in the opposite direction from other items; that is, a higher value of response in the original survey for these items indicated higher vaccine hesitancy, whereas higher values of response for other items indicated lower vaccine hesitancy. Individuals quickly completing a survey may mistake this directionality. To counter this issue and minimize bias, the order of questions could be randomized, and, if appropriate, the directionality of certain questions could be randomly flipped for some respondents.
Strengths and limitations
This study has several strengths and limitations. As a convenience sample, we are likely to have a population biased towards accepting vaccines and being more health conscious. However, we note that vaccines are mandatory in China for school entry, and in one study of migrant parents in Beijing, only 11.9% had not gone to an immunization clinic [30]. In addition, although residency status is an objective measure in that it is an official government designation, individuals can change their residency status (for example, through education or an official job). In this way our study could have found a diluted association between residency status and hesitancy if non-locals are transitioning into being locals. A strength of this study is that townships were selected throughout Shanghai, generating a population representative of city. Additionally, by selecting caregivers of young children (<3 months of age), we were able to interview parents at an early point in their child’s life, when they have likely had little prior exposure to vaccinations. A future longitudinal study could examine how attitudes towards vaccinations change in this population as the parents are exposed to more material from doctors, social media, and news media.
Conclusions
This study used a Vaccine Hesitancy Scale, developed by a the WHO SAGE [15] and validated within a Canadian population [16], to characterize vaccine confidence and beliefs about certain risks within Shanghai, China. China’s ability to add more vaccines into their vaccination schedule and to promote the distribution of non-mandatory vaccines could be limited by concerns about adverse events following immunization and the risks of newer vaccines. Countering these concerns will be particularly important in lower educated groups and among non-locals who have more recently moved into a city. Rapid urbanization is a hallmark of many LMICs, and so contrasting and comparing the values that migrants and non-migrants hold will be increasingly important when developing health promotion programs in China and other countries [31].
Supporting information
S1 File. Factor loadings from the vaccine Hesitancy Scale.
https://doi.org/10.1371/journal.pone.0209117.s001
(PDF)
Acknowledgments
We appreciate the staff at immunization clinics throughout the municipality of Shanghai who opened their clinics to our study.
References
- 1. Zheng J, Zhou Y, Wang H, Liang X. The role of the China Experts Advisory Committee on Immunization Program. Vaccine. 2010;28S: A84–A87. https://doi.org/10.1016/j.vaccine.2010.02.039
- 2. Wang L, Wang Y, Jin S, Wu Z, Chin D. Emergence and control of infectious diseases in China. Lancet. 2008;372: 34–41. https://doi.org/10.1016/S0140-6736(08)61365-3
- 3.
The Strategic Advisory Group of Experts (SAGE). Report of the SAGE working group on vaccine hesitancy [Internet]. 2014 [cited 14 Jun 2018]. http://www.who.int/immunization/sage/meetings/2014/october/SAGE_working_group_revised_report_vaccine_hesitancy.pdf
- 4. Omer SB, Salmon DA, Orenstein WA, Halsey N, DeHart MP, Halsey N, et al. Vaccine refusal, mandatory immunization, and the risks of vaccine-preventable diseases. N Engl J Med. 2009;360: 1981–8. https://doi.org/10.1056/NEJMsa0806477
- 5. Phadke VK, Bednarczyk RA, Salmon DA, Omer SB. Association Between Vaccine Refusal and Vaccine-Preventable Diseases in the United States. JAMA. 2016;315: 1149. https://doi.org/10.1001/jama.2016.1353
- 6. Napolitano F, D’Alessandro A, Angelillo IF. Investigating Italian parents’ vaccine hesitancy: A cross-sectional survey. Hum Vaccines Immunother. 2018;14: 1558–1565. https://doi.org/10.1080/21645515.2018.1463943
- 7. Henrikson NB, Anderson ML, Opel DJ, Dunn J, Marcuse EK, Grossman DC. Longitudinal trends in vaccine hesitancy in a cohort of mothers surveyed in Washington State, 2013–2015. Public Health Rep. 2017;132: 451–454. https://doi.org/10.1177/0033354917711175
- 8. Cobos Munoz D, Monzon Llamas L, Bosch-Capblanch X. Exposing concerns about vaccination in low- and middle-income countries: a systematic review. Int J Public Health. 2015;60: 767–780. https://doi.org/10.1007/s00038-015-0715-6
- 9. Lane S, MacDonald NE, Marti M, Dumolard L. Vaccine hesitancy around the globe: Analysis of three years of WHO/UNICEF Joint Reporting Form data-2015–2017. Vaccine. 2018;36: 3861–3867. https://doi.org/10.1016/j.vaccine.2018.03.063
- 10.
Chan K. China: internal migration. In: Ness I, Bellwood P, editors. The Encyclopedia of Global Human Migration. Hoboken, NJ: Wiley-Blackwell; 2012. https://doi.org/10.1002/9781444351071
- 11. Gong P, Liang S, Carlton EJ, Jiang Q, Wu J, Wang L, et al. Urbanisation and health in China. Lancet. 2012;379: 843–52. https://doi.org/10.1016/S0140-6736(11)61878-3
- 12. Li H, Wang Q, Shi W, Deng Z, Wang H. Residential clustering and spatial access to public services in Shanghai. Habitat Int. 2015;46: 119–129. https://doi.org/10.1016/j.habitatint.2014.11.003
- 13.
Yang S, Wang C, Wang M. Synergistic evolution of Shanghai urban economic development transition and social spatial structure. In: Wang MY, Kee P, Gao J, editors. Transforming Chinese Cities. New York, NY: Routledge; 2014. pp. 48–74.
- 14. Peng Y, Chang W, Zhou H, Hu H, Liang W. Factors associated with health-seeking behavior among migrant workers in Beijing, China. BMC Health Serv Res. 2010;10: 69. https://doi.org/10.1186/1472-6963-10-69
- 15. Larson HJ, Jarrett C, Schulz W, Chaudhuri M, Zhou Y, Dube E, et al. Measuring vaccine hesitancy: The development of a survey tool. Vaccine. 2015;33: 4165–4175. https://doi.org/10.1016/j.vaccine.2015.04.037
- 16. Shapiro GK, Tatar O, Dube E, Amsel R, Knauper B, Naz A, et al. The vaccine hesitancy scale: Psychometric properties and validation. Vaccine. 2018;36: 660–667. https://doi.org/10.1016/j.vaccine.2017.12.043
- 17. Wagner AL, Boulton ML, Sun X, Mukherjee B, Huang Z, Harmsen IA, et al. Perceptions of measles, pneumonia, and meningitis vaccines among caregivers in Shanghai, China, and the health belief model: A cross-sectional study. BMC Pediatr. 2017;17. https://doi.org/10.1186/s12887-017-0900-2
- 18. Pottinger HL, Jacobs ET, Haenchen SD, Ernst KC. Parental attitudes and perceptions associated with childhood vaccine exemptions in high-exemption schools. PLoS One. 2018;13: e0198655. https://doi.org/10.1371/journal.pone.0198655
- 19. Edwards KM, Hackell JM, The Committee on Infectious Diseases, The Committee on Practice and Ambulatory Medicine. Countering Vaccine Hesitancy. Pediatrics. 2016;138: e20162146. https://doi.org/10.1542/peds.2016-2146
- 20. Wagner AL, Boulton ML, Sun X, Huang Z, Harmsen IA, Ren J, et al. Parents’ concerns about vaccine scheduling in Shanghai, China. Vaccine. 2017;35. https://doi.org/10.1016/j.vaccine.2017.06.077
- 21. Phillips DE, Dieleman JL, Lim SS, Shearer J. Determinants of effective vaccine coverage in low and middle-income countries: a systematic review and interpretive synthesis. BMC Health Serv Res. BMC Health Services Research; 2017;17: 681. https://doi.org/10.1186/s12913-017-2626-0
- 22. Manca T. “One of the greatest medical success stories:” Physicians and nurses’ small stories about vaccine knowledge and anxieties. Soc Sci Med. 2018;196: 182–189. https://doi.org/10.1016/j.socscimed.2017.11.027
- 23. Napolitano F, Navaro M, Vezzosi L, Santagati G, Angelillo IF. Primary care pediatricians’ attitudes and practice towards HPV vaccination: A nationwide survey in Italy. PLoS One. 2018;13: e0194920. https://doi.org/10.1371/journal.pone.0194920
- 24. Smith LE, Amlôt R, Weinman J, Yiend J, Rubin GJ. A systematic review of factors affecting vaccine uptake in young children. Vaccine. 2017;35: 6059–6069. https://doi.org/10.1016/j.vaccine.2017.09.046
- 25. D’Alessandro A, Napolitano F, D’Ambrosio A, Angelillo IF. Vaccination knowledge and acceptability among pregnant women in Italy. Hum Vaccines Immunother. 2018;14: 1573–1579. https://doi.org/10.1080/21645515.2018.1483809
- 26. Napolitano F, Napolitano P, Liguori G, Angelillo IF. Human papillomavirus infection and vaccination: Knowledge and attitudes among young males in Italy. Hum Vaccines Immunother. 2016;12: 1504–1510. https://doi.org/10.1080/21645515.2016.1156271
- 27. Bianco A, Pileggi C, Iozzo F, Nobile CGA, Pavia M. Vaccination against human papilloma virus infection in male adolescents: Knowledge, attitudes, and acceptability among parents in Italy. Hum Vaccines Immunother. 2014;10: 2536–2542. https://doi.org/10.4161/21645515.2014.969614
- 28. McClure CC, Cataldi JR, O’Leary ST. Vaccine Hesitancy: Where We Are and Where We Are Going. Clin Ther. 2017;39: 1550–1562. https://doi.org/10.1016/j.clinthera.2017.07.003
- 29. Domek GJ, O’Leary ST, Bull S, Bronsert M, Contreras-Roldan IL, Bolaños Ventura GA, et al. Measuring vaccine hesitancy: Field testing the WHO SAGE Working Group on Vaccine Hesitancy survey tool in Guatemala. Vaccine. 2018;36: 5273–5281. https://doi.org/10.1016/j.vaccine.2018.07.046
- 30. Sun M, Ma R, Zeng Y, Luo F, Zhang J, Hou W. Immunization status and risk factors of migrant children in densely populated areas of Beijing, China. Vaccine. 2010;28: 1264–1274. https://doi.org/10.1016/j.vaccine.2009.11.015
- 31. Crocker-Buque T, Mindra G, Duncan R, Mounier-Jack S. Immunization, urbanization and slums—A systematic review of factors and interventions. BMC Public Health. 2017;17: 1–16. https://doi.org/10.1186/s12889-017-4473-7