Figures
Abstract
Introduction
Adverse events following immunization (AEFIs) are unwanted or unexpected health outcomes following vaccination, which may or may not be causally-linked to vaccines. AEFI reporting is important to post-marketing vaccine safety surveillance and has the potential to identify new or rare AEFIs, show increases in known AEFIs, and help to maintain public confidence in vaccine programs. Knowledge of the expected incidence (i.e. background rate) of a possible AEFI is essential to the investigation of vaccine safety signals. We selected three rarely reported AEFIs representing the spectrum of causal association with vaccines, from proven (immune thrombocytopenia [ITP]) to questioned (Kawasaki disease [KD]) to unsubstantiated (multiple sclerosis [MS]) and determined their background rates.
Methods
We extracted data on hospitalizations (CIHI Discharge Abstract Database) for ITP, KD, and MS among Ontario children for the period 2005 to 2014 from IntelliHEALTH. As ITP can be managed without hospitalization, we also extracted emergency department (ED) visits from the CIHI National Ambulatory Care Reporting System. For all conditions, we only counted the first visit and if the same child had both an ED visit and a hospitalization for ITP, only the hospitalization was included. We calculated rates by year, age group and sex using population estimates from 2005–2014, focusing on age groups within the Ontario immunization schedule around vaccine(s) of interest.
Results
Per 100,000 population, annual age-specific incidence of ITP in children age 1 to 7 years ranged from 8.9 to 12.2 and annual incidence of KD in children less than 5 years ranged from 19.1 to 32.1. Average annualized incidence of adolescent (11–17 years) MS across the study period was 0.8 per 100,000.
Citation: Wormsbecker AE, Johnson C, Bourns L, Harris T, Crowcroft NS, Deeks SL (2019) Demonstration of background rates of three conditions of interest for vaccine safety surveillance. PLoS ONE 14(1): e0210833. https://doi.org/10.1371/journal.pone.0210833
Editor: Mark Simonds Riddle, Uniformed Services University of the Health Sciences, UNITED STATES
Received: September 14, 2018; Accepted: January 2, 2019; Published: January 15, 2019
Copyright: © 2019 Wormsbecker et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Unfortunately, Public Health Ontario (PHO) cannot disclose the underlying data. Doing so would compromise individual privacy contrary to PHO’s ethical and legal obligations. Restricted access to the data may be available under conditions prescribed by the Ontario Personal Health Information Protection Act, 2004, the Ontario Freedom of Information and Protection of Privacy Act, the Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans (TCPS 2 (2014)), and PHO privacy and ethics policies. Data are available for researchers who meet PHO’s criteria for access to confidential data, by filling in a Data Access Request Form, which can be found at: https://www.publichealthontario.ca/en/About/Documents/PHO_Data_Request_Form.pdf.
Funding: The authors received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Monitoring for potential adverse events following immunization (AEFI), whether they are in relation to new vaccines, immunization schedule changes, or existing vaccines, is the core of post-marketing vaccine safety surveillance. In the Canadian province of Ontario, AEFI reporting by designated health care providers is mandatory under the Health Promotion and Protection Act 1990; however, vaccine recipients or their caregivers can also voluntarily report AEFI. With the exception of active surveillance at two children’s hospitals, most vaccine safety monitoring is through passive surveillance. A full description of AEFI surveillance in Ontario has been published elsewhere [1]. We undertook the present analysis in the context of Ontario’s broader efforts, including revised case definitions and the publication of an annual report [1], to strengthen its vaccine safety surveillance.
The background rate of diseases that could be AEFI can be used to estimate the number of expected events within a known population over a period of time. When AEFI are reported, further investigation may be warranted if the number of observed events is greater than would normally be expected based on the background rate of the condition of interest. Knowledge of background rates are especially useful when a new vaccine is introduced or a schedule change is made. Several studies during and following the 2009–10 H1N1 pandemic addressed the importance of understanding the existing background rates of relevant rare and serious medical conditions that could be reported as AEFI (e.g. Guillain-Barre syndrome [GBS] after influenza immunization) when assessing the safety of a mass vaccination campaign [2–4].
We used health care administrative data to explore Ontario’s background rates of three rarely reported AEFI in childhood that have the potential to raise concerns in relation to current or future immunization programs. The three conditions, immune thrombocytopenia (ITP), Kawasaki disease (KD) and multiple sclerosis (MS), respectively, represent a range of causal association with immunization–from proven, to questioned, to unsubstantiated. Thrombocytopenia has a proven causal relationship with measles-containing vaccine [5–8] and, to a lesser degree, other vaccines [9]. The median estimated incidence of thrombocytopenia due to the measles, mumps, rubella (MMR) vaccine is 2.6 cases per 100,000 doses (range 0.09 to 4 cases per 100,000 doses) [10]. It normally occurs within 6 weeks after vaccination [7] and is more likely to occur after the first dose [11]. Ontario’s publicly-funded vaccination program includes MMR vaccine at one year and measles-mumps-rubella-varicella (MMRV) vaccine at 4–6 years [12].
KD has been consistently reported in passive paediatric AEFI surveillance [1,13,14]. Given the proposed role of an immunologic response in the unknown etiology of this acquired systemic childhood vasculitis, immunization has been questioned as a possible cause [15,16], but a recent systematic review found no association between KD and immunization [17]. Of note, active hospital-based surveillance of KD in Ontario has shown that the incidence of KD in Ontario may be one of the highest outside of Asia [18].
Unsubstantiated hypotheses drawing public, media, and even legal, attention [19,20] have been raised that immunization, specifically hepatitis B and HPV vaccines [21,22], may have an association with multiple sclerosis (MS), but numerous studies and reports have found no causal relationship [23,24]. In Ontario, hepatitis B vaccine is given in Grade 7 (age approximately 12 years) and human papillomavirus (HPV) vaccine was initially given to Grade 8 females beginning in 2007, with the addition of males and change to Grade 7 in 2016.
Our study had three objectives. The first was to determine the rates of ITP, KD and MS in AEFI-relevant age groups and the second was to demonstrate that appropriate background rates can be found from readily available Ontario health care administrative data. Finally, we aimed to demonstrate the use of background rates by applying our findings to a hypothetical vaccinated cohort to determine an expected number of AEFI.
Materials and methods
Ontario is Canada’s largest province (population ~14 million, 2016 [25]) and has a single-payer, publicly-funded health care system. Provincial health care administrative data are collected by the Canadian Institute for Health Information (CIHI) and provided to the Ministry of Health and Long-Term Care (MOHLTC). CIHI data are included in the MOHLTC’s IntelliHEALTH knowledge repository, which contains de-identified data collected from various sectors of Ontario’s health care system. Public health, health care and government analysts, epidemiologists, planners, policy and decision makers, and researchers have secure online access to IntelliHEALTH. This access is provided through the MOHLTC’s information access policies, based on the Personal Health Information Protection Act. Of note, although IntelliHEALTH is readily available to local public health agencies, it does not contain immunization information.
While the Brighton Collaboration refers to low platelets after immunization simply as thrombocytopenia (platelets less than 150 x 109/L) rather than ITP, to differentiate vaccine-associated thrombocytopenia from that with unknown etiology [10], we studied ITP as a case would likely be recorded as ITP in health care administrative data prior to making an association with a vaccine. KD and MS were defined based on International Classification of Diseases (ICD-10) diagnostic codes in administrative data. We were not able to separate KD cases into definite, probable and possible KD in the manner of the Brighton Collaboration [17]. See S1 Table for ICD-10 codes used for each medical condition.
We carried out descriptive analyses of case counts and rates over time. Incidence of health care utilization for ITP, KD and MS in relevant age groups were calculated using data from CIHI’s Discharge Abstract Database (CIHI-DAD Most Responsible Diagnosis [MRDx] codes) and the National Ambulatory Care Reporting System (CIHI-NACRS), containing information about hospitalization and emergency department (ED) visits, respectively.
We extracted data for these three conditions on February 5, 2016 and included the time period from January 1, 2005 to December 31, 2014. We limited our population to children and adolescents, and specifically to age groups that reflect Ontario’s routine, publicly-funded immunization schedule [12] for vaccines questioned to have an association with each of our selected conditions. For ITP, we assessed children aged 1–7 years as both scheduled measles-containing vaccine doses should be given by then; for KD we studied all children 0–9 years, based on age-related risk of KD since there is no specific vaccine associated with this condition; and for MS we focused on adolescents 11–17 years as the school-based hepatitis B and HPV vaccines are given within the early part of this age band.
We only considered individuals once for each condition, in an attempt to separate incident events from repeat health care encounters for the same illness. For ITP, we included both hospitalized individuals and those who had an ED visit because ITP can be managed without hospitalization, however if they were present in both data sources, we only counted the hospitalization. We only included cases of KD and MS that were hospitalized given the likely need for hospitalization to make diagnosis and provide treatment at the time of first presentation in children. We also used the ITP incidence from our analysis to explore the expected number of AEFI cases given estimated MMR coverage and the associated ITP risk from the literature.
Results
Age-specific counts and incidences of health care utilization for ITP, KD and MS (also sex-specific) are shown for the age groups of interest in Table 1 for the study period overall.
A total of 1,053 children age 1 to 7 years received a diagnosis of ITP during the 10-year study period. The majority of these (n = 804, 76%) were hospitalized. Annual incidence rate of ITP was relatively constant; varying slightly between 8.9 per 100,000 (2013) and 12.2 per 100,000 (2012). The incidence of ITP was highest among children aged 1 year (17.6 per 100,000) and decreased in subsequent ages. Boys aged 1 year had the highest incidence of any group (20.1 per 100,000). Higher rates of ITP were observed in boys versus girls up to the age of 6 years, with the male to female ratio decreasing as age increased. At seven years of age, the female rate was higher than that of males (Fig 1).
A total of 2,122 children less than 10 years were hospitalized with a diagnosis of KD during the study period. The average annual incidence rate of KD in this age group was 14.4 per 100,000 for the entire study period. As displayed in Fig 2, the annual incidence varied between a low of 12.3 per 100,000 (2007) and a high of 19.5 per 100,000 (2005). When the age group was restricted to younger children, rates were higher. The average annual incidence among children less than 5 years was 22.7 per 100,000 children and ranged from a low of 19.1 (2012) to a high of 32.1 (2005) per 100,000. When assessed by age and sex, the rate of KD peaked at 1–2 years of age and decreased in subsequent age groups (Fig 3). Boys aged 1 and 2 years had the highest incidence at 31.7 and 30.7 per 100,000, respectively. Higher rates of KD were seen in males, with the male to female ratio decreasing from infancy into older childhood (Fig 3).
The total number of adolescent (age 11–17 years) MS hospitalizations in Ontario from 2005 to 2014 was very low (n = 88), and the average annualized incidence rate across the study period was 0.8 per 100,000 adolescents (Table 1). We found three times as many cases in girls (n = 66) than boys (n = 22).
To demonstrate how these findings are applicable to public health, we used the known risk of ITP following MMR immunization and immunization coverage of a population to determine the number of cases of ITP attributable to the first dose of MMR. The risk of ITP after MMR vaccine has been estimated as 2.6 cases per 100,000 doses [10]. The Ontario annual birth cohort is approximately 140,000, with an estimated 96% MMR coverage [26], (based on one dose rubella coverage as Ontario vaccination coverage is reported by antigen and measles and mumps coverage are only assessed after two doses). Thus, approximately 134,400 one-year-olds in Ontario are vaccinated with MMR each year. If the risk of ITP following MMR vaccine is applied to these 134,400 children, we would expect 3.5 cases of ITP each year to be attributable to the first dose of MMR. We found an average of 25 cases per year of ITP in children age one year, which means approximately 7% (3.5/ 25) of ITP cases in 1 year old children in Ontario could be due to MMR vaccine, using the risk estimate from the literature.
Discussion
The background rates we have estimated for ITP and KD are consistent with rates obtained through different methods in the same or similar settings, but we were not able to draw this conclusion regarding MS. With the caveat that caution should be used when comparing rates across literature and data sources, methods of analysis, health care systems and populations, other studies report similar incidence of childhood ITP to ours. We were unable to find comparable Ontario or Canadian data on the incidence of ITP in children, but a United Kingdom study using administrative data from 1990 to 2005 reported a rate of ITP of 6.8 per 100,000 person years in children <2 years and 7.2 per 100,000 person years in those aged 2–5 years [27]. A systematic review of ITP incidence among a broader age group of children in several European countries reported the incidence of ITP to be between 1.9–6.4 cases per 100,000 children < 16 years of age [28]. The single year prevalence of ITP (>2 ITP insurance claims separated by at least 30 days) in 2002 in the American state of Maryland was 9.3 per 100,000 children aged 1–5 years [29]. Although the setting, methods, and age groups were different, this finding is similar to the incidence we found (10.5 per 100,000 among children 1–7 years) for this acute condition. The sex distribution of pediatric ITP we observed was also similar in other studies, with the majority of cases being male, especially in the younger age groups [27,29–30].
Other investigators have validated the use of administrative data to determine the incidence of KD by comparing administrative data results to active surveillance mechanisms in Ontario [31]. Further, a review of hospitalization data estimated the annual incidence of KD in Ontario to be between 24–27 cases per 100,000 children < 5 years between 2004 and 2011 [32]. Our finding of 22.8 cases per 100,000 children <5 years is comparable with the slightly lower rate possibly owing to the different time period assessed and our more limited inclusion criteria (only most responsible diagnosis versus primary or secondary diagnosis codes [32]). Our data showed higher rates of KD in males, especially in the younger ages (i.e. less than 5 years of age), which is also consistently found in the literature [18,33–36].
As pediatric MS is rare, and can manifest differently than adult MS, the incidence and prevalence of adolescent MS is not well known. However, the Atlas of Multiple Sclerosis, a collaboration between the Multiple Sclerosis International Federation (MSIF) and the World Health Organization (WHO) estimated the 2013 prevalence of pediatric MS in Canada to be between 0.52–0.56 per 100,000 children [37]. Our calculated incidence rate (average of 0.8 per 100,000 adolescents aged 11–17 years) is higher, but we limited the denominator to a higher-risk age group (adolescents) and did not exclude those who had MS hospitalizations prior to 2005. On the other hand, our result could also be an underestimate, as we did not include outpatient visits. A recent validation of the use of Ontario administrative data in estimating the incidence of MS in the adult population found that optimal sensitivity (84%) and positive predictive value (86%) required one hospitalization or five physician office billings over 2 years [38]. It is unknown whether this algorithm would perform as well for adolescents, but highlights a limitation of our data, which did not include physician billings.
While we are pleased to have demonstrated that background rates can be estimated from intelliHEALTH, a readily available dataset, we are aware of the limitations of our study. We did not undertake a chart review to validate the diagnoses obtained from health care administrative data. Further, individuals who did not seek health care at all and those who did not attend a hospital (i.e. physician office visits only) are not included. As with any uncommon condition, rates can be unstable with small numerators, which would cause our results to be inaccurate. This concern is mitigated by consistent case definitions and monitoring temporal trends rather than rates per se. More broadly, there is a 3–6 month time lag in CIHI data availability through intelliHEALTH. This could limit its use in timely assessment of vaccine safety signals, which usually require a rapid response.
Limitations notwithstanding, the background rates we calculated also serve as a point of comparison against incidence of these conditions in our provincial AEFI surveillance. “Locally-relevant” background rates are an important component of vaccine safety surveillance [2]. Based on the provincial case definition for thrombocytopenia in Ontario’s AEFI surveillance, there was one case of thrombocytopenia in a child in 2015 [14] and there were three in 2016 [38]. None of the cases of thrombocytopenia had received measles-containing vaccine (Public Health Ontario, unpublished data) despite the established link and the number predicted in our calculations above (3.5 cases of ITP expected in children aged one year after MMR vaccine). The lack of MMR-associated ITP cases in recent Ontario AEFI reports [1,14,39], suggests under-reporting.
No case definitions exist for KD or MS in routine Ontario AEFI surveillance [40], but these conditions would be captured if reported as “other severe or unusual events”. Searching the 2015 and 2016 provincial vaccine safety data by key terms, there were five cases of children with KD and no cases of adolescent MS [41].
Calculation of these background rates serves to further strengthen our provincial vaccine safety surveillance system, which has been gaining momentum in recent years with increasing availability and quality of vaccine safety data [1,14,41], and recognition of Public Health Ontario’s work by the World Health Organization [42]. We are well positioned to rapidly assess safety reports related to these three conditions, comparing them to known background rates and to evaluate the sensitivity of the system to be able to capture reportable events. With this study as an example, we and others in Canada could consider the use of intelliHEALTH, or similar sources of administrative data, in future to determine background rates of conditions reported as AEFI.
Supporting information
S1 Table. ICD-10 most responsible diagnosis codes used to identify disease outcomes.
https://doi.org/10.1371/journal.pone.0210833.s001
(DOCX)
References
- 1.
Ontario Agency for Health Protection and Promotion (Public Health Ontario). Annual report on vaccine safety in Ontario, 2014. Toronto, ON: Queen's Printer for Ontario; 2015.
- 2. Black S, Eskola J, Siegrist CA, Halsey N, Macdonald N, Law B, et al. Importance of background rates of disease in assessment of vaccine safety during mass immunisation with pandemic H1N1 influenza vaccines. Lancet. 2009;374: 2115–2122. pmid:19880172
- 3. Wang Y, Wu L, Yu X, Zhao F, Russell A, Song M, et al. The expected number of background disease events during mass immunization in China. PLoS One. 2013;8: e71818. pmid:23977153
- 4. Deeks SL, Lim GH, Simpson MA, Rosella L, Mackie CO, Achonu C, et al. Estimating background rates of Guillain-Barre Syndrome in Ontario in order to respond to safety concerns during pandemic H1N1/09 immunization campaign. BMC Public Health. 2011;11: 329-2458-11-329.
- 5.
Institute of Medicine (US) Vaccine Safety Committee. Adverse Events Associated with Childhood Vaccines: Evidence Bearing on Causality. (Stratton K. R., C. J. Howe, & R. B. Johnston, Eds.) Washington, DC: The National Academies Press (US), 1994. https://doi.org/10.17226/2138
- 6. Black C, Kaye JA, Jick H. MMR vaccine and idiopathic thrombocytopaenic purpura. Br J Clin Pharmacol. 2003;55: 107–111. pmid:12534647
- 7. France EK, Glanz J, Xu S, Hambidge S, Yamasaki K, Black SB, et al. Risk of immune thrombocytopenic purpura after measles-mumps-rubella immunization in children. Pediatrics. 2008;121: e687–92. pmid:18310189
- 8. Rajantie J, Zeller B, Treutiger I, Rosthoj S, NOPHO ITP working group and five national study groups. Vaccination associated thrombocytopenic purpura in children. Vaccine. 2007;25: 1838–1840. pmid:17126957
- 9. Wise RP, Bonhoeffer J, Beeler J, Donato H, Downie P, Matthews D, et al. Thrombocytopenia: case definition and guidelines for collection, analysis, and presentation of immunization safety data. Vaccine. 2007;25: 5717–5724. pmid:17493712
- 10. Mantadakis E, Farmaki E, Buchanan GR. Thrombocytopenic purpura after measles-mumps-rubella vaccination: a systematic review of the literature and guidance for management. J Pediatr. 2010;156: 623–628. pmid:20097358
- 11. Sauve LJ, Bettinger J, Scheifele D, Halperin S, Vaudry W, Law B, et al. Postvaccination thrombocytopenia in Canada. Pediatr Infect Dis J. 2010;29: 559–561. pmid:20160658
- 12.
Ministry of Health and Long-Term Care. Publically Funded Immunization Schedules for Ontario–December 2016. Government of Ontario, Toronto. Available: http://www.health.gov.on.ca/en/pro/programs/immunization/docs/immunization_schedule.pdf. Accessed December 1, 2017
- 13. Hua W, Izurieta HS, Slade B, Belay ED, Haber P, Tiernan R, et al. Kawasaki disease after vaccination: reports to the vaccine adverse event reporting system 1990–2007. Pediatr Infect Dis J. 2009;28: 943–947. pmid:19755926
- 14.
Ontario Agency for Health Protection and Promotion (Public Health Ontario). Annual report on vaccine safety in Ontario, 2015. Toronto, ON: Queen's Printer for Ontario; 2016.
- 15. Abrams JY, Weintraub ES, Baggs JM, McCarthy NL, Schonberger LB, Lee GM, et al. Childhood vaccines and Kawasaki disease, Vaccine Safety Datalink, 1996–2006. Vaccine. 2015;33: 382–387. pmid:25444786
- 16. Esposito S, Bianchini S, Dellepiane RM, Principi N. Vaccines and Kawasaki disease. Expert Rev Vaccines. 2016;15: 417–424. pmid:26634312
- 17. Phuong LK, Bonetto C, Buttery J, Pernus YB, Chandler R, Felicetti P, et al. Kawasaki disease and immunisation: A systematic review. Vaccine. 2017;35: 1770–1779. pmid:28259442
- 18. Lin YT, Manlhiot C, Ching JC, Han RK, Nield LE, Dillenburg R, et al. Repeated systematic surveillance of Kawasaki disease in Ontario from 1995 to 2006. Pediatr Int. 2010;52: 699–706. pmid:20113416
- 19.
Off C, Douglas J. 'We failed.' Toronto Star publisher says the paper's HPV vaccine article let down readers. CBC radio: As It Happens. 11 Feb 2015. Available: http://www.cbc.ca/radio/asithappens/lynching-report-badger-nf-flood-risk-space-museum-rocket-and-more-1.2960346/we-failed-toronto-star-publisher-says-the-paper-s-hpv-vaccine-article-let-down-readers-1.2960366. Accessed 22 Feb 2018.
- 20. Vogel G. Decision by Europe's top court alarms vaccine experts. Science. 27 Jun 2017. Available: http://www.sciencemag.org/news/2017/06/decision-europe-s-top-court-alarms-vaccine-experts. Accessed 22 Feb 2018.
- 21. Monteyne P, Andre FE. Is there a causal link between hepatitis B vaccination and multiple sclerosis? Vaccine. 2000;18: 1994–2001. pmid:10706960
- 22. Meggiolaro A, Migliara G, La Torre G. Association between Human Papilloma Virus (HPV) vaccination and risk of Multiple Sclerosis: A systematic review. Hum Vaccin Immunother. 2018; 15:1–9.
- 23.
The World Health Organization. Global Vaccine Safety: The Global Advisory Committee on Vaccine Safety rejects association between Hepatitis B vaccination and multiple sclerosis (MS). November 2002. Available: http://www.who.int/vaccine_safety/committee/topics/hepatitisb/ms/en/. Accessed December 1, 2017
- 24. Mailand MT, Frederiksen JL. Vaccines and multiple sclerosis: a systematic review. J Neurol. 2017;264(6): 1035–1050 pmid:27604618
- 25.
Statistics Canada. Population estimates and population projections. Ontario Ministry of Health and Long Term Care. IntelliHEALTH ONTARIO. Extracted July 26, 2018
- 26.
Ontario Agency for Health Protection and Promotion (Public Health Ontario). Immunization coverage report for school pupils: 2016–17 School Year. Toronto, ON: Queen's Printer for Ontario; 2018.
- 27. Yong M, Schoonen WM, Li L, Kanas G, Coalson J, Mowat F, et al. Epidemiology of paediatric immune thrombocytopenia in the General Practice Research Database. Br J Haematol. 2010;149: 855–864. pmid:20377590
- 28. Terrell DR, Beebe LA, Vesely SK, Neas BR, Segal JB, George JN. The incidence of immune thrombocytopenic purpura in children and adults: A critical review of published reports. Am J Hematol. 2010;85: 174–180. pmid:20131303
- 29. Segal JB, Powe NR. Prevalence of immune thrombocytopenia: analyses of administrative data. J Thromb Haemost. 2006;4: 2377–2383. pmid:16869934
- 30. Zeller B, Rajantie J, Hedlund-Treutiger I, Tedgard U, Wesenberg F, Jonsson OG, et al. Childhood idiopathic thrombocytopenic purpura in the Nordic countries: epidemiology and predictors of chronic disease. Acta Paediatr. 2005;94: 178–184. pmid:15981751
- 31.
Manlhiot C, O'Shea S, Bernknopf B, Labelle M, Mathew M, Chahal N, et al. Validation of Kawasaki Disease Incidence Assessment as Derived from Health System Administrative Databases vs. Active Retrospective Surveillance in Ontario, Canada. International Kawasaki Disease Symposium, Final Program. 2015.
- 32.
McCrindle B, O'Shea S, Lew B, Masood S, Bock D, Lai L, et al. Epidemiology of Kawasaki Disease in Canada (2004–2011). International Kawasaki Disease Symposium, Final Program. 2015.
- 33. Holman RC, Curns AT, Belay ED, Steiner CA, Schonberger LB. Kawasaki syndrome hospitalizations in the United States, 1997 and 2000. Pediatrics. 2003;112: 495–501. pmid:12949272
- 34. Holman RC, Belay ED, Christensen KY, Folkema AM, Steiner CA, Schonberger LB. Hospitalizations for Kawasaki syndrome among children in the United States, 1997–2007. Pediatr Infect Dis J. 2010;29: 483–488. pmid:20104198
- 35. Makino N, Nakamura Y, Yashiro M, Ae R, Tsuboi S, Aoyama Y, et al. Descriptive epidemiology of Kawasaki disease in Japan, 2011–2012: from the results of the 22nd nationwide survey. J Epidemiol. 2015;25: 239–245. pmid:25716368
- 36. Scuccimarri R, Yeung R, Cabral D, Dahdah N, Human D, Hume H, et al. Kawasaki disease November 2013 to November 2014—Final Report. 2014 Results of the Canadian Paediatric Surveillance Program. 2015.
- 37.
Multiple Sclerosis International Federation. Atlas of Multple Sclerosis. 2013. Available: http://www.msif.org/about-us/advocacy/atlas/. Accessed: December 1, 2017
- 38. Widdifield J, Ivers NM, Young J, Green D, Jaakkimainen L, Butt DA, et al. Development and validation of an administrative data algorithm to estimate the disease burden and epidemiology of multiple sclerosis in Ontario, Canada. Mult Scler. 2015;21: 1045–1054. pmid:25392338
- 39.
Ontario Agency for Health Protection and Promotion (Public Health Ontario). Annual report on vaccine safety in Ontario, 2016. Toronto, ON: Queen's Printer for Ontario; 2017.
- 40.
Ontario Ministry of Health and Long Term Care. Infectious Diseases Protocol 2014—Appendix B Adverse Events Following Immunizatioin (AEFI) Case Definitions. 2015. Available: http://www.health.gov.on.ca/en/pro/programs/publichealth/oph_standards/docs/aefi_cd.pdf. Accessed: December 1, 2017.
- 41.
Ontario Agency for Health Protection and Promotion (Public Health Ontario). Annual report on vaccine safety in Ontario, 2016. Toronto, ON: Queen's Printer for Ontario; 2017.
- 42.
World Health Organization [Internet]. Global Vaccine Safety. Vaccine Safety–Public Health Ontario. Available: https://www.who.int/vaccine_safety/initiative/communication/network/pho/en/. Accessed: September 13, 2018.