http://orcid.org/0000-0002-3227-4941
Figures
Abstract
The German Infection Protection Act requires notifying certain cases of infectious diseases to local health departments (LHD) in Germany. LHDs transmit notifications meeting case definitions to the national health authority, where the proportion of discarded notifications is not known. The proportion of discarded cases at the level of LHDs can be expressed as the positive predictive value (PPV) of the notification system. The PPV can be used to assess the efficiency of the system. We quantified the proportion of discarded notifications to calculate the PPV of the German notification system at the level of LHDs using electronic notification data from Berlin LHDs from 2012. We also analysed reasons for discarding notifications by reviewing notification forms. Data was available from eight LHDs (67%) receiving 10,113 notifications in 2012. Overall PPV was 89% (minimum-maximum = 77–97% across LHDs) and ranging from 30% (Hepatitis B) to 99% (Rotavirus). Of 166 individual investigation forms 84% were on hepatitis B or C cases, most of them discarded because of previously diagnosed chronic disease. LHDs investigate many notifications that do not lead to public health action and useful surveillance data leading to inefficient use of resources. Adaptation of case definitions or the legal framework concerning notifications may increase the efficiency of the notification system and lead to better use of data from notified cases.
Citation: Blümel B, Diercke M, Sagebiel D, Gilsdorf A (2019) Positive predictive value of the German notification system for infectious diseases: Surveillance data from eight local health departments, Berlin, 2012. PLoS ONE 14(2): e0212908. https://doi.org/10.1371/journal.pone.0212908
Editor: Oliver Gruebner, University of Zurich, SWITZERLAND
Received: June 21, 2018; Accepted: February 12, 2019; Published: February 22, 2019
Copyright: © 2019 Blümel et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: The authors received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Regular evaluation of a surveillance system ensures its efficiency and effectiveness [1]. National and international guidelines for the evaluation of surveillance systems recommend the use of surveillance system attributes such as simplicity, flexibility, data quality, acceptability, sensitivity, positive predictive value (PPV), representativeness, timeliness, and stability to identify gaps and to further develop and improve the system [2–7].
Since 2001 many studies evaluated attributes of the German surveillance system including simplicity, acceptability, data quality, PPV, and timeliness. The studies examined system attributes at different levels of the surveillance system (local, regional, and national) either in a disease-specific or a more general, system-wide way [8–13]. Only one study examined the PPV of the German notification system from the viewpoint of the regional and national levels not including data from the local level [11].
The PPV in the context of a surveillance system is the proportion of all notifications that, after validation, are defined as true cases according to the case definition of the surveillance system [2]. A low PPV means that many notifications are not actual cases (according to the surveillance system case definition), false positive notifications are common. These notifications may still have to be investigated and if the proportion of false positives is very high, public health resources might not be used efficiently [2]. Therefore, the PPV is very important in the context of the limited resources available to the public health system and may be used to describe the efficiency of the system.
Since 2001, surveillance of infectious diseases in Germany is regulated by the Infection Protection Act (Infektionsschutzgesetz) [14]. According to this law, physicians and laboratories are required to notify certain infectious diseases (suspected cases, illness or deaths from these diseases) or the detection of certain pathogens to local health departments (LHDs). The investigation of notifications and the implementation of measures is the responsibility of LHDs. After investigation, LHDs transmit notifications that meet the German case definitions electronically and anonymised to the regional health departments. These then transmit cases further to the national health authority (Robert Koch Institute, RKI) that monitors trends in case numbers and conducts epidemiological analyses. Cases are filtered further by so called reference definitions. Final case numbers are then published by the Robert Koch Institute in weekly and yearly reports and on its website (www.rki.de/survstat). Based on respective case definitions case-based data is transmitted further to the international level (Fig 1).
German case definitions are developed and published by the Robert Koch Institute [15]. The main purpose of case definitions is the systematic validation and standardized assessment of notifications across Germany. Their use contributes to increased comparability of data across states and thus to improved surveillance data quality resulting in more reliable epidemiological analyses and recommendations for public health action. They are only used by health authorities, mainly at LHDs, and do not define criteria for notification or clinical diagnosis. Reference definitions are used to define which cases are considered true confirmed cases. These case numbers are then published in official RKI publications.
As LHDs are not required to transmit notifications that do not meet case definitions, the total number of notifications and thus the proportion of transmitted cases among all notifications (i.e., the PPV of the notification system) are not known at the national level. Consequently, the reasons for discarding notifications and the associated workload are also not known at the national level. In this study we aimed to quantify the PPV of infectious disease notifications in Germany to assess the efficiency of the notification system. We also aimed to identify reasons for discarding notifications and the diseases causing the most difficulties at LHDs in this concern.
Methods
Estimating the PPV
We conducted the study in the city state of Berlin because there the same software is used in all LHDs for the management of infectious disease notifications. Additionally, in Berlin regularly all notifications are entered in the electronic system as the number of notifications is used to estimate the human resources needed for infectious disease surveillance within the LHD. We collected data in 2013 for the year 2012 as this was the most recent data available.
We provided all LHDs in Berlin (n = 12) with detailed instructions explaining how to extract data stratified by disease, the number of notifications received at the LHD for all diseases that are notifiable to LHDs according to IfSG. For hepatitis C, we asked LHDs to extract data also by age group (0–24, 25–44, 45–64 and above 64 years) to evaluate differences of the PPV of a disease with increasing prevalence over age groups [16]. Tuberculosis was not included in our study as the data is collected by one specialised LHD for the whole city state. At RKI, we extracted by disease the number of notifications received at the national level from Berlin in 2012 and among them the number of those that meet the reference definitions.
We defined the proportion of cases that meet the case definition and are transmitted by LHDs to the regional level and RKI among all notifications received at the LHD as the PPV based on the case definition (PPVCD, Fig 2A). Additionally, we calculated the proportion of cases that meet the reference definition among those received at the LHD (PPV based on the reference definition (PPVRD, Fig 2B). PPVs were calculated overall and by disease. For hepatitis C we calculated the PPVCD by age group. For each PPV the highest and lowest values were provided (minimum-maximum (min-max)) across LHDs. Disease-specific PPVs were calculated for diseases with at least 10 notifications and was not calculated for suspected rabies exposure. Diseases with less than 10 notifications and suspected rabies exposure are shown in the category “Other” with PPVCD and PPVRD without minimum or maximum values. The definition of PPV used in this study does not aim to represent information about the certainty of the clinical diagnosis. The PPV shows what proportion of all notifications are considered to be true cases for surveillance purposes and should be interpreted as such [3].
Formulae of the positive predictive value based on the case definition (PPVCD, A) and the reference definition (PPVRD, B). CD: case definition; RD: reference definition; LHD: local health department.
Reasons for discarding notifications
In one LHD we reviewed individual investigation forms of discarded notifications available as paper forms from the archive to determine whether the notifications had been investigated. If they had been investigated, we analysed whether further measures for case management by the LHD were necessary. We considered an investigation having taken place when the patient, the treating physician or the laboratory had been contacted and thus at least one item of the first part of the investigation form (detailed demographic data, data about symptoms and history of the disease, medical history of the patient, laboratory data) was filled out. We considered measures having taken place when the “measures” part of the investigation form had been filled out. Measures included, among others, informing the patient, family members or contact persons about the disease; recommendation for vaccination; contact tracing. We collected only anonymised data and all results are presented as aggregated data.
All analyses were carried out with Microsoft Excel 2010 and STATA version 12.1 (StataCorp, College Station, TX, US).
No ethical approval was needed for this study as only anonymised surveillance data was used for the analyses.
Results
Estimating the PPV
Nine (75%) of the twelve LHDs in Berlin extracted and provided data for the study. One LHD was excluded as it declared that it did not keep all notifications in the electronic system. Hence, data from 8 LHDs (67%) could be included in the analyses. Overall, in 8 LHDs, 10,113 notifications were recorded in 2012. 8,989 notifications were transmitted resulting in an overall PPVCD of 89% (min-max: 76–97% across LHDs). 8,406 cases met the reference definition resulting in an overall PPVRD of 83% (min-max: 73–89% across LHDs) (Fig 3). PPVCD was lowest for hepatitis B (PPVCD = 30%, PPVRD = 10%), hepatitis C (PPVCD = 43%, PPVRD = 43%) and meningococcal meningitis (PPVCD = 55%, PPVRD = 55%) (Table 1). Across diseases, PPVCD and PPVRD was highest for rotavirus gastroenteritis (99%, min-max: 98–100%; 98%, min-max: 97–99% respectively).
LHD: local health department; RHD: regional health department; RKI: Robert Koch Institute.
Diseases that are notifiable by physicians and laboratories (e.g. meningococcal meningitis, measles) had a lower overall PPV compared to those notifiable only by laboratories (PPVCD = 42%, min-max: 28–69% versus PPVCD = 98%, min-max: 92–100%; PPVRD = 35%, min-max: 22–54% versus PPVRD = 92%, min-max: 89–95%).
Hepatitis C data for age group specific PVPCD was available from 7 LHDs. PVPCD was higher in the younger age groups and decreased with age (0–24 years: 61%, min-max: 0–100%; 25–44 years: 50%, min-max: 26–68%; 45–64 years: 40%, min-max: 20–70%; above 64 years: 34%, min-max: 0–60%).
Reasons for discarding notifications
Of 166 investigation forms of discarded notifications available from one LHD, 57% indicated that the LHD had conducted investigations. For 24% of the investigated notifications (14% of all 166 discarded notifications) further measures were necessary. Most discarded notifications were either for hepatitis B (n = 47) or hepatitis C (n = 93). Other diseases were measles, meningococcal meningitis, tick-borne encephalitis, invasive infections with methicillin-resistant Staphylococcus aureus (MRSA), rotavirus gastroenteritis, norovirus gastroenteritis, giardiasis, campylobacteriosis, salmonellosis, and cryptosporidiosis. The most common reason for discarding hepatitis B and C notifications was chronic hepatitis that was either previously diagnosed or notified (the patient or the physician reported the disease to be known or the LHD found information about the person in its database) or for hepatitis B newly diagnosed chronic disease. Other reasons were that the notified person could not be identified or reached, was a foreigner, or did not have the clinical symptoms necessary for the case definition to be met. For diseases other than hepatitis B or C, the most common reasons for discarding the notification were that the person could not be identified or reached, was a foreigner, did not have clinical symptoms necessary for the case definition to be met, that a specimen other than the one required by the case definition was taken, or that a laboratory method other than the one required by the case definition was used (Table 2).
Discussion
PPV
In this study we quantified the proportion of discarded notifications in the German notification system at the level of LHDs using data from the city state of Berlin. Considering all diseases, around 11% of notified cases are discarded by LHDs in Berlin. Extrapolating this percentage on the number of notifications received at the regional and national levels (around 300,000 notifications per year, data source: www.rki.de/survstat) we can assume that German LHDs discard around 33,000 of them. There is a wide range of these proportions between diseases with more than two thirds of hepatitis B and only 1% of rotavirus cases being discarded. Based on data from a single Berlin LHD most cases are discarded because of previously diagnosed chronic disease.
Differences in the nature and notification requirements of the diseases under surveillance may explain the range of PPVs across diseases. As physicians are required to report suspected cases of certain diseases (e.g. meningococcal meningitis, measles), it is not surprising that these diseases have generally a lower PPVCD and PPVRD compared to those that are notifiable only by laboratories. The notification of suspected cases is an important aim of the surveillance system as it makes possible that LHDs can take early action (e.g., contact tracing, prophylactic medication or vaccination in case of measles and meningococcal meningitis). Many of these notifications are not true cases at the end, but as the diseases concerned are usually very serious und require rapid intervention, such use of resources is part of the preventive role of public health surveillance that is also the main aim of the notification law. The low PPV of these diseases demonstrates the sensitivity of the alerting function of the system. However, our data suggest that the highest burden of discarded cases for the LHDs arise not from these diseases, but hepatitis B and C.
Our data shows that the range of PPVs across LHDs is wide. Such a difference may be explained by regional differences within the city of Berlin as the density of hospitals and specialty practices is different among districts. Differences can also arise from the varying interpretation of the case definitions resulting in the transmission of non-cases or non-transmission of true cases. Such varying interpretation of case definitions may be the result of misinterpretation (too complex case definitions or insufficient training in the use of them) [11], and, depending on the available resources at the respective LHD, of insufficient investigation of the case resulting in missing data that is relevant for the case definition. Non-transmission of true cases due to misinterpretation of case definitions and insufficient investigation of cases, consequently, may also result in the underestimation of disease incidence of notifiable diseases.
Discarded notifications
Most of the discarded notifications were hepatitis B and C notifications. For more than two thirds of them the reason for discarding was previously diagnosed chronic disease where some cases had also been notified earlier to the LHD. For 11% of discarded hepatitis B cases the notification was the initial diagnosis of chronic disease.
A direct comparison of our results with that of other countries is difficult as the way of notification and the use of case definitions at the level of LHDs is specific for Germany. However, the high burden of hepatitis B and C notifications that are not true cases from the point of view of the surveillance system, mainly due to non-differentiation between acute and chronic hepatitis and repeated notifications of chronic hepatitis, has been described in other countries [17, 18].
Discarding notified cases with initial diagnosis of chronic hepatitis B may contribute to the underestimation of disease incidence in Germany. In Germany, cases of acute hepatitis B and C (by the physician) and detection of hepatitis B and C virus in acute infection and first-time detection of hepatitis C virus in chronic hepatitis C (by the laboratory) were notifiable at the federal level at the time of this investigation. We can assume that many chronic hepatitis B cases are notified in Germany as some laboratories use automatic, computer-based notification and do not differentiate between acute or chronic disease. These notified and subsequently discarded cases, that are of relevance for estimating disease burden, do not reach higher levels of the public health system as LHDs sort them out using case definitions. The one discarded case of newly diagnosed chronic hepatitis C (Sheet “Discarded_cases_from_1LHD” in S1 File), is most likely the result of misinterpretation of the case definition as all newly diagnosed hepatitis C cases have to be transmitted by the LHD.
The lower PPV of hepatitis C notifications with increasing age could be explained with the increasing prevalence of chronic hepatitis C with age. Young people are more likely to have an acute infection and be tested for the first time in their life compared to older persons who are more likely to have chronic disease.
Limitations
Our study has several limitations. Although for Berlin we could include data from most LHDs (67%), our data is based on only one federal state of Germany representing about 4% of the population of Germany. However, across Germany notification requirements are the same for all physicians and laboratories and the same case definitions are used. For this reason large differences between states and/or LHDs are expected to be uncommon. Differences within Germany may exist in regard to urban versus rural areas and the varying compliance with mandatory notification across the federal states (e.g., if physicians do not report suspected cases). It would be useful to have comparison data from other states as such a comparison would allow an internal check of the reliability and reproducibility of the notification system in general.
It is also possible that some of the LHDs do not enter some of their discarded notifications in the electronic system leading to overestimation of the PPV. As notifications not entered are most likely those that did not require a high amount of resources (e.g., repeated notifications), their effect is less relevant for estimating the burden arising from discarded cases. The PPV calculated in our study does not measure workload quantitatively and, therefore, it cannot be analysed in economic terms. Together with information about the investigations and measures related to discarded cases, it allows to estimate the proportion of LHD resources that are used for discarded cases and assess the efficiency of the notification system. Additionally, amendments to the notification law have been implemented since conducting the study, including, among others, a shortened time limit for the transmission of a notification to the regional level and new notifiable diseases) [19]. These changes may also influence the overall and disease specific PPVs.
Conclusions
LHDs discard many notifications, of which a large number require investigation by the LHD. Differences in the nature and notification requirements of diseases under surveillance may explain the range of PPVs. The low PPV and the high case numbers of hepatitis B and C suggest that a relevant amount of LHD resources are used to investigate notifications that do not lead to public health action and useful surveillance data. On the other hand, our results also suggest that many hepatitis B cases considered relevant for disease burden may have already been notified to LHDs, although not required by the notification law, and thus can be used for estimation of disease incidence and epidemiological analyses. In 2015 an update of the German case and reference definitions has been published that requires that hepatitis B cases without symptoms of acute hepatitis are included in the calculation of diseases incidences (they meet the reference definition) [20]. Additionally, in 2017 the notification requirements were changed so that laboratories have to notify every case of hepatitis B or C they detect regardless of clinical symptoms and course of disease (acute or chronic). These cases are then transmitted further and included in disease incidence calculations. These changes are expected to increase the PPV for hepatitis B and C.
For diseases with low PPV, especially those where many true cases are discarded, further adaption of case definitions and reference definitions may lead to better use of data from already notified cases. Changes in the legal framework concerning notifications may further reduce the burden associated with discarded notifications. These may include notification law changes that require notifiers to provide additional data with the notification that helps differentiate chronic and acute hepatitis B and C cases. Additional instructions and clearer notification forms should also be provided for physicians and laboratories. The implementation of a centralized electronic notification system (in which previously notified cases of chronic diseases can be easily identified) could reduce workload by avoiding duplicate notifications [21].
Acknowledgments
This study would not have been possible without the technical support and participation of the local health departments in Berlin. We are thankful to Gabriele Sinn, Frank Groth and Jörg Bätzing-Feigenbaum for the helpful ideas and comments during the planning of the study. We thank Katharina Alpers for critically reading the manuscript and for her helpful comments.
References
- 1. Krause G. From evaluation to continuous quality assurance of surveillance systems. Euro Surveill. 2006;11(11):pii = 657. pmid:29208136
- 2. Centers for Disease Control and Prevention. Guidelines for evaluating surveillance systems. MMWR Morb Mortal Wkly Rep. 1988;37 Suppl 5:1–18. pmid:3122001
- 3. German RR, Lee LM, Horan JM, Milstein RL, Pertowski CA, Waller MN, et al. Updated guidelines for evaluating public health surveillance systems: recommendations from the Guidelines Working Group. MMWR Recomm Rep. 2001;50(RR-13):1–35. pmid:18634202
- 4.
Health Surveillance Coordinating Committee (HSCC), Population and Public Health Branch, Health Canada. Framework and tools for evaluating health surveillance systems. City: Health Canada; 2004. Available from: http://publications.gc.ca/collections/Collection/H39-4-46-2004E.pdf
- 5.
European Centre for Disease Prevention and Control (ECDC). Data quality monitoring and surveillance system evaluation—A handbook of methods and applications. Stockholm: ECDC; 2014. Available from: http://ecdc.europa.eu/en/publications/Publications/Data-quality-monitoring-surveillance-system-evaluation-Sept-2014.pdf
- 6.
National Public Health Institute (KTL) of Finland. Protocol for the evaluation of EU-wide surveillance networks on communicable diseases. Helsinki: KTL; 2004. Available from: http://ec.europa.eu/health/ph_projects/2002/com_diseases/fp_commdis_2002_frep_18_en.pdf
- 7.
World Health Organization (WHO). Communicable disease surveillance and response systems: guide to monitoring and evaluating. Geneva: WHO; 2006. Available from: http://www.who.int/csr/resources/publications/surveillance/WHO_CDS_EPR_LYO_2006_2.pdf
- 8. Altmann M, Spode A, Altmann D, Wadl M, Benzler J, Eckmanns T, et al. Timeliness of surveillance during outbreak of Shiga Toxin-producing Escherichia coli infection, Germany, 2011. Emerg Infect Dis. 2011 Oct;17(10):1906–9. pmid:22000368
- 9. Hauri AM, Westbrock HJ, Claus H, Geis S, Giernat S, Forssbohm M, et al. Electronic outbreak surveillance in Germany: a first evaluation for nosocomial norovirus outbreaks. PloS ONE. 2011;6(3):e17341. pmid:21408157
- 10. Krause G, Altmann D, Faensen D, Porten K, Benzler J, Pfoch T, et al. SurvNet electronic surveillance system for infectious disease outbreaks, Germany. Emerg Infect Dis. 2007;13(10):1548–55. pmid:18258005
- 11. Krause G, Brodhun B, Altmann D, Claus H, Benzler J. Reliability of case definitions for public health surveillance assessed by Round-Robin test methodology. BMC Public Health. 2006;6:129. pmid:16686946
- 12. Williams CJ, Schenkel K, Eckmanns T, Altmann D, Krause G. FIFA World Cup 2006 in Germany: enhanced surveillance improved timeliness and detection. Epidemiol Infect. 2009;137(4):597–605. pmid:18667109
- 13. Krause G, Altmann D, Claus H, Hellenbrand W, Buchholz U, Hamouda O, et al. Erste Bilanz des neuen Systems zur Überwachung meldepflichtiger Krankheiten nach dem Infektionsschutzgesetz. [First evaluation of the surveillance systems of notifiable diseases under the infectious disease control law in Germany]. German. Gesundheitswesen. 2003;65 Suppl 1:S8–12. pmid:12649788
- 14.
Gesetz zur Verhütung und Bekämpfung von Infektionskrankheiten beim Menschen (Infektionsschutzgesetz—IfSG) vom 20. Juli 2000. [Act on the Prevention and Control of Infectious Diseases in Man (Infection Protection Act) of 20 July 2000]. German. Available from: http://www.gesetze-im-internet.de/ifsg/index.html
- 15.
Robert Koch-Institut. Falldefinitionen des Robert Koch-Instituts zur Übermittlung von Erkrankungs- oder Todesfällen und Nachweisen von Krankheitserregern, Ausgabe 2007. [Case definitions of the Robert Koch Institute for the transmission of cases of illness or death from infectious diseases and of detected pathogens, 2007 edition]. German. ISBN 3-89606-097-X. Berlin: Robert Koch-Institut; 2007.
- 16.
Poethko-Müller C, Zimmermann R, Hamouda O, Faber M, Stark K, Ross RS, Thamm M. Die Seroepidemiologie der Hepatitis A, B und C in Deutschland: Ergebnisse der Studie zur Gesundheit Erwachsener in Deutschland (DEGS1). [Epidemiology of hepatitis A, B, and C among adults in Germany: results of the German Health Interview and Examination Survey for Adults (DEGS1)]. German. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz. 2013;56(5–6):707–15. https://doi.org/10.1007/s00103-013-1673-x
- 17. Cui F, Drobeniuc J, Hadler SC, Hutin YJ, Ma F, Wiersma S, et al. Review of hepatitis B surveillance in China: improving information to frame future directions in prevention and control. Vaccine. 2013;31 Suppl 9:J79–84. pmid:23770336
- 18. Mazurek J, Hutin Y, McNutt LA, Morse DL. Evaluation of hepatitis C surveillance in Poland in 1998. Epidemiol Infect. 2002;129(1):119–25. pmid:12211578
- 19. Schumacher J, Diercke M, Salmon M, Czogiel I, Schumacher D, Claus H, et al. Timeliness in the German surveillance system for infectious diseases: Amendment of the infection protection act in 2013 decreased local reporting time to 1 day. PLoS ONE. 2017;12(10): e0187037. pmid:29088243
- 20. Diercke M, Benzler J, Schöneberg I, Mücke I, Altmann D, Claus H, et al. Falldefinitionen für die Surveillance meldepflichtiger Infektionskrankheiten in Deutschland, Ausgabe 2015. [The 2015 edition of case definitions for the surveillance of notifiable infectious diseases in Germany]. German. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz. 2014;57(9):1107–10. pmid:25096238
- 21. Diercke M. Deutsches Elektronisches Melde- und Informationssystem für den Infektionsschutz (DEMIS). [German Electronic Notification und Information System for Infection Protection (DEMIS)]. German. Epid Bull. 2017;30:291–3.