Search strategy, databases, and search terms

Literature search and analysis was carried out until August 5, 2019. Data were extracted from the following databases: (1) Cochrane Infectious Diseases Group (CIDG) Specialised Register and Cochrane Central Register of Controlled Trials (CENTRAL); (2) Google Scholar; (3) Latin American and Caribbean Health Sciences Literature (LILACS); (4) PubMed; (5) Web of Science; and (6) WHO Library Database (WHOLIS). The reference lists of all included papers were screened for further relevant studies.

The inclusion criteria of articles were (1) primary research published in a peer-reviewed journal; (2) studies addressing any type of existing or developing prediction model; and (3) dealing with chikungunya, dengue, malaria, yellow fever, and Zika diseases or a combination of the diseases; presenting (a) experiences with existing EWS (stand-alone or integrated into the national surveillance systems); and (b) the development or implementation of EWS in a particular setting. EWS studies were excluded if merely investigating trends or correlations of particular alarm indicators. Studies were also excluded if they neither reported nor provided sufficient data to outline the type of mathematical model used or the temporal and spatial prediction of the tool. Further studies were excluded if they failed to demonstrate a developed or prototype of prediction model—i.e., merely presented and discussed candidate list of potential alarm indicators or studies with models constructed to elucidate transmission dynamics. No exclusion due to study design was made, except for conference abstracts, book chapters, or studies that are published by journals of a local institute.

Searches were conducted in English, assuming that most relevant studies are indexed in English, with title and abstract in English. No restrictions were applied for year of publication, geographical area, or language.

The terms “early warning,” “forecasting,” and “prediction” have been probed in a pilot search as Medical Subject Headings (MeSH) terms and as free-text terms. The terms “forecasting” and “prediction” have been discarded thereafter for the final search, as they did not show EWS in a public health sense predicting epidemics, but mostly clinical related studies predicting outcome of clinical disease.

The terms “chikungunya,” “dengue,” “malaria,” “yellow fever,” and “Zika” were used as MeSH terms and as free-text terms, depending on the function of the database. The searches were run for the combination of “early warning” AND the disease.

PubMed has been searched using MeSH terms for the diseases and free-text term for “early warning.” For Google Scholar, LILACS, and WHOLIS, free-text terms for both categories have been use. For CIDG Specialised Register and CENTRAL, the disease and “early warning” were searched as MeSH terms.

Quality assessment and assurance

Searches were performed by the team of authors and double-checked until consistent results were found. One author (LH or EG) screened all hits for their relevance. As for Google Scholar, a large number of hits were derived (>10,000 for dengue), hits were sorted by relevance, and the first 120 hits were established as a suitable number, with no additional relevant hits encountered toward the end of the search.

Two data extractors (LH and SRR for chikungunya, dengue, yellow fever, and Zika; LH and EG for malaria) independently assessed the abstracts and full text of the preselected references for potential eligibility by applying all inclusion and all exclusion criteria.

After full agreement, data were entered into a predefined Excel data extraction form by author, title, journal, publication date, and study design as well as different outcome parameters as they emerged from the studies, following the Consolidated Standards of Reporting Trials (CONSORT) 2010 checklist [19]. Extracted parameters are listed below in detail.

Risk of bias of the included studies has not been assessed, as no systematic but a scoping review was performed, but the quality was assured by strict application of inclusion and exclusion criteria, considering several quality aspects as described above.

Data extraction and analysis

The above described data extraction form was developed based on the study objectives and relevant predefined quantitative and qualitative outcome variables. Further categories were also considered including (1) studies addressing surveillance as structured form of information on disease cases, meteorological, epidemiological, and entomological information (indicator-based surveillance, IBS); and 2) studies on surveillance of unofficial or unstructured information (such as social media and community reports) or of social events (event-based surveillance, EBS). EWS using epidemiological, entomological, or climate data for outbreak prediction were termed “alarm-informed EWS.” EWS based on increased case numbers in comparison with historical data were termed “case-informed EWS.” Further characteristics of the studies taken into consideration were (a) methodological (case study, time series, and randomized control trials (RCT), etc.); (b) the models/tools that have been investigated (and their study scope); and (c) the disease(s) being investigated. Detailed information was extracted on specific outcomes as sensitivity, specificity, PPV/NPV outputs of the models, local coverage of the tool (central, district/province, or subdistrict level), integration (into national surveillance program), and the level of implementation (central, district, or subdistrict levels). Furthermore, the end user of the tool (e.g., public health officers, institutes, researchers, and others), the user-friendliness of the EWS tool, its temporal and spatial risk prediction and its integration feasibility including, resource needed to implement, prediction duration or timelag and, the duration of dataset were all captured in this review. The data extraction form has been further synthesized to evidence tables (Tables 1–3).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. ID, study design, model, and statistics used.

https://doi.org/10.1371/journal.pntd.0009686.t001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Characteristics of EWARS systems under investigation.

https://doi.org/10.1371/journal.pntd.0009686.t002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Main findings, conclusions, and limitations of the studies.

https://doi.org/10.1371/journal.pntd.0009686.t003

Evaluation criteria

Prediction models are typically evaluated according to their statistical and operational performance. Statistically, the sensitivity, specificity, PPV, or NPV measures are used to quantify the quality of prediction performance of the model. Based on common recommendations of appropriate cutoff for model performance, we attributed a statistical measure of >50% to high performance prediction models [20]. The spatial prediction is crucial in the context of vector control and action plan; spatial prediction alone is less useful and must be combined with the temporal model for effective public health response. Accordingly, we defined high temporal prediction by a prediction window of 1 to 12 weeks (allowing sufficient time between prediction and action taken). We defined high spatial prediction by the prediction of hotspots within geographically defined areas of relatively small spatial scale such as villages, neighborhood, or household levels in contrast to a resolution within district scale.

Study characteristics

A total of 46,477 hits were obtained, with 11,710 studies focused on dengue, 2,757 on Zika, 2,706 on chikungunya, 24,611 on malaria, and 4,693 on yellow fever. After review by title and abstract as well as the application of a cutoff after 120 Google Scholar hits, sorted by relevance, 279 dengue, 22 chikungunya, 39 Zika, 89 malaria, and 13 yellow fever articles were further assessed. Based on the inclusion criteria and removing of duplicates, 151 articles were assessed as full texts, but exclusion criteria finally retained 37 articles including 28 for dengue, 2 for Zika, and 7 for malaria. Fig 1 summarizes the search process.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Screening and selection of articles for a scoping review on EWS.

CDSR, Cochrane Database of Systematic Reviews; CENTRAL, Cochrane Central Register of Controlled Trials; CIDG, Cochrane Infectious Diseases Group; EWS, early warning system; LILACS, Latin American and Caribbean Health Sciences Literature.

https://doi.org/10.1371/journal.pntd.0009686.g001

All of the 37 studies were published between 2001 and 2019, of which 33 between 2011 and 2018. Of the 28 dengue studies, 3 [12,21,22] were conducted across different countries (multicounty) and 16 in Asia (5 in China, 4 in Indonesia, 3 in Singapore, and 1 each in Cambodia, Sri Lanka, Taiwan, and Thailand). Another 8 studies were conducted in the Americas (4 in Brazil, 2 in Colombia, 1 in Cuba, and 1 in French Guiana) and 1 [23] in Europe. The 2 Zika studies were performed in the Americas [24,25].

From the 7 malaria studies that were retrieved, 1 was conducted in Colombia and 6 in Africa (2 in Ethiopia and Kenya, 1 in Tanzania, and 1 in Uganda) and 1 in the Solomon Islands in the Pacific. For details of the papers, see Fig 2 and S1 Text (country distribution of included studies).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Geographic distribution of included studies.

https://doi.org/10.1371/journal.pntd.0009686.g002

Of the 28 studies addressing EWS for dengue outbreaks, only 4 studies presented the user perspective on implementing EWS within national programs [12,21,26,27], and 24 studies showed EWS under development (i.e., model exercise); see Table 1. Only 2 dengue studies used a prospective study design [28,29], and one applied both prospective and retrospective approaches [30]; the other 23 were retrospective analyses of surveillance data (1 of those specified as cohort study [21], 1 with a cross-sectional design [31], and 1 as a time series analysis [30]), and 2 studies were exclusively relying on mathematical simulation models [32,33]. The 2 Zika studies reported on exercises of testing EWS [24,25]. The 7 EWS malaria studies included only one existing EWS [34], while 6 other studies reported on potential EWSs or under development. The studies used a large variety of statistical methods as summarized in Table 1.

EWS features per disease

Outbreak indicators of EWSs varied considerably across the selected studies, and, hence, we stratified the findings of the relevant outcomes by the 3 diseases (no studies were retrieved for chikungunya and yellow fewer) that were ultimately retained from the search. In line with the respective case definition used in the diverse reporting systems across national surveillance programs and between diseases, the outbreak indicators were grouped as hospitalized, laboratory-confirmed (including “reported,” “confirmed,” or “imported from neighboring districts”), and suspected cases (including “probable” cases as they were often not distinguished from “suspected”) as well as case numbers resulting from mathematical modeling (no real cases but estimates based on other covariates). The alarm indicators were categorized as entomological (including an index of mosquito survival), meteorological (including the El Niño Southern Oscillation [ENSO] index and others), epidemiological (including sociodemographic information), and social media (such as “Google Trends”).

Dengue

Zika

Malaria

Dengue

All 28 identified dengue studies maintained an IBS approach, and these demanded routine and timely access to surveillance information potentially impacting the effectiveness of early warning tools. Published studies show that consistent and frequent case reporting has stronger predictive capacity, which usually tends to be delayed by monthly reporting schedules [54]. Our results show that weekly reporting of surveillance data correlate with increased sensitivity and PPV; however, only one-third of all dengue studies presented weekly data, negatively impacting the effectiveness and potentially cost-effectiveness of the early warning process [12,21,26–28,31,41,42,44].

A total of 27 dengue studies demonstrated outbreak forecasting, using multiple (9/27) and single alarm (18/27) indicators mostly meteorological ones. However, access to meteorological information on a regular time basis is challenging in several settings [55]. Three studies included regional or international data resources, which, however, demand highly skilled users and advanced digital systems for international data processing [23,33,56].

Furthermore, the mathematical model was identified to affect the predictive ability of EWS due to (1) the choice of the analytical approach employed; and (2) the range of lag times between independent variables and epidemic dengue transmission as demonstrated by Racloz and colleagues and others [52]. Among the retrieved dengue studies, almost all reported estimates on sensitivity, specificity, NPV, or PPV but their prediction performances varied substantially depending on the statistical model used and the data quality. Out of the 19 studies with reported high temporal prediction quality (Table 3), the Bayesian algorithms and generalized linear models were the most prevalent [29,31,36,39,45,57,58]. Additionally, 4 studies used LASSO [22,26,27,45], 3 with ARIMA and time series analysis [28,40,56] independently, and 2 used the Shewhart/endemic channel method [12,21]. Only 4 studies showed evidence of adaptation to less skilled users, which can be significant for public health use [12,21,26,27].

The interaction between changing climate and increasing human mobility as drivers for emerging diseases warrants novel frameworks for assessing the linkage between disease transmission, climate change, and public health intervention in order to reach effective EWS. The use of data mining techniques, such as social media or travel information, in combination with surveillance data has emerged as an alternative source of real-time high-resolution geospatial data on a large scale [59]. However, our review showed minimal evidence of studies exercising such potential alarm indicators, which limits their contribution to outbreak preparedness and response planning [59,60]. The LASSO model is one typical example of such a tool that can potentially contribute to this concept. However, LASSO or similar model concepts demand high-quality big data. As these resources are (a) typically scarce in many countries; and (b) having a tendency to complicate the interpretation of the prediction outputs [26,45], the application in data-constrained settings and by unskilled users is likely to be limited. While limited data accessibility and poor quality have been described by several studies [32,35,41,42,56], published reports highlight the benefits of combining temporal data for analysis of the temporal kinetics and spatial data for the identification of high risk areas [61]. Only 3 retrieved studies [27,28,38] demonstrated high spatial prediction abilities, all attributed to settings with advanced data and surveillance systems. Two of those showed high temporal and spatial prediction, allowing for identification of population at risk at smaller spatial units, which can significantly contribute to targeted vector control [27,38].

Zika

Since the Zika emergence in 2015, the number of PubMed references for ZIKV has risen from 181 to 516 in 2019, with a high proportion focusing on the consequences of ZIKV infection during pregnancy [62]. Only 2 studies have been identified in this review, with a focus on EWS for Zika outbreaks in the Americas. One study had explored the use of Google Trends data as a predictor [24], and the other used a set of meteorological information for generating outbreak predictions [25]. The use of “suspected cases” as outcome variables in both retrieved studies could possibly be explained by the complexity of the Zika diagnosis and the large number of mild cases [62]. For dengue, the prediction accuracy of EWS was superior when using hospitalized or laboratory-confirmed cases compared to suspected cases [12,21]. However, no measurements of sensitivity or PPV were used in both Zika studies, albeit both ARIMA and generalized linear models predicted up to 20 weeks ahead of the outbreak. Limitations of zika outbreak predictions are not discussed in the respective papers, but are likely to be similar as those shown for dengue predictions.

Malaria

Since the inception of Roll Back Malaria (RBM) in 1998, it was clear that the early detection, containment, and prevention of malaria epidemics were key elements of the Global Malaria Control Strategy and Malaria Early Warning Systems (MEWS) [63,64]. Some countries have developed epidemic risk monitoring using simple transmission risk indicators such as excess rainfall [65], but only Kenya has published the development and implementation of MEWS [34]. Since then, several studies and initiatives for EWS have been developed; however, the literature to address the tools’ scope of its predictability, implementability, and users’ perspectives are significantly scarce. The 7 studies identified in this review used different mathematical models and different combinations of indicators. Most of the mathematical models found in this review applied time series including additive and multiplicative models [34,46–48], but lacked vulnerability indicators such as low immunity or drug resistance, which might be prevalent in these study settings. Meteorological indicators ranged from common indicators like rainfall and humidity to land surface temperature or vegetation indices obtained through more sophisticated satellite systems that are usually provided by projects or partnerships, of which their sustainability was never assessed. Like the case with dengue, this wide range of indicators is augmented by additional data sources sought from regional or international entities. The development of implementable and user-friendly malaria EWSs, as shown in the reviews by Githeko and colleagues and Merkord and colleagues [34,46], is a key factor for better disease preparedness and timely response activities.

Public health implications for EWS applications

The EWS tool is primarily aimed at supporting district health managers and national health planners to mitigate or prevent disease outbreaks, ideally using tools that are integrated in the national surveillance programs [66,67]. To further ensure effective functions, EWS should conceptually be perceived as an information system designed to support the decision-making of national- and local-level institutions but also enable vulnerable groups in the society to take actions to mitigate the impacts of an impending risk. As apparent from this review, users of current EWSs were mostly from the central (MOH) levels, with only few tools facilitating district-level applications. The integration of EWS into existing national surveillance program was marginal with only 5 studies [12,21,26,27,34], out of 37, demonstrating their experience of integration (Table 1). Furthermore, the majority of studies have not assessed the feasibility of implementing EWS into national programs, and a few studies have declared the need for high-skilled users and resources—2 limiting factors that are unlikely to exist at small spatial levels in settings where disease outbreaks are public health burdens. Nevertheless, there is an observed trend toward applications of more advanced statistical models with higher predictive abilities that can further advance the prediction and control of disease outbreaks. Generally, early warning and response system that are capable of demonstrating evidence of prospective predictive ability and allows technical and practical adaptations of local public health responses while augmenting communications channels between users at central and district levels are tools that are more likely to be implemented into national surveillance programs. Advancing into frameworks that can facilitate at a low-cost IT maintenance and adapted to unskilled users are features of tools that are plausibly integrable into existing national systems.

As shown in the method section, the IBS and EBS are 2 main channels of information for a functioning EWS. Almost all studies reviewed in this paper maintained an IBS type of application to support EWS, with the exception of 1 Zika- and 1 malaria-related study using the EBS approach [24,46]. The combined use of both could potentially include other sources of information, such as sources from outside the health sector, which is the prime concept of the EBS. With the majority being of IBS-based EWS, the applications of the forecasting tools tend to be less efficient and deviate from the epidemic intelligence concept—the systematic collection, analysis, and communication of any information to detect, verify, assess, and investigate events and health risks with an early warning objective, as opposed to monitoring of disease trends or burdens [68]—which ideally combines both IBS and EBS for more robust outbreak detection.

Meteorological indicators are key predictors, but they are often inaccessible on a timely basis for health services managing the EWS. Benefiting from multiple assessments of users’ perspectives while defining the tool end users, countries like Mexico and Brazil, for instance, have managed to recognize the essence of the availability of local meteorological stations and have therefore organized an improved access to meteorological information [21,29,36,58,69].

Our literature review has identified very limited information for chikungunya and yellow fever outbreaks prediction, none of them fulfilling the inclusion criteria. Studies of EWS for yellow fever outbreaks are limited probably due to the existence of vaccines for this disease. However, chikungunya has now expanded from Africa, Latin America, and Asia to the European region [70,71]. It is quite worrying that studies on EWS tools for diseases like chikungunya are scarce. Although many EWS rely on meteorological information, only 3 studies [28,29,47] performed a prospective type analysis of early warning performance, with no rigorous study could be found.

Limitations of our study

The authors decided to include EWS on malaria to broaden the scope and eventually learn mutually from applications of different fields of disease control (Aedes borne versus Anopheles borne), but also by including yellow fever (vaccine preventable) and other vector-borne disease (mainly relying on vector control interventions). By virtue of their variability in terminology and definitions as well as difficulties in synthesis, the format of a scoping review was applied as a form of review design. Nevertheless, this scoping review follows the PRISMA criteria for conducting a systematic review without performing rigorous critical appraisal of included studies due to time constraints. However, we think that by including only peer-reviewed papers focusing on implemented EWSs or those under development provides a certain guarantee for high-quality papers, which will be an added value to the existing literature. The search strategy was limited to the target diseases not including search terms as “arboviruses” or “febrile illnesses” or “priority diseases” or others, which might be potentially relevant to the review, considering the broadness of the scoping review, Furthermore, stakeholders have been occasionally involved during the 14-month duration of this scoping process—during an Andean regional meeting in Bogota and in Colombia with an expert panel in charge of EWSs. Another limitation might be that we may have missed a few relevant studies; however, by including the Google Scholar search engine, we could overcome this issue, assuming that relevant papers could be identified, which were not indexed in other databases like PubMed. Due to the language barrier, we were unable to include publications in Chinese language, but several English papers from China were included, which could adequately represent the rich experience from the Chinese context.

Conclusions

This scoping review demonstrated gaps and challenges related to the structural, statistical, and operational designs of EWS, and these varied per disease and their corresponding settings. The country surveillance system is an integral part in the overall early warning process where the lack of accessibility to timely and quality data is crucial for establishing a reasonable EWS. Nevertheless, a substantial number of studies (except for dengue) failed to demonstrate any predictive power mainly that predictions based on complicated statistical models are difficult to carry out in low- and middle-income countries. This review has furthermore revealed a significant gap in effectively evaluating the role of EWS in the disease outbreak prediction and control given that the majority of EWS assessment studies have primarily been of retrospective designs. The lack of tool assessments regarding the implementation into existing routine surveillance as well as the feasibility of translating model outputs into local vector control and action plans will unlikely support the global health agenda for controlling disease outbreaks. Likewise, the missing user perspectives in the retrieved studies signals shows that most of the EWSs remain in the academic environment, and little effort has been spent on testing their effectiveness or cost-effectiveness in reducing disease outbreaks. Collectively, findings from this review claim the need for more pragmatic and context-adapted EWS tools, which address the user perspectives and its effectiveness in predicting outbreaks in local settings and trigger response activities.