http://orcid.org/0000-0001-6617-3631
Figures
Abstract
Background
Typhoid fevers are infections caused by the bacteria Salmonella enterica serovar Typhi (Salmonella Typhi) and Paratyphi A, B and C (Salmonella Paratyphi). Approximately 17.8 million incident cases of typhoid fever occur annually, and incidence is highest in children. The accuracy of current diagnostic tests of typhoid fever is poorly understood. We aimed to determine the comparative accuracy of available tests for the pediatric population.
Methods
We first conducted a systematic literature review to identify studies that compared diagnostic tests for typhoid fever in children (aged ≤15 years) to blood culture results. We applied a Bayesian latent-class extension to a network meta-analysis model. We modelled known diagnostic properties of bone marrow culture and the relationship between bone marrow and blood culture as informative priors in a Bayesian framework. We tested sensitivities for the proportion of negative blood samples that were false as well as bone marrow sensitivity and specificity.
Results
We found 510 comparisons from 196 studies and 57 specific to the pediatric population. IgM-based tests outperformed their IgG-based counterparts for ELISA and Typhidot tests. The lateral flow IgG test performed comparatively well with 92% sensitivity (72% to 98% across scenario analyses) and 94% specificity. The most sensitive test of those investigated for the South Asian pediatric population was the Reverse Passive Hemagglutination Assay with 99% sensitivity (98% - 100% across scenario analyses). Adding a Widal slide test to other typhoid diagnostics did not substantially improve diagnostic performance beyond the single test alone, however, a lateral flow-based IgG rapid test combined with the typhoid/paratyphoid (TPT) assay yielded improvements in sensitivity without substantial declines in specificity and was the best performing combination test in this setting.
Conclusion
In the pediatric population, lateral-flow IgG, TPT and Reverse Passive Hemagglutination tests had high diagnostic accuracy compared to other diagnostics. Combinations of tests may provide a feasible option to increase diagnostic sensitivity. South Asia has the most informed set of data on typhoid diagnostic testing accuracy, and the evidence base in other important regions needs to be expanded.
Author summary
Typhoid fever is an infection caused by the bacterium Salmonella Typhi. Typhoid fever is rare in developed countries but remains high in the developing world. Effective treatment is available but accurate diagnosis of typhoid fever is challenging as typhoid fever can be difficult to distinguish from other infections. Bone marrow culture is the most accurate diagnostic test for typhoid fever however is invasive and not feasible in many settings. New vaccines for typhoid and the need for improved estimates of burden increases the demand for improved understanding of diagnostic accuracy. Comparing the diagnostic accuracy of tests for typhoid fever is challenging as head-to-head studies are few. We applied newly developed methods for comparative evaluation of diagnostic tests for typhoid fever in children using statistical approaches that allowed for the proper incorporation of uncertainty and comparison of tests that had not been compared directly. The lateral-flow IgG, TPT and Reverse Passive Hemagglutination tests all had good diagnostic accuracy compared to other diagnostics. Combinations of tests may provide a feasible option to increase diagnostic sensitivity. Finally, while South Asia has the most informed set of data on typhoid diagnostic testing accuracy, the evidence base in other important regions needs to be expanded.
Citation: Arora P, Thorlund K, Brenner DR, Andrews JR (2019) Comparative accuracy of typhoid diagnostic tools: A Bayesian latent-class network analysis. PLoS Negl Trop Dis 13(5): e0007303. https://doi.org/10.1371/journal.pntd.0007303
Editor: Thomas C. Darton, The University of Sheffield, UNITED KINGDOM
Received: July 27, 2018; Accepted: March 13, 2019; Published: May 8, 2019
Copyright: © 2019 Arora et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: This work was funded by the Bill and Melinda Gates Foundation (contract number 46226, URL: https://www.gatesfoundation.org/). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Typhoid fever (also known as enteric fever) is a systemic infection caused by the Gram-negative bacteria Salmonella enterica serotypes Typhi or Paratyphi A,B and C[1],[2]. While rare in developed countries, the burden of typhoid remains high in developing countries. Recent annual estimates of typhoid fever cases in low- and middle-income countries range from approximately 17.8 million[3] to 26.9 million[4] cases worldwide and most of these are in South Asia. The pediatric population is of particular interest as most cases occur in those between 3 and 19 years of age[1], the highest incidence of typhoid occurs in those less than 5 years of age[5]. Recent modelling work reported a higher incidence among children aged two to four years compared to those less than two years.[3] With the recent World Health Organization pre-qualification of, and GAVI commitments towards, a typhoid conjugate vaccine for use in routine immunization programs, there is a need for better data on typhoid burden in young children, which requires better understanding of diagnostic accuracy. Prior meta-analyses have focused on all age groups without distinguishing performance in children; however, we hypothesize that diagnostic accuracy may differ between children and adults due to a greater degree of prior exposure to Salmonella and other pathogens in adults, leading to serologic cross reactivity. If diagnosed promptly, typhoid can be successfully treated with antibiotics. [1, 2]
Accurate diagnosis of typhoid fever has proved a major challenge. Clinical signs and symptoms are often non-specific, and typhoid can be difficult to distinguish from other acute febrile illnesses, including dengue, malaria, influenza, leptospirosis, and Rickettsial infections[6–8]. The definitive diagnosis for typhoid fever is via isolation of S. Typhi from blood, bone marrow or other sterile sites.[1] The most sensitive and specific diagnostic test for typhoid fever is bone marrow culture; however, as this test is invasive, carries risks of medical complications, and requires technical expertise and specialized equipment, it is not widely performed in endemic settings as a routine diagnostic procedure. Among culture-based methods, blood culture is the most commonly used typhoid diagnostic method, but results are not available for days, and many settings lack the resources required for proper culturing techniques. Furthermore, it has limited sensitivity (40–75% in most settings)[9, 10], which may be further diminished by prior antibiotic use.
The Widal test, developed in the late 19th century to measure antibodies against the O and H antigens of Salmonella, remains perhaps the most widely used typhoid diagnostic in the world. However, the Widal test only has moderate sensitivity and specificity, particularly in endemic settings, and there remains a challenge of determining a proper cut-off point for a positive result[5, 11]. Indeed, rapid and reliable (>90% sensitivity and specificity) diagnostics do not yet exist for invasive salmonellosis. The Reverse Passive Hemagglutination (RPHA) Test, that detects the S. Typhi antigen, was found to have a sensitivity and specificity that is comparable with the Widal test leading to suggestion that it could be used as an alternative to the Widal test in busy microbiology laboratories[12, 13]. Newer diagnostic tests, such as the antibody tests Typhidot and Tubex, have demonstrated moderate accuracy[14]. The typhoid/paratyphoid diagnostic assay (TPT test) has shown promising results.[15] Polymerase chain reaction (PCR) and other molecular, transcriptomic and metabolomic methods have been developed, but they have yet to be evaluated in large scale settings.
Assessing the comparative performance of diagnostic testing is challenging as few head-to-head evaluations exist and previous reviews of diagnostic testing have found a high level of variation in testing methods for typhoid fever globally and a lack of a single applicable gold standard, a challenge that is particularly acute given the low sensitivity of the most common reference standard, blood culture.[3, 9] We aimed to assess the comparative performance of typhoid diagnostics using newly developed methods for comparative evaluations [16]. In particular, we combined a Bayesian network meta-analysis (NMA) procedure with latent class analysis. [16].
Methods
We developed a comprehensive search strategy to identify relevant studies comparing diagnostic tests for typhoid disease. We particularly considered typhoid fever to include Salmonella Typhi and S. Paratyphi A. We searched the following databases: EMBASE, MEDLINE, ISI Web of Science and the Cochrane Central Register of Controlled Trials from inception to December 26, 2016. We also scanned references from systematic reviews on typhoid diagnostic tools identified via the above search. We conducted a grey literature search of Google Scholar and the National Institutes of Health Research Portfolio Online Reporting Tools (NIH RePORT). We searched conference proceedings of the International Conference on Typhoid and Other Invasive Salmonelloses and the American Society of Tropical Medicine and Hygiene Conference, and unpublished data submitted by the originator companies to the US Food and Drug Administration and the European Medicines Agency as part of diagnostic registration applications. Additionally, we performed manual searches of clinicaltrials.gov and the WHO International Clinical Trials Registry Platform to identify studies that have not yet been published but have results and were potentially eligible for inclusion. Specific search terms and results by database are provided in S1 Table. We also engaged key leaders from disparate agencies that conduct research in diagnostic development, including, but not limited to the U.S. Department of Defense (Walter Reed Army Institute of Research and Defense Advanced Research Projects Agency) and non-profit research institutions and diagnostic development organizations.
Data extraction
All abstract and full-text screening of studies was done in duplicate. Data extraction was completed using a standardized data extraction form. The extraction form was designed for this study and pilot tested by the authors. A copy of our extraction form is included in S2 Table. We extracted all comparisons across diagnostic tests as well as within any relevant subgroups presented in the included studies. Study characteristics of interest for extraction included: detailed description of diagnostic tests used including the details of any commercial tests used, types and volume of biological specimen, study location (detailed location, country and coded into World Bank region), broad age group of study population, duration of illness (most often reported as duration of fever), patient reported antibiotic self-treatment/use prior to study entry. For studies where subgroup data were not reported, study authors were contacted for age-specific contingency tables. Data were analyzed at the study level and at the level of individual test comparison (index test versus reference test) with both test result and disease status dichotomized.
Pair-wise meta-analysis or network meta-analysis was only done in a subset of studies. This subset was in populations of children, approximately aged 15 or younger (in some cases, it was clear that most subjects were children, but we could not be certain that teenagers and those over 15 years of age were not included) that used blood culture alone as the diagnostic reference test and were conducted in one of three World Bank regions: South Asia, East Asia & Pacific (EAP) and sub-Saharan Africa. These restrictions were introduced to reduce heterogeneity across studies, make synthesis results more interpretable, and focus on pediatric cases in typhoid endemic regions.
Statistical methods
Pairwise meta-analysis for diagnostic tests.
To generate summary estimates of sensitivity and specificity among a subset of diagnostic test comparisons, we conducted meta-analysis for diagnostic tests using methods proposed by Reitsma et al.[17] Briefly, diagnostic accuracy is generally summarized by two measures (usually sensitivity and specificity or likelihood ratios) and these measures are correlated.[18] Because of the correlated nature of the two measures synthesis of diagnostic testing accuracy estimates requires more involved methods than standard meta-analysis applications. This is true even in our “simple” situation where comparisons from each primary study are summarized as a 2 × 2 table of test results against true disease status, both of which have been dichotomized.[18] We used the bivariate model, developed by Reitsma et al.[17], that accounted for between-study heterogeneity as well as correlation between sensitivity and specificity (further details are provided in S1 Statistical appendix).
Bayesian latent class network meta-analysis of diagnostic tests.
To establish the comparative diagnostic accuracy between tests, diagnostic test network meta-analysis was performed. We built on the models previously proposed by Menten and Lesaffre[16], with some modifications to fit the data structure for typhoid diagnostic testing. The mathematical expressions of the model and the statistical code for the Bayesian diagnostic test network meta-analysis (programmed in OpenBUGS) are provided in S1 Statistical appendix.
Since a key limitation in typhoid diagnostic test research is the absence of a ‘gold reference standard’ across studies (i.e. bone marrow culture), conventional network meta-analysis of diagnostic test accuracy studies cannot provide comparative sensitivity and specificity estimates with respect to ‘the truth’. Rather, the most common reference test is blood culture, which is often assumed to yield in the range of 40–75% sensitivity and 100% specificity. More recent synthesis estimates have placed sensitivity estimates higher at 66% when compared to bone marrow[10]. To obtain comparative estimates of sensitivity and specificity with respect to bone marrow culture, we therefore applied a latent class extension to the conventional network meta-analysis model. The Bayesian latent class model proposed by Menten and Lesaffre[16] require good study population prevalence estimates, which was not available for typhoid disease since all studies only enrolled patient with suspected typhoid fever. Rather, we implemented known diagnostics properties of bone marrow culture and the relationship between bone marrow and blood culture as informative priors to facilitate a novel Bayesian latent class diagnostic test network meta-analysis. Particularly, it is estimated that the sensitivity of blood culture for diagnosis of typhoid is only 50–60%.[19] Thus, resampling these to become positive with a corresponding probability theoretically corresponds to a latent class gold standard. Further, applying highly informative priors on the sensitivity and specificity corresponding to that of bone marrow culture will aid in stabilizing the Bayesian model and posterior distributions converge to global maxima Markov states. Lastly, according to good Bayesian practice, use of informative priors should be subjected to sensitivity analysis, referring to different “scenarios”. We thus tested sensitivities for the proportion of negative blood samples that were false negative (base case 50%, sensitivity range 33.3% to 66.7%), as well as bone marrow sensitivity and specificity (base case 95% sensitive and 99% specific, scenario analysis 85% sensitive and 99% specific).
Because there was substantial heterogeneity in the specific types of serologic and molecular tests used, with very few studies utilizing the same antigen-isotype combinations, diagnostic platforms, or molecular targets, we aggregated diagnostic tests according to class (antibody tests, antigen tests, PCR-based tests) to present summary estimates for these diagnostic classes.
Estimating diagnostic accuracy of combinations of rapid tests.
Since the network analysis simultaneously links the sensitivity and specificity estimates (on the logit scale) to the latent class ‘gold standard’, it is possible to estimate the diagnostic accuracy of a combination of two tests within the MCMC sampling framework from the conditionality of the posterior distributions. In particular, the sensitivity of a combination test that is considered positive if either of the two tests are positive can be represented mathematically as the maximum of the two tests within a sampling scheme of individual patient outcomes. Within the MCMC sampling scheme, this should approximately correspond to sampling of the maximum sensitivity of the two sensitivity nodes for each MCMC iteration. Likewise, the specificity of a combination of tests that is considered negative only if both tests are negative can be represented with the minimum of the two.
Results
From a combined 1,749 records identified, there were 196 studies included for full-extraction (See Fig 1 for flow diagram). From these studies, 57 comparisons between tests from 32 studies were included for the NMA (studies listed in Table 1). Full datasets for study level characteristics and comparison level data are presented in S3 and S4 Tables. A glossary of terms is provided in S5 Table.
SLR descriptive characteristics for studies and comparisons
The summary results of the search are presented in Tables 2 and 3 separated by the full set of studies and the subset of studies included the NMA. The full set of studies includes all 196 identified studies in our search that represented 510 pairwise comparisons between two typhoid diagnostic tests. The subset of 32 studies used in NMA represented 57 comparisons.
Study level characteristics for 196 included studies are presented in S3 Table and summarized in Table 2. Among the full set of studies, the majority were conducted in areas of high typhoid endemicity (68.4%), and 72.4% of studies were conducted in either South or East Asia (World Bank Regions classification). There was a relatively even distribution of patient age mixes between adults and children in the studies. However, many studies did not report age, and among the 62 studies that included both adults and children, no subgroup results were reported by age. Just over half of the studies (60.4%) included less than 200 patients with few studies containing more than 1000 patients. There was a slightly higher proportion of newer (post 2000) studies in the full dataset with the majority of studies in the network analysis set being conducted in 2010 or later. In both the full set of studies and the network, the majority of studies (59.2%) did not provide details on the volume of biological specimen collected for the tests or the duration of symptoms (58.7%). Prior antibiotic use can greatly influence the sensitivity of blood culture; however, 72.3% of studies did not report on this characteristic. For those studies that did provide these data we have presented these in Table 2.
Network of evidence
Pairwise, summary estimates for meta-analysis of testing characteristics are presented in Table 3 and as forest plots in S1–S4 Figs. For our network, the numbers of comparisons across each of the six types of index and reference diagnostic tests categorized by date of publication is presented in Fig 2 and summarized in Table 4. The most common comparisons in the full set of 510 comparisons were index tests using antibody, Widal and molecular diagnostics contrasted to viable bacteria culture tests. While the Widal test is the most widely used diagnostic test for typhoid in endemic regions, the majority of the literature focused on evaluating the performance of other antibody tests. The graphical network of comparisons with the NMA set across all index and reference tests for is presented in a network structure in Fig 3A.
a-d. The Network of Comparisons for each Combination of Diagnostic Test of Typhoid Fever Identified in the Systematic Literature Review in a) all regions, b) East-Asia and Pacific, c) Sub-Saharan Africa and d) South Asia Footnotes: * Covers any 100% specific culture (blood, urine, bone marrow, “mix”. Analytically these will be treated as different tests. ** Covers tests for O, H and Vi antigens; titers ranging from 1:20, 1:40…1:320, 1:640 and “slide Widal”. *** Covers multiple S. Typhi antigens (also has 8 connections to Widal tests) “OMP antibody” and “Vi antibody” refers to either IgG or IgM results combined as most studies either did not report results separately by antibody class or reported them together.
Comparative sensitivity and specificity from Bayesian latent class network meta-analysis
A network of evidence was generated overall (Fig 3A) and for each World Bank Region under study (Fig 3B–3D). The testing characteristics generated from Bayesian analysis are presented in Tables 5–8.
Sensitivity and specificity in pediatric patients compared with a blood culture reference test or theoretical bone marrow culture test.
Sensitivity and specificity in pediatric patients compared with a blood culture reference test or theoretical bone marrow culture test.
Sensitivity and specificity in pediatric patients compared with a blood culture reference test or theoretical bone marrow culture test.
Sensitivity and specificity in pediatric patients compared with a blood culture reference test or theoretical bone marrow culture test.
Across all regions combined (Fig 3A and Table 5), rapid tests had both high sensitivity and specificity estimates. Among rapid tests, the reverse passive hemagluttination antigen test had 99% sensitivity (72% to 100% across scenario analyses) and 92% specificity; Typhidot IgM outperformed Typhidot IgG with 80% sensitivity (70% to 85% in scenario analyses) and 95% specificity; and Typhidot IgM or IgG had 91% sensitivity (86% to 93% in scenario analyses), however with specificity of 86%. ELISA IgM outperformed its IgG counterpart and the TPT test also performed very well with 94% sensitivity (76% to 100% in scenario analysis) and a specificity of 97%. The best Widal test appeared to be a 1:160 titer for the H-antigen slide test, yielding a sensitivity of 79% and a specificity of 98%. Lastly, the most sensitive test of all tests investigated for the pediatric population was the reverse passive hemagluttination antigen test however scenario analyses did yield fairly large model variability.
For EAP (Fig 3B and Table 6), the rapid test lateral flow IgM and PCR had very low sensitivity compared to the latent class bone marrow reference test (13% and 7% respectively). TUBEX TP, O12 was associated with a sensitivity of 79%, which was the highest among the investigated tests, and a specificity of 99%. ELISA IgG was inferior to ELISA IgM. The scenario analyses yielded modest sensitivity with ELISA IgM possibly yielding sensitivity up to 67%.
For Sub-Saharan Africa (Fig 3C and Table 7), ELISA Total Ig appeared superior to the other investigated tests with a sensitivity of 85% (81% to 88% in scenario analyses) and 92% specificity, which was the lowest specificity observed in the network analysis. Both Widal tests had very low sensitivity (<25% across all scenario analyses).
For South Asia (Fig 3D and Table 8), several rapid tests had both high sensitivity and specificity estimates. Among the rapid tests, the lateral-flow immunochromatographic dipstick IgG assay had 92% sensitivity (72% to 98% across scenario analyses) and 94% specificity; Typhidot IgM outperformed Typhidot IgG with 74% sensitivity (65% to 80% in scenario analyses) and 97% specificity; and Typhidot IgM or IgG had 79% sensitivity (76% to 91% in scenario analyses), however with specificity of 90%. ELISA IgM outperformed its IgG counterpart and the TPT test also performed very well with 90% sensitivity (72% to 99% in scenario analysis) and a specificity of 93%. The best Widal test appeared to be a 1:80 titer for the H-antigen slide test, yielding a sensitivity of 76% and a specificity of 99%. Lastly, the most sensitive test of all tests investigated for the South Asian pediatric population was Reverse Passive Hemagglutination with 99% sensitivity and scenario analyses did not yield large model variability.
Sensitivity and specificity of hypothetical combination tests are presented in Table 9 and were estimated for the South Asian population only, since none of the rapid tests in our subset of data were associated with good test performance characteristics in the two other World Bank regions. For acute care pediatric subjects tested in the South Asian setting, adding the ‘best’ Widal test (i.e., H-antigen slide test with cut-off 1:80) to any of the three highest performing rapid tests (reference tests: lateral flow IgG, TPT, and Typhidot IgM or IgG) did not yield marked improvements. Conversely, adding a lateral flow-based IgG rapid test to the TPT approach yielded improvements in sensitivity without substantial declines in specificity and was the best performing test combination.
Sensitivity and specificity in pediatric patients compared with a theoretical bone marrow culture test.
Discussion
The results of this analysis builds the evidence base for typhoid diagnostics and is the first attempt to apply newly developed comparative methods for diagnostics testing accuracy.[16] This review and approach yielded several key insights. First, the body of studies on typhoid diagnostics and within study estimates of diagnostic accuracy were highly heterogeneous, even when restricting to studies with similar populations and study designs. Second, despite this heterogeneity, certain diagnostics consistently outperformed others; in particular, IgM-based ELISA and Typhidot outperformed their IgG-based counterparts, and the IgA-based TPT Test performed well in South Asia. Finally, the analytic methods allowed us to generate estimates for test performance based on combinations of tests. We found that combinations of existing sensitive and specific diagnostics may overcome the accuracy limitations inherent in single diagnostics, achieving what may be sufficient accuracy for use in certain clinical settings. Applying these methods allows us to generate estimates for test performance based on combinations of tests. This analysis has also provided comparative estimates of diagnostic testing accuracy for specific tests and targets across a more homogenous set of studies with similar age ranges, geographies and reference tests. This is an important addition because of the wide variety of test types within a family of targets such as antibody or antigen. Though there is an issue of regional variation in antibody response, the majority of our studies were from typhoid endemic regions likely with similar diagnostic titer cut-offs. This expanded and more detailed evidence base allows for more precise comparative assessments of diagnostic testing accuracy via indirect comparisons or network analysis.
The methods and results of this meta-analysis differ from previous meta-analyses of typhoid diagnostics, including those of Storey et al[9] and Wijedorou et al[51] in several ways. First, previous studies have focused on specific products rather than antigen/antibody combinations and performed single comparisons against a reference standard (a composite reference standard or blood culture), without performing between study comparisons through a network framework. We used latent class analysis to account for imperfect reference standards, which is critical given the low sensitivity of blood culture. Additionally, prior analyses focused on single diagnostics without examining their performance in combination and concluded that accuracy was insufficient. By focusing on diagnostic types and their combinations, and utilizing a network meta-analytic framework, we found that certain combinations of diagnostics exceeded 90% sensitivity and specificity.
Our analysis provides evidence that IgM-based ELISA and Typhidot assays diagnostics outperformed their IgG counterparts. Thriemer et al[14] performed a SLR and meta-analysis of the performance of Tubex TF and Typhidot in typhoid endemic countries and concluded that neither test was exclusively reliable for the diagnosis of the disease. Storey et al.[9] also concluded that no single test has sufficiently good performance but suggested that some existing diagnostics could be useful as part of a composite reference standard.
Our exploration of combination tests found, in the South Asian pediatric setting, combining a lateral flow IgG assay with the IgA-focused TPT test yields a high performing diagnostic combination. Combinations of the widely used Widal test and tests with good performance characteristics in Bayesian latent class analysis (lateral flow IgG or TPT test) did not yield substantial improvements to the individual tests alone.
We found that DNA-based tests, whether nested or not, performed similarly with limited sensitivity but high specificity. DNA diagnostic tests were few in our selected group of studies in children, likely due to the small blood volumes drawn from children and the need for substantial volumes for direct molecular diagnostics. The appeal of molecular diagnostics is that they can be more specific than serologies, more rapid than culture, and potentially less affected by prior antibiotic use. The main limitation is that the organism burden in blood during typhoid fever has been estimated at 0.1–1 CFU/ml[52]. For detection to be possible, a large volume of blood is needed, together with highly efficient DNA extraction, concentration and amplification. As a result, in practice, sensitivity is variable but often modest.
There are strengths and limitations to our analysis. Strengths include the extensive searching and identification of published and unpublished data. A further strength is the application of hierarchical modelling using the latent class analysis as it examines the strength of statistical relationships among variables. The analysis was also strengthened by our efforts to limit between-study heterogeneity through only including studies where: a reference test was included, the patient population consisted of children, and select geographical regions were examined. We assessed the potential for regional differences in diagnostic performance by dividing countries into World Bank regions; while these divisions are imperfect and the epidemiology may vary substantially within regions, there was not substantial variation in results in the NMA dataset, with few countries providing the majority of data. Our results were derived from data among children, who may be less likely to have prior exposure to typhoid and other infections compared with adults. It is possible that serologic cross reactivity to other pathogens may be more common in adults, and diagnostic accuracy may be lower. Therefore, we caution against extrapolating these findings to other age groups.
This study had several limitations. These were predominantly related to lack of studies in populations of interest to us. The majority of studies have been small, with over half of studies having less than 200 patients. In these studies–the risk of bias is high due to lack of statistical power and the higher chance of sampling bias. Furthermore, many of the studies were done using convenience sampling which leads to undefined study populations as whomever presented with index symptoms were included. Our results suggest there is a need for additional large sample studies of new methods/technologies to be confidently judged for their diagnostic accuracy. This echoes the conclusions of previous reviews and meta-analyses despite an enlarged and enhanced evidence base.[9] Further, in studies where a composite reference is used–there is a need for additional standardization of techniques and what constitutes a composite standard. In our attempt to extract specific data reference tests, different combinations of tests were used as the composite standard which complicates comparison across studies.
One of the challenges in summarizing evidence across diagnostic tests, such as serologic tests and molecular tests, is that very few studies used the same diagnostic approaches. The studies evaluating serologies used various combinations of antigens (e.g. Vi, Omp, LPS), antibody isotypes (IgG, IgM, IgA), and assay formats (commercial versus in-house ELISA, immunoblot, lateral flow), while studies evaluating molecular diagnostics used varying gene targets, extraction methods and PCR platforms. We therefore aggregated these diagnostics into “antibody”, “antigen” and “PCR” based tests to facilitate analysis of overall accuracy by general broad method; however, this precluded a more nuanced synthesis of evidence on which specific approaches and targets perform better.
A fundamental challenge with evaluating the accuracy of typhoid diagnostics is the lack of perfect reference standards. Bone marrow culture has the highest sensitivity, but was not used in most studies due to its invasiveness. Blood cultures, widely used due to their near perfect specificity, are only 50–65% sensitive. As a result, studies may inaccurately classify individuals with negative cultures as not having typhoid, which can in turn lead to under-estimates of the specificity of serologic diagnostics. To address this challenge and obtain comparative estimates of sensitivity and specificity with respect to bone marrow culture, we therefore applied a latent class extension to the conventional network meta-analysis model. The Bayesian framework allowed us to implement known diagnostics properties of bone marrow culture and the relationship between bone marrow and blood culture as informative priors to more accurately estimate the performance of various diagnostics.
Serologic tests for S. Typhi pose a particular challenge because, while surface antigens for typhoidal Salmonella are generally conserved, they are also shared with many other Enterobacteriaceae.[53] This means that diagnostic kits aimed at a general mix of S. Typhi antigens frequently suffer from low specificity.[53] Further the titres and specificities of antibodies to the classical typhoidal antigens O, H and Vi, vary a great deal, as demonstrated by studies of typhoidal antibody titres in endemic settings[54]. These issues pose challenges to the development of serologic assays built on these targets.
In conclusion, our analysis found a heterogeneous body of evidence for typhoid diagnostics. There is a high degree of variability in diagnostic testing characteristics across tests and regions even after restricting on patient population age, geographic region and reference test. Nevertheless, there are good combinations of existing tests that may provide opportunities in both for individual diagnosis as well as population-based surveillance. South Asia has the most informed set of data on typhoid diagnostic testing accuracy and the evidence base in other important regions needs to be expanded as the performance of diagnostics could vary by region and specific setting. In South Asia, there is evidence for good test performance of some rapid tests, but the evidence is variable due to limited numbers of studies once the data is stratified down by test type. Further work, particularly in the area of novel antigen detection, enhanced molecular diagnostic techniques, host transcriptional assays, metabolomic profiling and low-cost culture techniques all hold potential to drive real gains in the typhoid diagnostics space. Novel antigens specific for S. Typhi, as proposed by Baker et al[53], remains an exciting area of work given the variability of typhoid presentation. An important challenge would be the development of a panel of specific S. Typhi antigens that identify different stages of infection. These could be generated by testing cohorts of patients with protein microarrays in various specimen types to identify specific patterns of infection. Such studies, if fruitful, could lead to the development of low-cost assays. Novel culture techniques that are efficient and require minimal laboratory infrastructure would allow for improved burden estimation and a more accurate diagnosis, and therefore appropriate treatment.[55] To advance the evaluation of these new diagnostics, standardized clinical specimen biobanks representing multiple countries, populations and age groups should be established to facilitate direct comparison of multiple diagnostics against one another. Such a collaborative effort could help further overcome the limitations of population and diagnostic heterogeneity and imperfect reference standards that have limited diagnostic evaluation thus far, and accelerate the identification of accurate diagnostics for typhoid fever.
Supporting information
S3 Table. Study level characteristics of 196 included studies.
https://doi.org/10.1371/journal.pntd.0007303.s003
(XLSX)
S4 Table. Comparison level data for 510 test comparisons.
https://doi.org/10.1371/journal.pntd.0007303.s004
(XLSX)
S1 Checklist. Preferred reporting items for systematic reviews and meta-analyses (PRISMA) 2009 checklist.
https://doi.org/10.1371/journal.pntd.0007303.s006
(DOCX)
S1 Fig. Forest plot of pair-wise meta-analysis of diagnostic test performance in East-Asia and Pacific.
https://doi.org/10.1371/journal.pntd.0007303.s007
(TIF)
S2 Fig. Forest plots of pair-wise meta-analysis of diagnostic test performance in sub-Saharan Africa.
https://doi.org/10.1371/journal.pntd.0007303.s008
(TIF)
S3 Fig. Forest plot of pair-wise meta-analysis of diagnostic test performance in South Asia part 1.
https://doi.org/10.1371/journal.pntd.0007303.s009
(TIF)
S4 Fig. Forest plot of pair-wise meta-analysis of diagnostic test performance in South Asia part 2.
https://doi.org/10.1371/journal.pntd.0007303.s010
(TIF)
S1 Statistical appendix. Methods summary of pairwise meta-analysis of diagnostic tests, model selection data for the network meta-analysis and OpenBUGS code for network meta-analysis of sub-Saharan Africa data.
https://doi.org/10.1371/journal.pntd.0007303.s011
(DOCX)
Acknowledgments
We thank Yibing Ruan for assistance in generating supplemental pair-wise meta-analysis figures. We thank Justin J. Slater for assistance with programming and the network meta-analysis. We thank Neha Sati for assistance in formatting the manuscript. We thank Edward J. Mills for critical feedback during the study design. We thank Aranka Anema for important discussions at project initiation and in securing funding. The following researchers kindly provided additional data for this analysis: Shanta Dutta, Florian Marks, Firdausi Qadri, Helen Storey and Paul Newton.
References
- 1.
Initiative for Vaccine Research of the Department of Vaccines and Biologicals. Background document: the diagnosis, treatment and prevention of typhoid fever. Geneva, Switzerland: 2003.
- 2. Andrews JR, Ryan ET. Diagnostics for invasive Salmonella infections: Current challenges and future directions. Vaccine. 2015;33:C8–C15. doi: papers3://publication/doi/10.1016/j.vaccine.2015.02.030. pmid:25937611
- 3. Antillón M, Warren JL, Crawford FW, Weinberger DM, Kürüm E, Pak GD, et al. The burden of typhoid fever in low- and middle-income countries: A meta-regression approach. PLoS Negl Trop Dis. 2017;11(2):e0005376. Epub 2017/02/27. pmid:28241011; PubMed Central PMCID: PMC5344533.
- 4. Buckle GC, Walker CL, Black RE. Typhoid fever and paratyphoid fever: Systematic review to estimate global morbidity and mortality for 2010. J Glob Health. 2012;2(1):010401. pmid:23198130; PubMed Central PMCID: PMC3484760.
- 5. Bhutta ZA. Current concepts in the diagnosis and treatment of typhoid fever. BMJ. 2006;333(7558):78–82. doi: papers3://publication/doi/10.1136/bmj.333.7558.78. pmid:16825230
- 6. Ross IN, Abraham T. Predicting enteric fever without bacteriological culture results. Trans R Soc Trop Med Hyg. 1987;81(3):374–7. pmid:3686631.
- 7. Vollaard AM, Ali S, Widjaja S, van Asten H, Visser LG, Surjadi C, et al. Identification of typhoid fever and paratyphoid fever cases at presentation in outpatient clinics in Jakarta, Indonesia. Trans Roy Soc Trop Med Hyg. 2005;99(6):440–50. PubMed PMID: WOS:000229087700006. pmid:15837356
- 8. Hosoglu S, Geyik MF, Akalin S, Ayaz C, Kokoglu OF, Loeb M. A simple validated prediction rule to diagnose typhoid fever in Turkey. Trans Roy Soc Trop Med Hyg. 2006;100(11):1068–74. PubMed PMID: WOS:000241118200011. pmid:16697432
- 9. Storey HL, Huang Y, Crudder C, Golden A, de los Santos T, Hawkins K. A Meta-Analysis of Typhoid Diagnostic Accuracy Studies: A Recommendation to Adopt a Standardized Composite Reference. PLoS ONE. 2015;10(11):e0142364–2. doi: papers3://publication/doi/10.1371/journal.pone.0142364. pmid:26566275
- 10. Mogasale V, Ramani E, Mogasale VV, Park J. What proportion of Salmonella Typhi cases are detected by blood culture? A systematic literature review. Ann Clin Microbiol Antimicrob. 2016;15(1):32. pmid:27188991; PubMed Central PMCID: PMC4869319.
- 11. Andualem G, Abebe T, Kebede N, Gebre-Selassie S, Mihret A, Alemayehu H. A comparative study of Widal test with blood culture in the diagnosis of typhoid fever in febrile patients. BMC research notes. 2014;7:653. Epub 2014/09/19. pmid:25231649; PubMed Central PMCID: PMC4177418.
- 12. Kalhan R, Kaur I, Singh RP, Gupta HC. Rapid diagnosis of typhoid fever. Indian J Pediatr. 1998;65(4):561–4. pmid:10773905.
- 13. Coovadia YM, Singh V, Bhana RH, Moodley N. Comparison of passive haemagglutination test with Widal agglutination test for serological diagnosis of typhoid fever in an endemic area. J Clin Pathol. 1986;39(6):680–3. pmid:2424936; PubMed Central PMCID: PMC499994.
- 14. Thriemer K, Ley B, Menten J, Jacobs J, van den Ende J. A Systematic Review and Meta-Analysis of the Performance of Two Point of Care Typhoid Fever Tests, Tubex TF and Typhidot, in Endemic Countries. PLoS ONE. 2013;8(12). PubMed PMID: WOS:000328735700005. pmid:24358109
- 15. Khanam F, Sheikh A, Sayeed MA, Bhuiyan MS, Choudhury FK, Salma U, et al. Evaluation of a typhoid/paratyphoid diagnostic assay (TPTest) detecting anti-Salmonella IgA in secretions of peripheral blood lymphocytes in patients in Dhaka, Bangladesh. PLoS Negl Trop Dis. 2013;7(7):e2316. pmid:23951368; PubMed Central PMCID: PMC3708850.
- 16. Menten J, Lesaffre E. A general framework for comparative Bayesian meta-analysis of diagnostic studies. BMC Med Res Methodol. 2015;15:70. pmid:26315894; PubMed Central PMCID: PMC4552463.
- 17. Reitsma JB, Glas AS, Rutjes AW, Scholten RJ, Bossuyt PM, Zwinderman AH. Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol. 2005;58(10):982–90. pmid:16168343.
- 18. Harbord RM, Whiting P. metandi: Meta-analysis of diagnostic accuracy using hierarchical logistic regression. The Stata Journal. 2009;9(2):8.
- 19. Crump JA, Luby SP, Mintz ED. The global burden of typhoid fever. Bull World Health Organ. 2004;82(5):346–53. doi: papers3://publication/uuid/C688B399-3A1D-4EA9-BF78-788A14E1C014. pmid:15298225
- 20. Castonguay-Vanier J, Davong V, Bouthasavong L, Sengdetka D, Simmalavong M, Seupsavith A, et al. Evaluation of a simple blood culture amplification and antigen detection method for diagnosis of Salmonella enterica serovar typhi bacteremia. J Clin Microbiol. 2013;51(1):142–8. pmid:23100346; PubMed Central PMCID: PMC3536227.
- 21. Handojo I, Dewi R. The diagnostic value of the ELISA-Ty test for the detection of typhoid fever in children. Southeast Asian J Trop Med Public Health. 2000;31(4):702–7. pmid:11414416.
- 22. Limpitikul W, Henpraserttae N, Saksawad R, Laoprasopwattana K. Typhoid Outbreak in Songkhla, Thailand 2009–2011: Clinical Outcomes, Susceptibility Patterns, and Reliability of Serology Tests. PLoS ONE. 2014;9(11):6. PubMed PMID: WOS:000344402600060. pmid:25375784
- 23. Moore CE, Pan-Ngum W, Wijedoru LPM, Sona S, Nga TVT, Duy PT, et al. Evaluation of the Diagnostic Accuracy of a Typhoid IgM Flow Assay for the Diagnosis of Typhoid Fever in Cambodian Children Using a Bayesian Latent Class Model Assuming an Imperfect Gold Standard. American Journal of Tropical Medicine and Hygiene. 2014;90(1):114–20. PubMed PMID: WOS:000329587200020. pmid:24218407
- 24. Nugraha J, Marpaung FR, Tam FC, Lim PL. Microbiological culture simplified using anti-O12 monoclonal antibody in TUBEX test to detect Salmonella bacteria from blood culture broths of enteric fever patients. PLoS ONE. 2012;7(11):e49586. Epub 2012/11/21. pmid:23166719; PubMed Central PMCID: PMC3500315.
- 25. Alam AS, Rupam FA, Chaiti F. Utility of A Single Widal Test in The Diagnosis of Typhoid Fever. 2012. 2012;35(2):6. Epub 2012-04-16.
- 26. Ambati SR, Nath G, Das BK. Diagnosis of typhoid fever by polymerase chain reaction. Indian J Pediatr. 2007;74(10):909–13. Epub 2007/11/06. pmid:17978448.
- 27. Anusha R, Ganesh R, Lalitha J. Comparison of a rapid commercial test, Enterocheck WB((R)), with automated blood culture for diagnosis of typhoid fever. Ann Trop Paediatr. 2011;31(3):231–4. pmid:21781418.
- 28. Beig FK, Ahmad F, Ekram M, Shukla I. Typhidot M and Diazo test vis-a-vis blood culture and Widal test in the early diagnosis of typhoid fever in children in a resource poor setting. Braz J Infect Dis. 2010;14(6):589–93. pmid:21340299.
- 29. Das S, Rajendran K, Dutta P, Saha TK, Dutta S. Validation of a new serology-based dipstick test for rapid diagnosis of typhoid fever. Diagn Microbiol Infect Dis. 2013;76(1):5–9. pmid:23420012.
- 30. Dutta S, Sur D, Manna B, Sen B, Deb AK, Deen JL, et al. Evaluation of new-generation serologic tests for the diagnosis of typhoid fever: data from a community-based surveillance in Calcutta, India. Diagn Microbiol Infect Dis. 2006;56(4):359–65. pmid:16938421.
- 31. Islam K, Sayeed MA, Hossen E, Khanam F, Charles RC, Andrews J, et al. Comparison of the Performance of the TPTest, Tubex, Typhidot and Widal Immunodiagnostic Assays and Blood Cultures in Detecting Patients with Typhoid Fever in Bangladesh, Including Using a Bayesian Latent Class Modeling Approach. PLoS Neglected Tropical Diseases. 2016;10(4):e0004558–10. doi: papers3://publication/doi/10.1371/journal.pntd.0004558. pmid:27058877
- 32. Khan IH, Abu Sayeed M, Sultana N, Islam K, Amin J, Faruk MO, et al. Development of a Simple, Peripheral-Blood-Based Lateral-Flow Dipstick Assay for Accurate Detection of Patients with Enteric Fever. Clinical and Vaccine Immunology. 2016;23(5):403–9. PubMed PMID: WOS:000377456600003. pmid:26961857
- 33. Khanam F, Sheikh A, Sayeed A, Bhuiyan S, Choudhury FK, Salma U, et al. Evaluation of a Typhoid/Paratyphoid Diagnostic Assay (TPTest) Detecting Anti-Salmonella IgA in Secretions of Peripheral Blood Lymphocytes in Patients in Dhaka, Bangladesh. PLoS Neglected Tropical Diseases. 2013;7(7):9. PubMed PMID: WOS:000322321500030. pmid:23951368
- 34. Kulkarni ML, Rego SJ. Value of single Widal test in the diagnosis of typhoid fever. Indian Pediatr. 1994;31(11):1373–7. pmid:7896336.
- 35. Kumar KS, Suganya M, Sathyamurthi B, Anandan H. Reliability of Typhidot Rapid Immunoglobulin M and Immunoglobulin G in the Diagnosis of Typhoid Fever. International Journal of Scientific Study. 2016;4(2):256–59.
- 36. Narayanappa D, Sripathi R, Jagdishkumar K, Rajani HS. Comparative study of dot enzyme immunoassay (Typhidot-M) and Widal test in the diagnosis of typhoid fever. Indian Pediatr. 2010;47(4):331–3. pmid:19430063.
- 37. Nizami SQ, Bhutta ZA, Siddiqui AA, Lubbad L. Enhanced detection rate of typhoid fever in children in a periurban slum in Karachi, Pakistan using polymerase chain reaction technology. Scand J Clin Lab Invest. 2006;66(5):429–36. pmid:16901852.
- 38. Prakash P, Mishra OP, Singh AK, Gulati AK, Nath G. Evaluation of nested PCR in diagnosis of typhoid fever. J Clin Microbiol. 2005;43(1):431–2. pmid:15635006; PubMed Central PMCID: PMC540097.
- 39. Prakash P, Sen MR, Mishra OP, Gulati AK, Shukla BN, Nath G. Dot enzyme immunoassay (Typhidot) in diagnosis of typhoid fever in children. J Trop Pediatr. 2007;53(3):216–7. Epub 2007/03/28. pmid:17387102.
- 40. Rahman M, Siddique AK, Tam FC, Sharmin S, Rashid H, Iqbal A, et al. Rapid detection of early typhoid fever in endemic community children by the TUBEX O9-antibody test. Diagn Microbiol Infect Dis. 2007;58(3):275–81. pmid:17350203.
- 41. Saha SK, Ruhulamin M, Hanif M, Islam M, Khan WA. Interpretation of the Widal test in the diagnosis of typhoid fever in Bangladeshi children. Ann Trop Paediatr. 1996;16(1):75–8. pmid:8787370.
- 42. Shehabi AA. The value of a single Widal test in the diagnosis of acute typhoid fever. Trop Geogr Med. 1981;33(2):113–6. pmid:7281209
- 43. Sheikh A, Bhuiyan MS, Khanam F, Chowdhury F, Saha A, Ahmed D, et al. Salmonella enterica serovar Typhi-specific immunoglobulin A antibody responses in plasma and antibody in lymphocyte supernatant specimens in Bangladeshi patients with suspected typhoid fever. Clin Vaccine Immunol. 2009;16(11):1587–94. pmid:19741090; PubMed Central PMCID: PMC2772369.
- 44. Srivastava L, Srivastava VK. Serological diagnosis of typhoid fever by enzyme-linked immunosorbent assay (ELISA). Ann Trop Paediatr. 1986;6(3):191–4. pmid:2430509.
- 45. Tennant SM, Toema D, Qamar F, Iqbal N, Boyd MA, Marshall JM, et al. Detection of Typhoidal and Paratyphoidal Salmonella in Blood by Real-time Polymerase Chain Reaction. Clinical Infectious Diseases. 2015;61:S241–S50. PubMed PMID: WOS:000362955100002. pmid:26449938
- 46. Zaka-ur-Rab Z, Abqari S, Shahab T, Islam N, Shukla I. Evaluation of salivary anti-Salmonella typhi lipopolysaccharide IgA ELISA for serodiagnosis of typhoid fever in children. Arch Dis Child. 2012;97(3):236–8. pmid:22215815.
- 47. Al-Emran HM, Hahn A, Baum J, Cruz Espinoza LM, Deerin J, Im J, et al. Diagnosing Salmonella enterica Serovar Typhi Infections by Polymerase Chain Reaction Using EDTA Blood Samples of Febrile Patients from Burkina Faso. Clinical Infectious Diseases. 2016;62:s37–s41. pmid:26933018.
- 48. Cheesbrough JS, Taxman BC, Green SD, Mewa FI, Numbi A. Clinical definition for invasive Salmonella infection in African children. Pediatr Infect Dis J. 1997;16(3):277–83. pmid:9076815.
- 49. Ley B, Mtove G, Thriemer K, Amos B, von Seidlein L, Hendriksen I, et al. Evaluation of the Widal tube agglutination test for the diagnosis of typhoid fever among children admitted to a rural hdospital in Tanzania and a comparison with previous studies. BMC Infect Dis. 2010;10:180. pmid:20565990; PubMed Central PMCID: PMC2898821.
- 50. Ley B, Thriemer K, Ame SM, Mtove GM, von Seidlein L, Amos B, et al. Assessment and comparative analysis of a rapid diagnostic test (Tubex(R)) for the diagnosis of typhoid fever among hospitalized children in rural Tanzania. BMC Infect Dis. 2011;11:147. pmid:21609455; PubMed Central PMCID: PMC3123569.
- 51. Wijedoru L, Mallett S, Parry CM. Rapid diagnostic tests for typhoid and paratyphoid (enteric) fever. Cochrane Database Syst Rev. 2017;5:CD008892. Epub 2017/05/26. pmid:28545155; PubMed Central PMCID: PMC5458098.
- 52. Wain J, Diep TS, Ho VA, Walsh AM, Nguyen TT, Parry CM, et al. Quantitation of bacteria in blood of typhoid fever patients and relationship between counts and clinical features, transmissibility, and antibiotic resistance. J Clin Microbiol. 1998;36(6):1683–7. pmid:9620400; PubMed Central PMCID: PMC104900.
- 53. Baker S, Favorov M, Dougan G. Searching for the elusive typhoid diagnostic. BMC Infect Dis. 2010;10(1):45. doi: papers3://publication/doi/10.1186/1471-2334-10-45.
- 54. Abdullah J, Saffie N, Sjasri FAR, Husin A, Abdul-Rahman Z, Ismail A, et al. Rapid detection of Salmonella Typhi by loop-mediated isothermal amplification (LAMP) method. Brazilian Journal of Microbiology. 2014;45(4):1385–91. PubMed PMID: WOS:000350200100032. pmid:25763045
- 55. Andrews JR, Prajapati KG, Eypper E, Shrestha P, Shakya M, Pathak KR, et al. Evaluation of an Electricity-free, Culture-based Approach for Detecting Typhoidal Salmonella Bacteremia during Enteric Fever in a High Burden, Resource-limited Setting. PLoS Neglected Tropical Diseases. 2013;7(6):e2292–8. doi: papers3://publication/doi/10.1371/journal.pntd.0002292. pmid:23853696