Figures
Abstract
Background
Pediatric hospital mortality from infectious diseases in resource constrained countries remains unacceptably high. Improved methods of risk-stratification can assist in referral decision making and resource allocation. The purpose of this study was to create prediction models for in-hospital mortality among children admitted with suspected infectious diseases.
Methods
This two-site prospective observational study enrolled children between 6 months and 5 years admitted with a proven or suspected infection. Baseline clinical and laboratory variables were collected on enrolled children. The primary outcome was death during admission. Stepwise logistic regression minimizing Akaike’s information criterion was used to identify the most promising multivariate models. The final model was chosen based on parsimony.
Results
1307 children were enrolled consecutively, and 65 (5%) of whom died during their admission. Malaria, pneumonia and gastroenteritis were diagnosed in 50%, 31% and 8% of children, respectively. The primary model included an abnormal Blantyre coma scale, HIV and weight-for-age z-score. This model had an area under the curve (AUC) of 0.85 (95% CI, 0.80–0.89) with a sensitivity and specificity of 83% and 76%, respectively. The positive and negative predictive values were 15% and 99%, respectively. Two alternate models with similar performance characteristics were developed withholding HIV and weight-for-age z-score, for use when these variables are not available.
Conclusions
Risk stratification of children admitted with infectious diseases can be calculated based on several easily measured variables. Risk stratification at admission can be used for allocation of scarce human and physical resources and to guide referral among children admitted to lower level health facilities.
Citation: Lowlaavar N, Larson CP, Kumbakumba E, Zhou G, Ansermino JM, Singer J, et al. (2016) Pediatric in-Hospital Death from Infectious Disease in Uganda: Derivation of Clinical Prediction Models. PLoS ONE 11(3): e0150683. https://doi.org/10.1371/journal.pone.0150683
Editor: Delmiro Fernandez-Reyes, University College London, UNITED KINGDOM
Received: October 25, 2015; Accepted: February 18, 2016; Published: March 10, 2016
Copyright: © 2016 Lowlaavar et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data can be accessed on Figshare: https://dx.doi.org/10.6084/m9.figshare.2640835.
Funding: The authors received no specific funding for this work. Matthew Wiens was supported by fellowships from the Canadian Institutes of Health Research, the Michael Smith Foundation for Health Research and the Canadian Child Health Clinician Scientist Program.
Competing interests: The authors have declared that no competing interests exist.
Introduction
The fourth United Nations Millennium Development Goal (MDG4) aims to reduce global under five mortality by two thirds from the levels seen in 1990 before the end of 2015.[1] Despite substantial worldwide progress and significant gains in some regions, this achievement will not be reached in most countries of sub Saharan Africa.[2] Currently, over six million children under five years of age continue to die annually, most from infectious diseases.[3,4]
Severe infections including pneumonia, malaria and diarrhea remain the most common cause of death in children worldwide. Pneumonia alone kills more than 1.2 million children annually in resource constrained countries.[4,5] To help address this burden, the World Health Organization introduced the Integrated Management of Childhood Illness (IMCI) guidelines offering hospitals and health centers a syndrome based approach to identify the major causes of death in children under five.[6] Whereas the IMCI guidelines provide an approach to the identification and management of common pediatric diseases, they do not provide any risk scores for the identification of children at high risk of mortality. While the IMCI guidelines do provide a list of danger signs for each condition, facilitating an approach for referral decisions, these were not developed for risk scoring per se.
Several risk scoring models have been developed for use in developed countries to determine in-hospital mortality in pediatric patients. The Pediatric Risk of Mortality (PRISM), Pediatric Index of Mortality (PIM) and (PIM2), are models that predict mortality in children admitted to a pediatric intensive care units.[7–9] These models calculate risk based on several physiologic characteristics and can help identify children requiring more focused care. In resource constrained countries efficient resource utilization is especially important and models that can effectively predict risk could be used to facilitate such prudent resource utilization. This may be especially important for children admitted to lower-level health facilities where the capacity to treat critically ill children is absent or limited. In this context, risk scores could play a crucial role in guiding referral to higher level health centers and hospitals which may be better equipped to care for these children. Models such as PRISM and PIM were derived in developed countries and do not take into consideration major differences seen in children from resource constrained countries, in particular the high prevalence of HIV and malnutrition. Health system limitations for diagnosis, treatment and referral, and social factors such as the prevalence of poor health seeking behavior are also substantially different. Further, the criteria used in the application of these models (base excess, FiO2, time to ventilation etc.) are often not available or relevant in a resource constrained context where mechanical ventilation and modern critical care techniques are the exception. New models developed in and for a resource constrained environment are urgently needed. Where these models have been evaluated in resource constrained countries, it has been done in environments atypical to what is generally available in this context. Recently, a risk scoring algorithm for prediction of in-hospital mortality in subjects with respiratory disease has been developed for use in more a typical East African resource constrained context.[10] However, this scoring tool is limited to those admitted with respiratory disease and would need to be combined with other predictive tools to be used to evaluate risk in any child with a suspected infectious illness.
The objective of this research is to develop a prediction model of in-hospital death among children admitted with proven or suspected infectious diseases of any etiology.
Materials and Methods
Population
This two-site study was conducted at the Mbarara Regional Referral Hospital (MRRH) and the Holy Innocents Children’s Hospital (HICH), both in Mbarara, Uganda. MRRH is a public hospital funded by the Uganda Ministry of Health and is associated with the Mbarara University of Science and Technology Faculty of Medicine. The pediatrics “Toto” ward admits approximately 5000 patients per year. HICH is a Catholic children’s hospital offering subsidized fee-for-service outpatient and in-patient care in Mbarara and admits approximately 2500 patients annually.
This was a prospective observational study conducted between March 2012 and December 2013 was approved by the institutional review boards at the University of British Columbia (Vancouver, Canada) and the Mbarara University of Science and Technology (Mbarara, Uganda).
Eligibility
Children aged 6–60 months who were admitted for treatment of a proven or suspected infectious illness were eligible for enrollment. Subjects previously enrolled were excluded (i.e. patients could not be re-enrolled during a subsequent admission). Enrollment was continuous and all children meeting inclusion criteria who were admitted during study working hours or within 8 hours of a study shift were considered eligible. Parents or the legal guardians of eligible children were required to provide written informed consent prior to enrollment.
Study Procedures
Study enrollment occurred at the time of patient admission. Following enrollment, a research nurse obtained and recorded clinical signs including a 1 minute respiratory rate, blood pressure (automated), axillary temperature, Blantyre coma scale, and using a Phone Oximeter[11], 1 min photoplethysmogram (PPG), blood oxygen saturation (SpO2) and heart rate. Anthropometric data including height, weight, mid-upper arm circumference (MUAC) was also measured and recorded. Anthropometric data collected at enrollment were converted to weight for age, weight for height and height for age z-scores according to the World Health Organization Child Growth Standards.[12] The age corrected heart rate and respiratory rate z-scores were obtained by standardizing the raw measurements using the median and standard deviation values provided by Fleming et al.[13] The age corrected z-scores for systolic blood pressure were calculated using subjects’ height, according to the procedures previously described.[14] To incorporate the clinical belief that both excessively high or low temperature reflect deteriorated health conditions, a transformed temperature was used, which was calculated as 17×log10(37.5- temperature) when temperature was less than 37 and as 1.95* temperature -71.3 otherwise. A physiological transformation based on the shape of the relationship between oxygen saturation and virtual shunt [70×log10(104-SpO2)-57] was used.[15] This virtual shunt was used as an index of disease severity.
A blood sample was taken for measurement of hemoglobin, HIV and a malaria blood smear (microscopy). HIV status was determined using rapid diagnostic test serial algorithm. All positive tests were confirmed by a separate test. Children under 12 months of age with a positive test were confirmed using PCR. Hemoglobin was measured on a Beckman Coulter Ac.T 5diff Cap Pierce Hematology analyzer.
An interview was conducted with the subject’s parent/guardian and information about previous admissions, distance from health facility, transportation costs, bed-net use, maternal education, maternal age, maternal HIV status, history of sibling deaths and drinking water safety were elicited. Subjects received routine care during their hospital stay and were discharged at the discretion of the treating medical team. The discharge status of all enrolled subjects was recorded as death, referral, discharged alive, and discharged against medical advice. The diagnoses made by the medical team were also recorded.
Sample Size
For the derivation of prediction models, standard calculations of sample size do not apply since these calculations do not account for the model selection process (i.e., the optimization to achieve specified sensitivity and specificity cut-offs). One hundred events, corresponding to a total sample of approximately 1000 live-discharges (assuming a mortality rate of 10%, estimated using historical ward data), would be needed to obtain 80% power for ensuring that the lower 95% confidence limit on sensitivity will be at least 75%. An interim analysis of the study showed a mortality rate of approximately 5%. Funding was sufficient to increase enrollment to 1307 subjects.
Statistical Analysis
All variables were assessed using univariate logistic regression to determine their level of association with the primary outcome. Continuous variables were preferentially analyzed as continuous rather than categorical and were assessed for model fit using the Hosmer-Lemeshow test.[16] Missing data were imputed using the multiple imputation by chained equations method.[17] Following univariate analysis all variables were included in a multivariable logistic model and the primary model was developed using a stepwise selection process. Variables were removed or added individually based on using Aikaike’s Information Criterion (AIC). AIC was used as a summary measure to compare the overall predictive value across the models. This method is considered asymptotically equivalent to cross-validation and bootstrapping.[18,19] All models which having an AIC value within 10% of the lowest value were considered as reasonable candidates. The final selection of a model was judged on model parsimony (the simpler the better), availability of the predictors (with respect to minimal resources and cost), and the attained sensitivity (with at least 50% specificity). These criteria were determined a priori. The area under the curve (AUC) was reported for each model and represents the overall discriminatory ability of the model. All analyses were conducted using R (Vienna, Austria; http://www.R-project.org).
Results
During the study period, 1824 children met the age criteria (six months to five years) and were screened for eligibility. Of these, 517 (28%) were excluded as they were not admitted with a suspected or proven infectious illness. The most common reasons for exclusion included: malnutrition without concurrent infection (n = 192, 37%), already enrolled (n = 51, 10%) living outside of catchment area (n = 35, 7%) and refusal of consent (n = 22, 4%), (Fig 1). In total, 1307 children were enrolled. The median age at admission was 18.2 months (IQR 11.9–33.1) and 717 (54.8%) of subjects were male (Table 1). The proportion of children severely underweight (weight for age z-score less than -3) was 15.7% and 66 (5.1%) of children were HIV positive. The most common clinician assigned diagnoses included clinical malaria (49.7%) pneumonia (31.4%) and gastroenteritis/diarrhea (7.8%).
Mortality
During the course of admission, 65 (5.0%) subjects died in the hospital and 1242 (95%) were discharged alive. The median time to death was 2 days from admission (IQR 1–5). Among those discharged, 120 (9.7%) were discharged against medical advice. Twenty four variables were tested for their univariate association with mortality (Table 2). Blantyre Coma Scale, dichotomized as normal (score of 5) and abnormal (score of < 5), was highly associated with mortality and provided the highest area under the receiver operating characteristic curve, 0.73 (95% CI 0.67–0.79) and an abnormal score being associated with an odds ratio of 11.1 (95% CI 6.59–18.7). All anthropometric variables were associated with mortality during hospitalization. Low weight for age z-scores (underweight) and weight for height/length z-scores (wasting) provided the best discriminatory power for in hospital death with AUC’s of 0.64 (95% CI 0.56–0.71) and 0.63 (95% CI 0.55–0.70), respectively. Both systolic and diastolic blood pressure were associated with mortality, with raw diastolic pressure providing the highest AUC, 0.65 (95% CI 0.58–0.73). Other clinical variables including oxygen saturation, HIV diagnosis, and temperature were also associated with mortality but had lower areas under the ROC curve. Several variables including heart rate, respiratory rate, systolic blood pressure, hemoglobin concentration, and parasitemia were not associated with mortality.
Multivariate Prediction Model
Three models were developed for prediction of mortality. A primary model was developed using any of the available variables. Subsequent models were derived which selectively excluded certain variables from the primary model to ensure that prediction could be possible in the absence of certain variables which may not be available at all centers.
The first model included weight for age z-score, Blantyre coma scale and HIV status. The model equation was: logit [Pr (In-patient mortality)] −1.78+(−0.26; weight for age z-score) −2.50 (normal Blantyre coma scale) +1.32 (positive HIV diagnosis) and the area under the receiver operator characteristic curve was 0.85 (95% CI 0.80–0.89). At a probability cut-off of 0.030, this model had a sensitivity of 0.83 (95% CI 0.74–0.92) and a specificity of 0.76 (95% CI 0.73–0.78). We would expect the positive predictive value to be 0.15 (95% CI 0.11–0.19) and the negative predictive value to be 0.99 (95% CI 0.98–1.00) (Tables 3 and 4, Fig 2).
Model 2 replaced weight for age z-score in model 1 with mid-upper arm circumference, a variable that is more easily obtainable, especially in poorly resourced areas. The area under the ROC curve was 0.84 (95% CI 0.79–0.89). At a probability cut-off of 0.030 this model had a sensitivity of 0.80 (95% CI 0.70–0.90) and specificity of 0.76 (95% CI 0.74–0.79) and we would expect the positive predictive value and negative predictive value of 0.15 (95% CI 0.11–0.19) and 0.99 (95% CI 0.98–1.00), respectively, in a population similar to the derivation cohort (Tables 3 and 4).
Model 3 included MUAC and Blantyre coma scale and excluded HIV. At a probability cut-off of 0.30 this model had a sensitivity of 0.82 (95% CI 0.72–0.91) and a specificity of 0.71 (95% CI 0.68–0.73). The expected positive and negative predictive values would be 0.13 (95% CI 0.10–0.16) and 0.99 (0.98–0.99) (Tables 3 and 4).
Discussion
This study represents a systematic approach to creating an in-hospital mortality prediction tool for the under-five pediatric population admitted with an infectious illness in a resource constrained environment. The model developed is parsimonious, using only age, weight, Blantyre coma scale and HIV status to determine the probability of in-hospital mortality. Variables used in the development of the prediction model included only those thought to be both easily and reliably obtainable in most resource constrained contexts. Alternate models were developed incorporating different elements to ensure prediction would be possible in situations where certain variables may not be available.
In the context of limited resources, rapid risk determination is of critical importance. The implementation of an improved triage and training system (ETAT) alongside improvements in patient flow was shown to decrease in-patient mortality, especially early in-patient mortality.[20] Prediction models such as the one derived in this study could be used alongside a comprehensive strategy such as ETAT to improve care at the point of admission and focus human (eg. nursing) and clinical (eg. oxygen) resources on those children at highest risk of mortality. The determination of risk could play a unique role in sub-Saharan Africa where critically ill children admitted to lower level health centers require sufficient time to travel to referral hospitals. Further, as referral decisions are often made by non-physician health care providers, decision tools such as these could offer substantial aid with minimal training to ensure that those at high risk of mortality are referred. Using a risk-cut-off of 0.30, the referral population would have a mortality risk of 15% (95% CI 11% - 19%) compared to a mortality risk of 1% (95% CI 0% - 2%) in those not referred, if similar to the admitted sample in this study. Future research must, however, examine the effect of transporting critically ill children as identified with these models as transportation itself may confer its own risks.
Other models developed for mortality prediction in settings without resource constraints have been evaluated in resource constrained settings have been shown to not perform adequately. A critical difference in the populations in whom models such as PRISM were derived is the much lower prevalence of moderate and severe malnutrition. In this present study an anthropometric measure played a crucial role in all derived models, providing a majority of the discriminatory power in final model. Malnutrition as an independent and critical contributor to infectious disease morbidity and mortality cannot be over emphasized and should be considered as an important component of the evaluation of any infectious illness. In addition to malnutrition, an abnormal Blantyre coma scale was also of critical importance, proving more discriminatory than vital signs in predicting mortality.
Although the Integrated Management of Childhood Illness algorithm was not designed as a prediction tool, it does provide referral criteria, listed as danger signs. A recent study from Tanzania evaluated the predictive utility of these criteria for in-hospital mortality. Among 387 children aged two months to five years, and an overall mortality rate of 7.4%, one or more IMCI danger sign had a sensitivity of 72% (95% CI 56% - 88%) for predicting in-hospital death and would identify 38% of subjects as high-risk.[21] Using a cut-off of 0.30, our model, using fewer and more reliably obtained variables, has a higher sensitivity of 0.83 (95% CI 0.74–0.92) and would only identify approximately 25% as high-risk, allowing for a more efficient utilization of resources.
This study was limited by fewer cases of the primary outcome (in-hospital mortality) than was initially anticipated. Although designed to derive prediction models using 100 outcomes, this enrolled only 65 children who died in-hospital. Despite the relatively low number of outcomes our primary model had an AUC of the receiver operating characteristic curve of 0.85 with the lower limit being 0.80, highlighting the excellent discriminating ability of the final model and the utility of each of the predictive variables in the model. A further limitation is that currently it is not known whether the identification of high-risk children can actually result in saved lives, although it would be difficulty to justify not acting on such information. Another limitation of this study are the mathematical calculations involved with the use of the proposed models. To address this important limitation our research group has developed and evaluated a mobile application of this model and one for post-discharge mortality. Finally, the lack of external validity limits widespread use and scaling of this model.[22] Wider validation studies must be conducted prior to the implementation of this model in any setting.
Although a major strength of this study was its development using more than one hospital, these results require confirmation at other hospitals in other resource constrained countries for further calibration prior to recommending their uptake. This is of particular importance if used to aid in referral decision making at lower level health centers, as admission criteria are certainly different than in larger hospitals producing a unique population of children. These models could also aid in the development of a standardized in-patient mortality prediction score could also be helpful in the designing of clinical trials of interventions in resource constrained settings.
In conclusion, a parsimonious prediction tool using easily collected predictors can be used to efficiently predict in-hospital mortality. Further research to externally validate this model is required prior to widespread implementation.
Acknowledgments
We gratefully acknowledge the contributions of Annet Twinomuguni, Justine Kamazima, Agaba Collins, Clare Komugisha, Solome Kobugyenyi, Alexander Mutungi and Hassan Bariahikwa. Without their dedication this study could not have been completed.
Author Contributions
Conceived and designed the experiments: CPL EK JMA JS NK HW AN J. Kabakyenga J. Kiwanuka MOW. Performed the experiments: EK CPL JMA MOW. Analyzed the data: NL GZ JMA JS HW MOW. Wrote the paper: NL MOW. Interpretation of data: NL CPL EK JMA JS NK AN J. Kabakyenga J. Kiwanuka MOW. Critically revised the manuscript: CPL EK GZ JMA JS NK HW AN J. Kabakyenga J. Kiwanuka.
References
- 1.
United Nations, General Assembly 56th session. Road map towards the implementation of the United Nations millennium development declaration: report of the Secretary-General (UN document no. A/56/326). New York; 2001.
- 2. Wang H, Liddell CA, Coates MM, Mooney MD, Levitz CE, Schumacher AE, et al. Global, regional, and national levels of neonatal, infant, and under-5 mortality during 1990–2013: a systematic analysis for the Global Burden of Disease Study 2013. Lancet. 2014; 957–979. pmid:24797572
- 3. Liu L, Oza S, Hogan D, Perin J, Rudan I, Lawn JE, et al. Global, regional, and national causes of child mortality in 2000–13, with projections to inform post-2015 priorities: an updated systematic analysis. Lancet. 2015;385: 430–440. pmid:25280870
- 4. Liu L, Johnson HL, Cousens S, Perin J, Scott S, Lawn JE, et al. Global, regional, and national causes of child mortality: an updated systematic analysis for 2010 with time trends since 2000. Lancet. 2012;379: 2151–61. pmid:22579125
- 5. Izadnegahdar R, Cohen AL, Klugman KP, Qazi SA. Childhood pneumonia in developing countries. Lancet Respir Med. Elsevier Ltd; 2013;1: 574–84.
- 6.
World Health Organization. Handbook IMCI: Integrated management of childhood illness [Internet]. World Health Organization; 2006. Available: http://books.google.com/books?hl=en&lr=&id=4Zhk4Y6UshcC&oi=fnd&pg=PR9&dq=Handbook:+IMCI+integrated+management+of+childhood+illness&ots=Ik9u40zA5O&sig=vgnON6F-7WFXos4n3emes18IKsg
- 7. Pollack M, Ruttimann U, Getson P. Pediatric risk of mortality (PRISM) score. Crit Care Med. 1988;16: 1110–1116. Available: http://journals.lww.com/ccmjournal/Abstract/1988/11000/Pediatric_risk_of_mortality__PRISM__score.6.aspx pmid:3048900
- 8. Shann F, Pearson G, Slater A, Wilkinson K. Paediatric index of mortality (PIM): a mortality prediction model for children in intensive care. Intensive Care Med. 1997;23: 201–207. pmid:9069007
- 9. Slater A, Shann F, Pearson G. PIM2: a revised version of the Paediatric Index of Mortality. Intensive Care Med. 2003;29: 278–85. pmid:12541154
- 10. Emukule GO, McMorrow M, Ulloa C, Khagayi S, Njuguna HN, Burton D, et al. Predicting mortality among hospitalized children with respiratory illness in Western Kenya, 2009–2012. PLoS One. 2014;9: e92968. pmid:24667695
- 11.
Karlen W, Dumont G, Petersen C, Gow J, Lim J, Sleiman J, et al. Human-centered phone oximeter interface design for the operating room: Pulse Oximeter Interfaced to a Mobile Device for Anesthesia Monitoring in the Developing World. Proceedings of the International Conference on Health Informatics. Rome, Italy: SciTePress; 2011. pp. 433–38.
- 12.
The WHO Child Growth Standards [Internet]. 2006 [cited 1 Mar 2015]. Available: http://www.who.int/childgrowth/standards/en/
- 13. Fleming S, Thompson M, Stevens R, Heneghan C, Pluddemann A, MacOnochie I, et al. Normal ranges of heart rate and respiratory rate in children from birth to 18 years of age: A systematic review of observational studies. Lancet. 2011;377: 1011–1018. pmid:21411136
- 14. The fourth report on the diagnosis, evaluation, and treatment of high blood pressure in children and adolescents. Pediatrics. 2004;114: 555–76. pmid:15286277
- 15. Benatar S, Hewlett A, Nunn J. The use of iso-shunt lines for control of oxygen therapy. Br J Anesth. 1973;45: 711–718. Available: http://bja.oxfordjournals.org/content/45/7/711.short
- 16. Hosmer DW, Lemeshow S. Goodness of fit tests for the multiple logistic regression model. Commun Stat—Theory Methods. 1980;9: 1043–1069. Available: http://www.tandfonline.com/doi/abs/10.1080/03610928008827941
- 17. Van Buuren S, Groothuis-Oudshoorn K. Multivariate Imputation by Chained Equations. J Stat Softw. 2011;45: 1–67.
- 18. Stone M. An Asymptotic Equivalence of Choice of Model by Cross-Validation and Akaike’s Criterion. J R Stat Soc Ser B-Methodological. 1977;39: 44–47.
- 19. Shibata R. Bootstrap estimate of Kullback-Leibler information for model selection. Stat Sin. 1997;7: 375–394. Available: http://www3.stat.sinica.edu.tw/statistica/oldpdf/A7n27.pdf
- 20. Molyneux E, Ahmad S, Robertson A. Improved triage and emergency care for children reduces inpatient mortality in a resource-constrained setting. Bull World Health Organ. 2006;84: 314–319. pmid:16628305
- 21. Clifton DC, Ramadhani HO, Msuya LJ, Boniface NN, Kinabo GD, Buchanan AM, et al. Predicting mortality for paediatric inpatients where malaria is uncommon. Arch Dis Child. 2012;97: 889–894. pmid:22872067
- 22. English LL, Dunsmuir D, Kumbakumba E, Ansermino JM, Larson CP, Lester R, et al. An mHealth Intervention for Pediatric Post-Discharge Mortality: Design, Usability, and Feasibility of the PAediatric Risk Assessment, (PARA) Mobile App. JMIR mHealth uHealth (in press),