Skip to main content
Advertisement
  • Loading metrics

Evaluation of a Smartphone Decision-Support Tool for Diarrheal Disease Management in a Resource-Limited Setting

  • Farhana Haque ,

    Contributed equally to this work with: Farhana Haque, Robyn L. Ball

    Affiliations Institute of Epidemiology, Disease Control, and Research (IEDCR), Bangladesh Ministry of Health and Family Welfare, Dhaka, Bangladesh, Infectious Diseases Division (IDD), International Centre for Diarrhoeal Disease Research, Bangladesh (icddr,b), Dhaka, Bangladesh

  • Robyn L. Ball ,

    Contributed equally to this work with: Farhana Haque, Robyn L. Ball

    Affiliation Quantitative Sciences Unit, Stanford University School of Medicine, Stanford, California, United States of America

  • Selina Khatun,

    Affiliation Institute of Epidemiology, Disease Control, and Research (IEDCR), Bangladesh Ministry of Health and Family Welfare, Dhaka, Bangladesh

  • Mujaddeed Ahmed,

    Affiliation Institute of Epidemiology, Disease Control, and Research (IEDCR), Bangladesh Ministry of Health and Family Welfare, Dhaka, Bangladesh

  • Saraswati Kache,

    Affiliation Department of Pediatrics, Stanford University School of Medicine, Stanford, California, United States of America

  • Mohammod Jobayer Chisti,

    Affiliation Nutrition and Clinical Services Division, International Centre for Diarrhoeal Disease Research, Bangladesh (icddr,b), Dhaka, Bangladesh

  • Shafiqul Alam Sarker,

    Affiliation Nutrition and Clinical Services Division, International Centre for Diarrhoeal Disease Research, Bangladesh (icddr,b), Dhaka, Bangladesh

  • Stace D. Maples,

    Affiliation Geospatial Center, Stanford University Libraries, Stanford, California, United States of America

  • Dane Pieri,

    Affiliation Independent Technology Developer, San Francisco, California, United States of America

  • Teja Vardhan Korrapati,

    Affiliation BeeHyv Software Solutions Pvt. Ltd., Hyderabad, India

  • Clea Sarnquist,

    Affiliation Department of Pediatrics, Stanford University School of Medicine, Stanford, California, United States of America

  • Nancy Federspiel,

    Affiliation Department of Medicine, Stanford University School of Medicine, Stanford, California, United States of America

  • Muhammad Waliur Rahman,

    Affiliations Institute of Epidemiology, Disease Control, and Research (IEDCR), Bangladesh Ministry of Health and Family Welfare, Dhaka, Bangladesh, Infectious Diseases Division (IDD), International Centre for Diarrhoeal Disease Research, Bangladesh (icddr,b), Dhaka, Bangladesh

  • Jason R. Andrews,

    Affiliation Department of Medicine, Stanford University School of Medicine, Stanford, California, United States of America

  • Mahmudur Rahman,

    Affiliation Institute of Epidemiology, Disease Control, and Research (IEDCR), Bangladesh Ministry of Health and Family Welfare, Dhaka, Bangladesh

  •  [ ... ],
  • Eric Jorge Nelson

    eric.nelson@ufl.edu

    Current address: Emerging Pathogens Institute (Room 273), Department of Pediatrics, College of Medicine, University of Florida, Gainesville, Florida

    Affiliation Department of Pediatrics, Stanford University School of Medicine, Stanford, California, United States of America

  • [ view all ]
  • [ view less ]

Abstract

The emergence of mobile technology offers new opportunities to improve clinical guideline adherence in resource-limited settings. We conducted a clinical pilot study in rural Bangladesh to evaluate the impact of a smartphone adaptation of the World Health Organization (WHO) diarrheal disease management guidelines, including a modality for age-based weight estimation. Software development was guided by end-user input and evaluated in a resource-limited district and sub-district hospital during the fall 2015 cholera season; both hospitals lacked scales which necessitated weight estimation. The study consisted of a 6 week pre-intervention and 6 week intervention period with a 10-day post-discharge follow-up. Standard of care was maintained throughout the study with the exception that admitting clinicians used the tool during the intervention. Inclusion criteria were patients two months of age and older with uncomplicated diarrheal disease. The primary outcome was adherence to guidelines for prescriptions of intravenous (IV) fluids, antibiotics and zinc. A total of 841 patients were enrolled (325 pre-intervention; 516 intervention). During the intervention, the proportion of prescriptions for IV fluids decreased at the district and sub-district hospitals (both p < 0.001) with risk ratios (RRs) of 0.5 and 0.2, respectively. However, when IV fluids were prescribed, the volume better adhered to recommendations. The proportion of prescriptions for the recommended antibiotic azithromycin increased (p < 0.001 district; p = 0.035 sub-district) with RRs of 6.9 (district) and 1.6 (sub-district) while prescriptions for other antibiotics decreased; zinc adherence increased. Limitations included an absence of a concurrent control group and no independent dehydration assessment during the pre-intervention. Despite limitations, opportunities were identified to improve clinical care, including better assessment, weight estimation, and fluid/ antibiotic selection. These findings demonstrate that a smartphone-based tool can improve guideline adherence. This study should serve as a catalyst for a randomized controlled trial to expand on the findings and address limitations.

Author Summary

Diarrheal disease is responsible for one in ten deaths among children less than five years of age globally. Innovative interventions to address gaps in the clinical care of these patients are lacking, yet will likely reduce the morbidity and mortality from diarrheal diseases. Therefore, the objective of this pilot study was to take a technology-enabled approach to improve guideline adherence, including antibiotic selection for diarrheal disease management in a resource-limited setting. To do this we adapted WHO guidelines to a smartphone platform and evaluated the approach in Bangladesh at two rural hospitals. The platform was durable and demonstrated positive improvement in guideline adherence. The results suggest that the decision-support tool was associated with a decrease in intravenous fluid use while maintaining safety, an increase in use of the recommended antibiotic, and a decrease in use of medications not recommended. This study represents a critical first step towards technology-enabled decision-support tools for diarrheal disease in resource-limited settings.

Introduction

The provision of high-quality clinical care in resource-limited settings is challenged by logistical, educational, and temporal constraints that are exacerbated by a high volume of patients. This is especially true for the management of diarrheal diseases, which may overwhelm health facilities amid outbreaks or seasonal swells of disease. These diseases disproportionately burden poor communities and remain the second leading cause of death for children less than 5 years of age [13]. While outpatient decision-support tools like the paper-based World Health Organization (WHO) Integrated Management of Childhood Illness (IMCI) have improved community efforts despite limitations [48], inpatient references like the WHO Pocketbook of Hospital Care for Children are often scarce or anecdotally don’t meet providers needs in high-volume situations [9].

These collective challenges manifest in poor guideline adherence and exacerbate the ongoing struggle to improve care for patients with diarrheal disease, especially during large-scale outbreaks. For example, weight-based dosing is fundamental to pediatric care, however equations to estimate weight are rarely both accurate and user-friendly/accessible at the bedside [1013]. Secondly, the inappropriate use of antibiotics is driving the proliferation of drug resistant pathogens; however prescription behavior change is challenging with complex unpredictable barriers, especially in developing countries [14]. Lastly, complications from basic interventions, such as fluid resuscitation, are more nuanced in resource-limited settings than previously thought [15, 16]. Relatively high rates of post-discharge mortality suggest that improved in-hospital efforts might decrease overall morbidity and mortality [17].

This pilot study aimed to address a subset of these challenges by adapting the WHO diarrhea management guidelines onto a smartphone platform and evaluating the approach in resource-limited hospitals. A simple ‘Rehydration Calculator’ was developed based on end-user feedback; functionality included weight-for-age estimation, a dehydration assessment guide, and treatment recommendations. The calculator was evaluated in a clinical study at two remote hospitals in Bangladesh to determine how technology-enabled decision-support impacts standard of care for the management of diarrheal disease.

Methods

Ethics Statement

This pilot study was approved by the Institutional Review Boards (IRB) at the IEDCR (IEDCR/IRB/2015/03) and Stanford University School of Medicine (6208). Written informed consent was obtained from all adult participants and guardians of minors (< 18 years); assent was obtained for children 11–17 years of age. The human experimentation guidelines of the US Department of Health and Human Services were followed during the conduct of this research.

Technology Development

Two software modalities were developed and utilized in this study (Fig 1A; S1 Movie; S1 Software). First: the WHO-guidelines were adapted onto a smartphone for decision-support and coined the ‘Rehydration Calculator’ (Fig 1B and 1C). Its development relied on end-user guidance (e.g. field clinical providers) to make it fast, desired, and durable in high-volume, low-resource settings. The calculator captures no personal health information and is not password protected. The calculator does, however, capture basic information to calculate the recommended treatment plan (aka age, gender, watery/bloody stool, five clinical signs of dehydration, allergies, and danger signs). On the back-end, these symptoms are encrypted, stamped with the time and GPS location, and can be aggregated for syndrome-based surveillance. Second: a wireless data collection and aggregation platform (aka the Outbreak Responder platform) was built for the research team (aka ‘response team’). It was developed based on end-user guidance (e.g. field research staff and epidemiologists) to be durable in settings with limited connectivity, yet capture essential epidemiologic data. The user interface is organized as a medical chart and collectively captures critical demographic, syndromic, laboratory, and outcomes information. The sections are ‘Patient Information’, ‘History of Present Illness,’ ‘Exam’, and ‘Results’ (Fig 1D). The Outbreak Responder captures personal health information and is therefore, password protected, encrypted, and built to industry standards for health information data security. Both technologies are Android-based, function on/off line, and are available upon request (S1 Software). Although the software was built and tested using a locally produced low-cost smartphone (Walton, Bangladesh; 80 USD), a Samsung Note 3 device (200–400 USD) was used in this study because of the improved battery life, ideal screen size (5 in.), and screen responsiveness.

thumbnail
Fig 1. Rehydration calculator.

(A) Home page with the Rehydration Calculator for decision-support (upper) and the data collection platform (lower). (B) Data input page. (C) WHO derived recommendations for medication and fluid administration; units can be changed by side-swiping. “i” represents a link to educational summaries and primary publications. (D) Outbreak Responder data collection and aggregation platform for the research team (aka ‘Response Team’). ‘QI’ is a portal for customized quality improvement questions intended to be used by researchers only.

https://doi.org/10.1371/journal.pntd.0005290.g001

Enrollment and Study Design

Location.

This pilot study was conducted in the rural northern district of Netrokona, Bangladesh (population of 2.2 million). Netrokona was chosen because it was remote, extremely resource-limited, and has frequent diarrheal disease outbreaks, including cholera [18]. There is one district and ten sub-district government hospitals in Netrokona that collectively receive approximately 5000 patients with diarrheal disease per month; the study was conducted at the main district hospital and at one sub-district hospital (Madan) by the Bangladesh government’s Institute of Epidemiology, Disease Control and Research (IEDCR) in collaboration with Stanford University.

Timeline.

The study consisted of a 6 week pre-intervention and 6 week intervention period; standard of care was maintained throughout the study except that the Rehydration Calculator was used by the admitting clinician during the intervention period only. The study was conducted from September to December 2015, when cholera outbreaks typically occur in northern Bangladesh [19]. The pre-intervention and intervention periods each spanned 36 study days from September 19 to November 5, 2015, and from November 15 to December 29, 2015, respectively. Prior to both the pre-intervention and intervention periods, onboarding periods were required from September 5 to September 18, 2015 and from November 6 to November 14, 2015, respectively. During this time, additional individualized training was provided (see implementation and training section below) to the research staff prior to the pre-intervention and to admitting clinicians prior to the intervention. These onboarding periods also helped mitigate a possible Hawthorne effect from the presence of the research staff (pre-intervention) or presence of the Android device (intervention) at admission.

Inclusion criteria.

Inclusion criteria were patients two months of age and older presenting with diarrheal disease (three or more loose stools per 24 hours) at the district hospital’s diarrhea ward or the sub-district hospital’s diarrhea corner [17, 20]. During the pre-intervention period, patients with co-morbidities were excluded, with the exception of severe malnutrition, which allowed for minimal disruption of the standard of care. During the intervention period, patients with clinical signs of severe malnutrition were also excluded because severely malnourished patients require highly detailed clinical management that was beyond the capacity of the two sites and the current design of the calculator. Patients with severe malnutrition were referred to a pediatric sub-specialist or to a tertiary care facility.

Primary outcome.

The primary outcome was adherence to WHO guidelines for diarrheal disease management with and without the decision-support from the Rehydration Calculator. Increased adherence was defined as a statistically significant increase in orders of recommended treatment during intervention compared to pre-intervention (e.g. IV fluid use and volume ordered, use and type of antibiotic ordered, use of zinc).

Implementation.

Before the pre-intervention period, approximately 100 clinicians and medical staff received didactic training in the respective hospital conference rooms (2 hours). The curriculum included an epidemiologic review of diarrheal disease in the study area and explanation of the study goals and design [to observe standard of care with and without the use of the Rehydration Calculator]. It was emphasized that the study design was planned to minimize disruption. A refresher on rehydration guidelines was not performed and paper-based decision-support was not provided. Physicians were told to assess dehydration per their training; in general, this region had been trained to use the Dhaka Method (icddr,b method) during past cholera outbreaks (see clinical assessment and management section). It was required that the physicians record onto the admission sheet the physician’s name, patient name, patient age/ gender, diarrhea type (e.g. watery, bloody), dehydration assessment (no, some, severe), and treatment plan (fluids and medications). The role of the research staff was limited to data and sample collection; a separate multi-day training was provided to the research staff on how to perform consent/ assent, collect clinical data, collect samples, and transfer data from the paper record to the Outbreak Responder software.

Prior to the intervention, a second training was conducted. Clinicians were trained on how to use the Rehydration Calculator in a didactic format with trainees using test phones (2 hours). The curriculum consisted of a review of the study design and goals. Materials included diagrams highlighting the use of the Rehydration Calculator by the admitting physician. The clinicians were trained to assess and generate the treatment recommendations with the Rehydration Calculator and to transcribe recommendations from the Rehydration Calculator onto the admission sheet. Clinicians were free to deviate from the recommendations based on situational concern.

One smartphone (Samsung Note 3) was allocated to each field research assistant and medical technologist. One shared smartphone was provided to each admitting group of doctors. When the clinical phone needed to be charged, the phone was exchanged for a charged phone by the research staff. Additional individualized clinician training was provided as needed during the onboarding periods; the need for additional training was directed towards users with no prior smartphone experience. Data collection and recording in the intervention was conducted in the same manner as the pre-intervention. Monitoring and evaluation were conducted under the guidance of a research study coordinator/ physician: calls were placed daily to research staff and senior clinical representatives of the study sites, onsite evaluation was performed every 2–3 weeks, and review of the hard and soft copy registry was conducted at the study midpoint and endpoint. The onsite evaluation consisted of group feedback sessions to identify any clinical or study concerns.

Clinical Assessment and Management

Weight estimation.

Many resource-limited health care facilities lack scales. When faced with this situation in Bangladesh, the standard of practice is to use a ‘best guess’ to estimate weight-for-age. The 2011 and 2014 Bangladesh Demographic and Health Survey reported that the average healthy pediatric weight in the Dhaka Division (contains Netrokona District) is at the 2006 WHO -1 weight for age z-score [21]. For children between 2 months and less than 10 years of age, the calculator was therefore designed to make sex-specific weight estimates to the WHO -1 z-score [22]. For children 10 to less than 15 years of age, sex-specific weight estimates were set to the USA Centers for Disease Control (CDC) -1 z-score [23] since WHO weight-for-age data are not available for these ages. For ages 15 to less than 20 years of age, estimates were set to 42 kg (females) and 45 kg (males). For patients 20 years of age and older, estimates were set to 45 kg (females) and 50 kg (males); these estimates were based on weight measurements collected during the pre-intervention period. In order to maintain consistency with the guidelines and standard of care, no correction was made based on dehydration status.

Clinical assessment.

During the pre-intervention period, clinicians classified dehydration status by standard of care, independent of the calculator. However during the intervention period, the clinician used the Rehydration Calculator to gather essential data and to calculate the treatment plan based on the WHO-derived rehydration guideline called the Dhaka Method (Fig 1) [9, 13]. Although there are multiple assessment methods with different strengths [24, 25], the Dhaka Method was chosen because the local staff were most familiar with this modality. Dehydration status was classified into ‘none’, ‘some’, and ‘severe’ dehydration which approximates 0–4%, 5–9%, and 10% weight loss, respectively. The calculator used a Bangladesh Ministry of Health and Family Welfare/ icddr,b adjustment to the WHO algorithm for dehydration assessment [9, 13]. In brief, ‘some’ dehydration requires at least two findings of restlessness/irritability, sunken eyes, drinks eagerly or thirsty, and the skin pinch goes back slowly (two to less than three seconds) with at least one key finding being restlessness/irritability or drinks eagerly/thirsty. ‘Severe’ dehydration occurs when the criteria for ‘some’ dehydration are met plus at least one key finding in the highest category of lethargy/unconscious, skin goes back very slowly (three or more seconds), drinks poorly/unable to drink, or uncountable/absent peripheral pulse. No dehydration (‘none’) is the default when neither criteria for ‘some’ nor ‘severe’ are met.

Clinical management.

During the pre-intervention period, the clinicians provided standard of care without use of the Rehydration Calculator; however, during the intervention period, the clinicians used the Rehydration Calculator to assess and calculate the WHO fluid and medication recommendations [4, 9, 26]. In brief, severely dehydrated patients (10% fluid loss by body weight) were recommended to receive intravenous fluid resuscitation of a total volume of 100 ml per kg; the time course of administration was age dependent [9]. Azithromycin was recommended for severely dehydrated patients with acute watery diarrhea, which was based on antibiotic sensitivity patterns at the icddr,b [2729]. Given that the study was conducted in a cholera endemic region during the fall period when cholera outbreaks typically occur, azithromycin was also recommended for patients with some dehydration (5–9% fluid loss by body weight) from acute watery diarrhea [30]. Based on icddr,b and WHO recommendations, the calculator recommended pivmecillinam [31, 32] for patients with bloody diarrhea, suggestive of shigellosis. The calculator recommended zinc for all patients less than 5 years of age; although zinc supplementation will not cause harm, it was not recommended for patients older than 5 years of age [9].

Microbiological Assays

A subset of stool samples (first and last patient per day) underwent targeted culture and sensitivity testing for Vibrio cholerae, Salmonella spp. and Shigella spp. [33] in the IEDCR and the International Centre for Diarrhoeal Diseases, Bangladesh (icddr,b) laboratories; stool swabs were placed in Cary Blair media, stored at four degrees Celsius, and transported from the remote field site on a bimonthly to monthly basis. Assays for additional pathogens (e.g. ETEC, cryptosporidium, norovirus, rotavirus) were not performed because the intent of the cultures was to identify antibiotic sensitivity patterns for pathogens that clinically warrant antibiotic treatment, not for comprehensive surveillance.

Data Collection for Hospital and Post-Discharge Course

Basic secondary outcomes were collected from the discharge record (e.g. discharge type, mortality). At 10-days post-discharge, the research staff called the patient/guardian to collect secondary outcomes (e.g. readmission, mortality). Data were aggregated and reported by the research staff via the Outbreak Responder platform; the post-discharge calls were also placed from within the software.

Statistical Analysis

Patient characteristics were described by percentages for dichotomous (yes/no) variables and by the median, 1st and 3rd quartiles (Q1, Q3) for continuous data. For dichotomous variables, significant differences were assessed by the two-sided Fisher’s Exact test at the 0.05 level; the risk ratio (RR) was calculated by dividing the proportion of individuals with the event during the intervention period by the proportion of individuals with the event during the pre-intervention period; thus, an RR < 1 indicates the event is less likely to occur in the intervention and an RR > 1 indicates the event is more likely to occur in the intervention. For continuous variables, significance at the 0.05 level was assessed by either the two-sided Wilcoxon signed rank test (comparison to a median) or the two-sided Wilcoxon rank sum test. When calculating the weight-adjusted IV fluid volume and the recommended IV fluid volume, measured weights were used where available, and otherwise (i.e., sub-district intervention), estimated weights were used. To calculate congruence during the intervention between IV fluid volume ordered and the recommended IV fluid volume, we used the Wilcoxon rank sum test to test if volume ordered was more than 30% different than what was recommended.

Standard of care at both study hospitals relies on a best guess weight estimation without a measured weight, however weight was independently measured to evaluate the weight estimation performed by the calculator. To assess how well estimated weights matched measured weights for children younger than 15 years of age, we calculated the difference between estimated and measured weight and used the Wilcoxon signed rank test to evaluate the differences; measured weights were not adjusted for dehydration status. For groups of patients aged 15 and older, we compared the distribution of measured weights to the gender-specific weight estimates with the Wilcoxon signed rank test. Measured weights during the intervention at the sub-district were unfortunately not obtained, and therefore, to assess how well the estimated weights of patients younger than 15 years of age matched the measured weights, we included district measurements and only sub-district pre-intervention measurements. For those aged 15 and older, we compared estimated weights to district intervention measurements only (since we used measured weights during pre-intervention to obtain estimated weights for these ages). After determining that the weight estimates for children less than 5 years of age were too high, we evaluated WHO percentiles and identified the percentile that best approximated weight for age in this cohort as the percentile that minimized the root mean square error (RMSE) between measured weights and gender-specific WHO percentile estimated weights. All summary statistics and statistical analyses were completed in the statistical software package R version 3.2.2 [34]. All data except for a few restricted items are provided in the supplementary materials (S3 Dataset).

Results

Population Characteristics

A total of 841 patients were enrolled and their records analyzed (Table 1). In the pre-intervention period, we enrolled 325 patients (204 district and 121 sub-district), and in the intervention period, we enrolled 516 patients (430 district and 86 sub-district). Onboarding periods prior to both the pre-intervention and intervention periods were permitted to accommodate for unanticipated additional training and mitigate possible Hawthorne effects (see timeline section). During these periods, 113 additional patients were enrolled but were excluded from the analysis. No patients were excluded for co-morbidities because in practice, these patients were admitted directly to the non-diarrheal wards for more advanced care prior to study screening. Seven patients (0.8%) had less than 3 loose stools in 24 hours and were excluded from the analysis (see inclusion criteria). Comparing the pre-intervention and intervention periods in the district hospital, we observed significant differences (p < 0.001) in the proportions of children less than 5 years and adults 20 years and older (Table 1); therefore, we controlled for age in subsequent analyses. A total of 50 doctors participated in the study with participation generally spanning study periods; 35 doctors participated at the district hospital (27 pre-intervention period; 17 intervention) and 15 doctors participated at the sub-district hospital (13 pre-intervention; at least 3 intervention)

thumbnail
Table 1. Population characteristics during the pre-intervention and intervention periods.

https://doi.org/10.1371/journal.pntd.0005290.t001

Microbiologic Findings

A total of 277 targeted cultures were performed and identified V. cholerae (N = 19; 7%); Aeromonas spp. (N = 19; 7%), Shigella spp (N = 5; 2%), and Salmonella spp (N = 3; 1%); note that the approach was targeted and did not test for viral pathogens and other common bacterial pathogens. V. cholerae strains were sensitive to azithromycin (19/19) and ciprofloxacin (19/19), had intermediate sensitivity to cotrimoxazole (16/19) and tetracycline (14/17), and were resistant to erythromycin (19/19). One cholera patient was co-infected with at least Aeromonas spp.

Clinical Assessment and Management

Weight estimation.

Standard of care at both study hospitals relied on a ‘best guess’ weight because scales historically have not been available. During the intervention, the Rehydration Calculator provided weight estimates (see methods). For children less than 5 years of age, the sex-specific estimated weight-for-age (WHO -1 z-score) compared to weights independently measured by the study team were high for both sexes (both p < 0.001; Fig 2A females and Fig 2B males). The analysis revealed that measured weights for girls (N = 120) better matched the WHO 5th percentile (RMSE = 1.17; Fig 2A) while boys’ measured weights (N = 216; Fig 2B) were lower and better matched the WHO 3rd percentile (RMSE = 1.25). Differences between estimated and measured weights for all other age groups were not significant (Fig 2C and 2D).

thumbnail
Fig 2. Weight estimation.

(A and B) Measured weights for female (N = 120) and male patients (N = 216) less than 5 years, respectively. (C and D) Measured and estimated weights during the pre-intervention and intervention periods for older female and male patients, respectively. Data are presented as standard box plots; estimated weights used during the study are 42 kg for females 15–19 years; 45 kg for female 20+ years; 45 kg for males 15–19 years; 50 kg for males 20+ years.

https://doi.org/10.1371/journal.pntd.0005290.g002

Clinical assessment.

During the pre-intervention period, dehydration status was assessed by the clinician, without guidance by the Rehydration Calculator (Fig 3A), and we observed a large difference between the percentages of patients admitted with severe dehydration at the district hospital (4/204; 2%) and the sub-district hospital (99/121; 82%). However, once the Rehydration Calculator was used in the intervention period, there was no significant difference in the percentage of patients assessed as severely dehydrated between the two sites (66/430, 15% district; 20/86, 23% sub-district; p = 0.17).

thumbnail
Fig 3. Dehydration status and IV fluid administration.

(A) Percentages of patients with dehydration status ‘none’, ‘some’, or ‘severe’; IV fluids were recommended for severe dehydration (red). (B) Percentages of patients who received IV fluids. (C) Weight-adjusted IV fluid volume ordered by age group (ml ordered/patient weight in kg; the value 0.10 represents the recommended volume for severely dehydrated patients). *p<0.05, **p<0.01, ***p<0.001.

https://doi.org/10.1371/journal.pntd.0005290.g003

Clinical management.

During the intervention, the proportion of patients prescribed IV fluids significantly decreased at both hospitals (both p < 0.001; Fig 3B). At the district, the RR was 0.5 (pre-intervention 77% and intervention 41%) and at the sub-district, the RR was 0.2 (pre-intervention 99% and intervention 23%). Although the proportion of patients prescribed IV fluids decreased, the weight-adjusted volume of IV fluids increased for children less than 5 years (p < 0.001), adults ages 40 to 59 (p = 0.003), and adults 60 and older (p = 0.03; Fig 3C). Considering only those patients in the intervention period for whom we had objective measures of dehydration status (Fig 4A; N = 516 total) and for whom IV fluids were ordered (Fig 4B; N = 197), we found that the IV fluid volume ordered in severely dehydrated patients was congruent (did not differ by more than 30%) with the IV fluid volume recommended (p = 0.1; Fig 4B). Although IV fluids are generally recommended for severely dehydrated patients, IV fluids were ordered for patients with ‘none’ and ‘some’ dehydration (Fig 4).

thumbnail
Fig 4. Dehydration status and IV fluid administration in the intervention period.

(A) Percentages of patients in the intervention period (N = 516) for whom IV fluids were/were not ordered by dehydration status (‘none’, ‘some’, or ‘severe’). (B) Comparison of volume of IV fluid recommended versus ordered during the intervention period for those patients who received IV fluids (N = 197); diagonal line represents agreement between the volume recommended and ordered; dashed lines represent differences of 30%.

https://doi.org/10.1371/journal.pntd.0005290.g004

Antibiotic use approached 100% during the pre-intervention period (320/325) and remained high in the intervention period (494/526); this was in the context of the majority of patients diagnosed with acute watery diarrhea with moderate or severe dehydration (Table 1). Given cholera in the study area, the recommended antibiotic for acute watery diarrhea is azithromycin. To determine antibiotic guideline adherence, we analyzed only those records in which azithromycin was recommended (199/204; 98% district pre-intervention, 385/430; 90% district intervention, 117/121; 97% sub-district pre-intervention, and 86/86; 100% sub-district intervention) and found that proportions of azithromycin prescriptions significantly increased at both hospitals (district p < 0.001; 13% pre-intervention to 87% intervention; sub-district p = 0.35; 63% pre-intervention to 99% intervention) with RRs of 6.9 and 1.6, respectively (Fig 5A and 5B). During the intervention, use of antibiotics not recommended for acute watery diarrhea in our study location (e.g. ciprofloxacin and metronidazole) significantly decreased. For ciprofloxacin, the RR was 0.1 at the district hospital (pre-intervention 70%, 5% intervention, p < 0.001), and ciprofloxacin use ceased entirely at the sub-district hospital (69% pre-intervention, 0% intervention, p < 0.001). Metronidazole was rarely used at the district hospital and use ceased entirely at the sub-district hospital (40% pre-intervention and 0% intervention; p < 0.001). For patients less than 5 years of age, zinc use was high and adhered well to guidelines during both periods (Fig 5C); the small increase with the use of the calculator was not significant at either hospital (p = 0.80 district, 82% pre-intervention and 90% intervention; p = 0.89 sub-district, 91% pre-intervention and 98% intervention) with RRs of 1.1 at both hospitals. Zinc is not typically indicated for patients older than 5 years of age (Fig 5D) and in this group, zinc use significantly decreased with RRs at the district hospital of 0.4 (pre-intervention 89%, intervention 31%) and at sub-district hospital of 0.1 (pre-intervention 59%, intervention 5%; both hospitals p < 0.001).

thumbnail
Fig 5. Antibiotic and zinc guideline adherence.

(A and B) Percentages of patients for whom antibiotics were prescribed when azithromycin was recommended at the district and sub-district hospitals, respectively. (C) Percentages of zinc prescriptions for patients under 5 years (recommended). (D) Percentages of zinc prescriptions for patients over 5 years (not recommended). *p<0.05, **p<0.01, ***p<0.001.

https://doi.org/10.1371/journal.pntd.0005290.g005

Hospital and post-discharge course.

During hospital admission, primary measures included the duration of admission, the type of discharge, and complications (Table 2). At the district hospital, there was an increase in the proportion of patients with ‘discharged on request’ (DOR) during the intervention period but the duration of admission was not significantly different. A DOR is a discharge level that is permitted but not encouraged by the physician. It is not at the legal level of ‘discharged on risk-bond’ (aka discharged against medical advice). At the sub-district hospital, the duration of admission was shorter (27.9 versus 24.6 hours, p < 0.001) during the intervention period. The post-discharge course was evaluated at 10 days for most patients (N = 596; 71%). Differences between the pre-intervention and intervention periods included a one day reduction in post-discharge days of diarrhea at the district hospital for adults (p < 0.001), and a one-day increase in post-discharge days of diarrhea at the sub-district hospital for children less than 5 years and adults (p < 0.001 for both). Although not significantly different, there was one mortality of a young child that occurred during the intervention period at the district hospital and a verbal autopsy was performed. In brief, diarrhea and emesis were resolving on post-discharge day 1, abdominal swelling of unknown cause was noted on post-discharge day 2, the family received outpatient treatment on post-discharge day 3, and the patient decompensated and died on post-discharge day 4. The death was reported to both the IEDCR and Stanford IRBs and determined to be independent of the intervention.

Discussion

The objective of this clinical pilot study was to evaluate the impact of a smartphone adaptation of the World Health Organization (WHO) diarrheal disease management guidelines. The calculator was associated with significant and positive prescription change towards guideline adherence. The findings support our hypothesis that technology-enabled decision-support tools can promote evidenced-based practice in resource-limited settings.

With respect to basic medical care, all medical facilities should have a functional and suitable weight scale. However, this minimal standard is frequently untenable in resource-limited settings like those in this study, and therefore, clinicians are forced to estimate weight. We designed the calculator to estimate pediatric weight for age by -1 z-score during the study; however, post-study analysis showed measured weights approximated best to the 5th (females) and 3rd (males) WHO percentiles for children less than 5 years of age while estimates for older age groups did not differ significantly from measured weights. These low percentiles in children under 5 years are not likely to be due to the degree of dehydration at admission alone, but suggest that admitted patients in this age range may be a particularly vulnerable subset of the population. Future larger studies are required to expand on these findings. Additional anthropomorphic measurements (e.g. mid-upper-arm circumference length; MUAC) may also provide evidence that decision-support tools must be built to leverage locally available data in order to optimize weight and other anthropomorphic estimations. In the preparatory phase, we experienced surprising resistance to the use of MUAC bands for weight estimation or for malnutrition assessment [35, 36] because of past experience with the time required to take an accurate MUAC and difficulty maintaining MUAC strip supplies.

Rehydration Calculator deployment was associated with a reduction in overall IV fluid prescriptions but an increase in the volume ordered when IV fluids were prescribed. Previous studies have shown that transitioning low and medium risk patients towards oral rehydration solution reduces the risk of complications, provides cost-savings benefit to both patients and institutions, and remains effective [37]. The weight measurements enabled the calculator to provide weight-based dosing of fluids for the pediatric patients. The adult patients were often dosed with categorical fluid volumes (e.g. 1, 2, and 3 liters) which may be more practical for staff. In the intervention period, the fluid volumes ordered versus recommended for severely dehydrated patients during the intervention were congruent. In a setting like Netrokona District that treats 5000 diarrheal patients monthly, a safe decrease in IV fluid use represents significant overall cost-savings to the medical system while empowering clinicians with the ability to increase fluids volumes to the recommended dose when IV fluids are indicated. Therefore in a venue like Netrokona with low diarrhea-associated mortality, a safe reduction in IV fluid orders may be an important advantage of decision-support tools like the Rehydration Calculator. Future studies in areas with high morbidity and mortality are required to evaluate the physiologic benefits.

The Rehydration Calculator was associated with significant and positive antibiotic class-switching to the recommended agent azithromycin. Almost all patients were described as having acute watery diarrhea with moderate or severe dehydration. For these reasons, azithromycin was recommended for almost all patients out of concern for cholera. However, 7% of samples cultured were positive for V. cholerae [all tested strains were azithromycin-sensitive]. The presumed remaining 93% of patients with non-bloody (e.g. non-Shigellosis, non-Salmonellosis, non-Campylobacter) and non-cholera diarrhea likely did not require antibiotics. Although the calculator provided objective dehydration classification, we must also encourage clinicians to accurately report ‘acute watery diarrhea’ and deploy point-of-care diagnostic surveillance for V. cholerae to decrease the prescription of antibiotics. Although there was a small increase (1 day) in the duration of post-discharge days of diarrhea at the sub-district hospital, a larger study will be required to support this finding and determine if this finding has clinical significance. More importantly, this study should spark discussion on the utility of using the phrase “acute watery diarrhea” given that casual use may have contributed to an overuse of antibiotics. A reasonable alternative, despite some culture and physiologic nuance, would be ‘rice-water stool’ because this phrase implies that the clinician highly suspects cholera.

The first goal of this study was to improve adherence to WHO-derived guidelines that are evidence-based and considered the gold standard for safety and effectiveness. Although we were only reformatting these guidelines to a smartphone medium, we monitored secondary safety related outcomes, and found only minor differences in length of stay and type of discharge. At the district hospital during the intervention, there was one post-discharge death that occurred after the majority of the diarrheal symptoms had resolved (see hospital and post-discharge course). This event generated a case fatality rate (CFR) of 0.1% for the entire study (1 out of 954 total patients including patients in the onboarding periods) and 0.3% for the district intervention which are similar to expected CFRs at similar sites [17, 19]. These preliminary data suggest that the Rehydration Calculator is a safe tool to improve guideline adherence for diarrheal disease decision-support. Future studies that include more detailed physiologic assessments may provide further avenues to assess safety and support these initial findings.

A second goal was to develop the Rehydration Calculator to be scalable and desired by medical professionals in hospitals like those in Netrokona. Frameworks are being built to guide development and evaluation of smartphone decision-support tools [38]. Scalability of the Rehydration Calculator will likely depend on the use of personal smartphones. This is feasible because the Rehydration Calculator has minimal to no associated cost, captures no personal health information, and does not rely on connectivity. Venues for Rehydration Calculator deployment are broad. For example, Bangladesh and Pakistan both have considerable diarrheal disease, including cholera, and rank among the top ten globally for cell phone penetration per capita [39]. Professionals in these countries are rapidly transitioning to affordable Android smartphones. It also remains possible for institutions to embrace the Rehydration Calculator. This would require purchasing dedicated phones, training, and technical support to maintain the phones. Future studies will require rigorous qualitative and quantitative analysis to further these goals of scalability for both individual and/or institutional deployment.

Scalability also relies on desirability. Although an anonymous user acceptance survey was not performed, feedback was obtained in a group format to assess end-user experience. Clinicians expressed that the calculator expedited both clinical assessment and reduced the time required to generate a treatment plan (3–5 minutes). Additionally, use of the calculator conferred a level of professional prestige that was attractive. Physicians were allowed to deviate from the guidelines as needed. Clinicians vocalized that deviation was important when patients expected IV fluids or when emesis made oral rehydration inappropriate. These experiences likely manifested in IV fluid orders for patients with some or no dehydration (Fig 4A). Despite the lack of user acceptance surveys, the design of the Rehydration Calculator appeared to match clinician needs. Future studies are required to explore these preliminary observations.

The findings in this study should be viewed within the context of the limitations of the study design and available data. The study site is likely generalizable to remote rural settings with a significant burden of water-borne disease. However, the remoteness also made exhaustive oversight difficult and contributed to the limitations: (i) The first limitation was a suspected under-enrollment of patients at the district hospital during the pre-intervention period compared to intervention period. This suspected under-enrollment may have contributed to the age difference between the pre-intervention and intervention periods; we therefore accounted for age in the analysis. (ii) The second limitation occurred during the intervention period at the sub-district hospital where weight measurements were not obtained due to a work-flow issue. Thus, when evaluating the accuracy of weight estimates, we excluded sub-district patients enrolled during the intervention period. (iii) There was no independent assessment of dehydration during the pre-intervention phase, medications and fluids prescribed were not confirmed to have been administered, and objective physiologic outcome measures were not reliably obtained. The data collection method also relied heavily on the physician transcribing the assessment and plan to the paper chart, and then the researcher transcribing this information to the digital data collection platform. Future studies need a tractable method to validate accuracy through this chain of events; independent assessment would provide added quality control for these steps. (iv) The study was designed as a pilot study, and although the results from the two sites were similar, there was no concurrent control group; this puts the study at risk of temporal variation during the 12-week study. (v) Exclusion criteria were unbalanced because severely malnourished patients were excluded during the intervention because of limited clinical capacity, however, this limitation actually had no impact on the results because no patients were excluded for severe malnutrition alone. (vi) The study was not designed to perform sub-analysis at the level of individual clinicians because of logistical limitations and sensitivity concerns amongst stakeholders. Despite these limitations, the core results from this study were robust and can be viewed as a critical step towards improving the quality of care in remote regions inflicted with diarrheal disease. These findings warrant further investigation with a cluster randomized controlled trial (cRCT). One approach to address physician gestalt and make the cRCT more generalizable and standardized would be to randomize the control study arm to use a paper-based version of WHO guidelines via a laminated card and the intervention arm to the WHO guidelines on the Rehydration Calculator.

In conclusion, this pilot study highlights that significant improvement towards guideline adherence can be achieved when clinicians are provided a tool such as the Rehydration Calculator. The study also demonstrates how a simple tool can standardize the assessment of dehydration status, which is a critical first step to understanding physiologic determinants of severe disease and identifying opportunities to improve clinical acumen. Upon further investigation in a cRCT, we hope that tools like the Rehydration Calculator in the hands of frontline providers will contribute to the ongoing drop in morbidity and mortality from diarrheal disease as well as usher in a new era of clinical approach and scientific understanding.

Supporting Information

S1 Movie. Rehydration calculator overview movie.

Overview movie of the version of the Rehydration Calculator deployed in the pilot study. Access at http://purl.stanford.edu/sv492bk0032.

https://doi.org/10.1371/journal.pntd.0005290.s001

(MOV)

S1 Software. Rehydration calculator software.

The most recent version of the software can be downloaded directly from the google playstore by searching for “Outbreak Responder” or directly at https://play.google.com/store/apps/developer?id=Outbreak+Responder&hl=en. Performance is best on a Samsung Android device with a 5 inch screen; compatibility does vary between companies and Android models. At the time of publication, the only software update from the version in the pilot study was a change to estimate weight for age in Bangladesh (5th percentile and 3rd percentile for females and males, respectively). Future changes will be documented in the information tabs from within the software platform.

Disclaimer: Access to the this software is provided for academic evaluation only. While the software remains under clinical study, the prototype is not intended for clinical practice. Accordingly, the software is password protected. After downloading the software, please contact the corresponding author Dr. Eric Nelson, MD PhD at eric.nelson@ufl.edu or outbreakresponder@gmail.com for the application key. Login credentials for the data collection module can also be provided upon request.

https://doi.org/10.1371/journal.pntd.0005290.s002

(DOCX)

S1 Dataset. All data except for a few restricted items to protect the identification of patients are provided in the dataset.

In the de-identified data sheet, the single mortality was removed, the study location name and measured weight for children less than 15 years was removed, the ages for the two patients older than 89 years were grouped as “greater.than.89” for age in years and “greater.than.1068” for age in months, and a calculated hospital admission duration was provided (e.g. not based on dates). The ID number was randomly generated and assigned. The de-identified data sheet for the weight of patients less than 15 years old contains only measured weights (no measured weights were available in the intervention period in Madan), estimated weights, gender and age.

https://doi.org/10.1371/journal.pntd.0005290.s003

(XLSX)

Acknowledgments

We thank the patients that participated in this study, the diligent project staff, and compassionate clinical team who made this study possible. We are grateful to F. Bill, J. Farrell, T. Winograd, G. Schoolnik, S. Luby, M. Barry, M. Baiocchi, N. Greguras, C. Morrison and Y. Maldonado for their guidance and support at Stanford University, A. Weil and J. Harris for their initial thoughts on the concept of a ‘rehydration calculator’ as well as F. Qadri, E. Gurley, and J. Clemens at the icddr,b for their support and helpful discussions. We also thank our technology partners at BeeHyv Software Solutions Pvt. Ltd. Hyderabad, India (USA and India) led by Suresh Gopalakrishnan, Founder and CEO.

Author Contributions

  1. Conceptualization: FH RLB SKa MJC SAS CS NF JRA MR EJN.
  2. Data curation: RLB FH SKh EJN.
  3. Formal analysis: RLB EJN.
  4. Funding acquisition: FH MR EJN.
  5. Investigation: FH SKh MA MWR MR EJN.
  6. Methodology: FH RLB SKa MJC SAS SM CS NF JRA MR EJN.
  7. Project administration: FH MR EJN.
  8. Resources: FH MR EJN.
  9. Software: RLB SDM DP TVK.
  10. Supervision: FH MR EJN.
  11. Validation: FH RLB SKh JRA MR EJN.
  12. Visualization: RLB DP JRA EJN.
  13. Writing – original draft: RLB EJN.
  14. Writing – review & editing: RLB FH JRA EJN.

References

  1. 1. Lozano R, Naghavi M, Foreman K, et al. Global and regional mortality from 235 causes of death for 20 age groups in 1990 and 2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet 2012; 380(9859): 2095–128. pmid:23245604
  2. 2. Liu L, Johnson HL, Cousens S, et al. Global, regional, and national causes of child mortality: an updated systematic analysis for 2010 with time trends since 2000. Lancet 2012; 379(9832): 2151–61. pmid:22579125
  3. 3. Preventing diarrhoea through better water, sanitation and hygiene: exposures and impacts in low- and middle-income countries. Exposures and impacts in low- and middle-income countries: World Health Organization, 2014.
  4. 4. Handbook: IMCI Integrated Management of Childhood Illnesses. 4th ed: World Health Organization, 2005.
  5. 5. Bryce J, Victora CG, Habicht JP, Black RE, Scherpbier RW, Advisors M-IT. Programmatic pathways to child survival: results of a multi-country evaluation of Integrated Management of Childhood Illness. Health policy and planning 2005; 20 Suppl 1: i5–i17.
  6. 6. Ahmed HM, Mitchell M, Hedt B. National implementation of Integrated Management of Childhood Illness (IMCI): policy constraints and strategies. Health policy 2010; 96(2): 128–33. pmid:20176407
  7. 7. Chopra M, Binkin NJ, Mason E, Wolfheim C. Integrated management of childhood illness: what have we learned and how can it be improved? Archives of disease in childhood 2012; 97(4): 350–4. pmid:22278806
  8. 8. Nguyen DT, Leung KK, McIntyre L, Ghali WA, Sauve R. Does integrated management of childhood illness (IMCI) training improve the skills of health workers? A systematic review and meta-analysis. PloS one 2013; 8(6): e66030. pmid:23776599
  9. 9. Pocket Book of Hospital Care for Children: guidelines for the management of common childhood illnesses. 2nd ed. Geneva, Switzerland: World Health Organization, 2013.
  10. 10. Luscombe M, Owens B. Weight estimation in resuscitation: is the current formula still valid? Archives of disease in childhood 2007; 92(5): 412–5. pmid:17213259
  11. 11. Luscombe MD, Owens BD, Burke D. Weight estimation in paediatrics: a comparison of the APLS formula and the formula 'Weight = 3(age)+7'. Emergency medicine journal: EMJ 2011; 28(7): 590–3. pmid:20659877
  12. 12. Seddon C, Lockitt L, Dhanjal S, Eisenhut M. Validation of advanced paediatric life support formulas for weight calculation in a multiethnic population. ISRN pediatrics 2012; 2012: 869634. pmid:23050162
  13. 13. Alam NH, Islam S, Sattar S, Monira S, Desjeux JF. Safety of rapid intravenous rehydration and comparative efficacy of 3 oral rehydration solutions in the treatment of severely malnourished children with dehydrating cholera. Journal of pediatric gastroenterology and nutrition 2009; 48(3): 318–27. pmid:19274788
  14. 14. Alsan M, Schoemaker L, Eggleston K, Kammili N, Kolli P, Bhattacharya J. Out-of-pocket health expenditures and antimicrobial resistance in low-income and middle-income countries: an economic analysis. The Lancet infectious diseases 2015; 15(10): 1203–10. pmid:26164481
  15. 15. Maitland K, Kiguli S, Opoka RO, et al. Mortality after fluid bolus in African children with severe infection. The New England journal of medicine 2011; 364(26): 2483–95. pmid:21615299
  16. 16. Hahn S, Kim S, Garner P. Reduced osmolarity oral rehydration solution for treating dehydration caused by acute diarrhoea in children. The Cochrane database of systematic reviews 2002; (1): CD002847. pmid:11869639
  17. 17. Kotloff KL, Nataro JP, Blackwelder WC, et al. Burden and aetiology of diarrhoeal disease in infants and young children in developing countries (the Global Enteric Multicenter Study, GEMS): a prospective, case-control study. Lancet 2013; 382(9888): 209–22. pmid:23680352
  18. 18. Haque F, et al. Cholera Outbreak in Netrokona Municipality, 2013. Health Science Bulletin 2014; 12(2).
  19. 19. Paul RC, Faruque AS, Alam M, et al. Incidence of severe diarrhoea due to Vibrio cholerae in the catchment area of six surveillance hospitals in Bangladesh. Epidemiology and infection 2015: 1–13.
  20. 20. Platts-Mills JA, Babji S, Bodhidatta L, et al. Pathogen-specific burdens of community diarrhoea in developing countries: a multisite birth cohort study (MAL-ED). The Lancet Global health 2015; 3(9): e564–75. pmid:26202075
  21. 21. Bangladesh Demographic and Health Survey 2014: https://dhsprogram.com/pubs/pdf/FR311/FR311.pdf; 2011: http://dhsprogram.com/pubs/pdf/fr265/fr265.pdf. National Institute of Population Research and Training Dhaka, Bangladesh. Accessed June 10, 2016.
  22. 22. Global Health Observatory at: http://www.who.int/gho/en/. World Health Organization (WHO). Accessed Aug. 7, 2016.
  23. 23. Clinical Growth Charts at: http://www.cdc.gov/growthcharts/clinical_charts.htm. Centers for Disease Control and Prevention (USA). Accessed Aug. 7, 2016.
  24. 24. Levine AC, Glavis-Bloom J, Modi P, et al. External validation of the DHAKA score and comparison with the current IMCI algorithm for the assessment of dehydration in children with diarrhoea: a prospective cohort study. The Lancet Global health 2016; 4(10): e744–51. pmid:27567350
  25. 25. Levine AC, Glavis-Bloom J, Modi P, et al. Empirically Derived Dehydration Scoring and Decision Tree Models for Children With Diarrhea: Assessment and Internal Validation in a Prospective Cohort Study in Dhaka, Bangladesh. Glob Health Sci Pract 2015; 3(3): 405–18. pmid:26374802
  26. 26. The treatment of diarrhoea—A manual for physicians and other senior health workers. Geneva, Switzerland, World Heath Organization. 4th rev. — 2005.
  27. 27. Das SK, Klontz EH, Azmi IJ, et al. Characteristics of Multidrug Resistant Shigella and Vibrio cholerae O1 Infections in Patients Treated at an Urban and a Rural Hospital in Bangladesh. ISRN microbiology 2013; 2013: 213915. pmid:24455398
  28. 28. Klontz EH, Das SK, Ahmed D, et al. Long-term comparison of antibiotic resistance in Vibrio cholerae O1 and Shigella species between urban and rural Bangladesh. Clinical infectious diseases: an official publication of the Infectious Diseases Society of America 2014; 58(9): e133–6.
  29. 29. Klontz KC, Singh N. Treatment of drug-resistant Shigella infections. Expert review of anti-infective therapy 2015; 13(1): 69–80. pmid:25399653
  30. 30. Nelson EJ, Nelson DS, Salam MA, Sack DA. Antibiotics for both moderate and severe cholera. The New England journal of medicine 2011; 364(1): 5–7. pmid:21142691
  31. 31. Tanphaichitra D, Bussayanond A, Christensen O. The combination of pivmecillinam and pivampicillin compared to co-trimoxazole in the treatment of enteric fever. Infection 1982; 10(2): 81–4. pmid:6284651
  32. 32. Tanphaichitra D, Srimuang S, Chiaprasittigul P, Menday P, Christensen OE. The combination of pivmecillinam and pivampicillin in the treatment of enteric fever. Infection 1984; 12(6): 381–3. pmid:6569851
  33. 33. Manual of Clinical Microbiology. 8th ed. Washington, D.C. (USA): American Society for Microbiology Press, 2003.
  34. 34. R Core Team (2015). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org/. Accessed Aug. 1, 2016.
  35. 35. Modi P, Nasrin S, Hawes M, et al. Midupper Arm Circumference Outperforms Weight-Based Measures of Nutritional Status in Children with Diarrhea. The Journal of nutrition 2015; 145(7): 1582–7. pmid:25972523
  36. 36. Mwangome MK, Fegan G, Prentice AM, Berkley JA. Are diagnostic criteria for acute malnutrition affected by hydration status in hospitalized children? A repeated measures study. Nutr J 2011; 10: 92. pmid:21910909
  37. 37. Ahmed T, Ali M, Ullah MM, et al. Mortality in severely malnourished children with diarrhoea and use of a standardised management protocol. Lancet 1999; 353(9168): 1919–22. pmid:10371570
  38. 38. Agarwal S, LeFevre AE, Lee J, et al. Guidelines for reporting of health interventions using mobile phones: mobile health (mHealth) evidence reporting and assessment (mERA) checklist. Bmj 2016; 352: i1174. pmid:26988021
  39. 39. Bangladesh Telecommunication Regulatory Commission. Mobile Phone Subscribers in Bangladesh. web access at http://enwikipediaorg/wiki/List_of_countries_by_number_of_mobile_phones_in_use. 2013. Accessed Aug. 1, 2016.