Study design and setting

This open cluster-randomized implementation research trial was conducted within the existing public health system involving 22 PHCs serving a total population of 4,76,943 in predominantly tribal areas in the southern part of Gujarat, India. Eleven PHCs were randomized to the intervention and 11 to the control arm. Each PHC served a population of approximately 22,000. Cluster randomization was chosen because this was a complex intervention to be delivered at the community level. PHCs were selected as a cluster to reduce contamination because participants of the trial included ASHAs and staff of the PHC. Each PHC had a team of two medical doctors/officers, one Auxiliary Nurse Midwife (ANM), and one male multipurpose worker for every 4,000 to 6,000 people; one female and male health supervisor and one ASHA for approximately 1,000 people; and administrative staff. An ASHA is a female village-based frontline health worker who is a native of the village, has at least eight years of formal education, spends 3–5 hours working in the community every day, and is provided incentives based on performance [24]. An ANM is a qualified health provider who provides MNCH services (immunization, ANC, delivery, etc.) at the community and facility level. The medical officer is a medical doctor who leads the PHC team [12].

The study area included six blocks of the Bharuch and Narmada districts identified as high-priority blocks by the state government because of poor health and development indicators compared to other areas and relatively lower literacy of 65% and inhabited by a tribe called Bhil or Vasava [25–28].

Twenty-six PHCs (except two PHCs where ImTeCHO was implemented already as part of an earlier pilot) were eligible to be included in these six blocks, having a 100% rural population and with at least 45% of the population consisting of scheduled tribe caste. Four PHCs that had more than 10% villages without availability of internet through mobile data were excluded, considering the study’s need for at least a few minutes every day for intervention to work.

This study was conducted by SEWA Rural, which is a nongovernmental, voluntary grassroots organization located at the border of the study area in active partnership with the Department of Health and Family Welfare, Government of Gujarat, and an information technology (IT) partner, Argusoft India Ltd. The Department of Health and Family Welfare, Government of Gujarat provided written approval and formal notifications for conducting the study within existing public health system, and one of the senior state-level officers joined the team of investigators. No major changes were made to the design of the study after its commencement.

Interventions

We used Medical Research Council (Swindon, UK) framework and user-centered design to develop and evaluate a complex health intervention in the form of a mobile-phone-technology–based job aid for the ASHAs and PHC staff with an aim to make their work easier and more effective [29]. A formative evaluation, as part of a pilot, was done in 45 villages in 2014 in a location outside the study area [22]. An online video is available that provides a short demonstration of the intervention [30].

Table 1 shows various mHealth strategies included in the application, and S1 Table shows details of service delivery in the intervention and control arms [23]. ASHAs were given Samsung Galaxy Star Pro (GT-S7262) android smart phone (cost US $86) with a postpaid data plan costing US $3 per month per ASHA. ASHAs registered all existing and new pregnant women and children under the age of two years in their villages using the ImTeCHO mobile phone application throughout the study period. Subsequently, the software prepared an entire schedule of home visits as per the national standards and sent reminders to ASHAs. Every day, ASHAs logged in to the application to review their schedule, conducted a home visit based on the task list, and completed and submitted a digital form during the home visits to close out the scheduled task. To emphasize key health and wellness messages, ASHAs showed short video clips on the mobile phone to family members during the home visits. The digital forms had checklists to screen for complications. The application had a decision support system to show probable diagnosis with risk stratification to ASHAs based on the entries made in the digital forms and a customized management plan that included tools to call emergency transport vehicles and suggested home-based remedies along with names and doses of drugs in compliance with national standards for the ASHA program. Additionally, ASHAs recorded details of the services, including immunization, delivered by the ANMs during monthly Village Health and Nutrition Days (VHNDs) in the ImTeCHO application as well. Based on the data entered by an ASHA throughout a month, her performance report was generated and shared with each ASHA on the first day of the subsequent month through the ImTeCHO application, and performance-based incentives were calculated automatically to ensure timely payment. The data transfer happened through the internet; however, data entry was possible even offline. Each medical officer was provided with a Samsung Galaxy Tab3V (cost US $165) tablet to use the ImTeCHO web interface. The medical officers and PHC staff used the web interface to track high-risk cases, view reports, and manage incentives and supplies.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Intervention strategies of ImTeCHO.

https://doi.org/10.1371/journal.pmed.1002939.t001

Complete enumeration of study population and the preintervention baseline survey in the study area was conducted from February, 2015 to July, 2015 [23]. SEWA Rural and government staff conducted training using audiovisual training tools and hands-on practice in groups from August, 2015 to January, 2016. The intervention was implemented from February, 2016 to January, 2017. The postintervention endline survey was conducted from February, 2017 to July, 2017, and its methodology was published as part of the protocol [23].

Control arm (comparator)

The government and other providers continued to provide usual health services in the control area. A refresher training (three days) and one-time supply of commodities were provided to ASHAs from the control and intervention area [23].

Participants

All health providers (ASHAs, ANMs, medical officers, and PHC support staff) belonging to PHCs in the intervention and control arms were the participants of the study. All pregnant women, neonates, and infants from study area were the participants. The mHealth facilitator and helpline (description in S1 Table) were only present in the intervention arm. The mHealth facilitators (one per 50 ASHAs) and one counselor at helpline were part of the project team from SEWA Rural [23].

Management

The project team stationed at the headquarters included an implementation coordinator and project associate who supervised the team of mHFs and helpline counselor. It intervened in cases of emergencies, organized a training at the beginning of the trial, paid additional token monetary incentives to intervention ASHAs (ranging from US $4 to $11 monthly per ASHA based on performance), managed SIM cards, and coordinated with partner organizations.

The project team did not conduct community mobilization, regular field-level supervision, or separate meetings with ASHAs other than participating in regular monthly PHC meetings. In addition, they also did not deliver any health services at the community level or provide any other kind of support to ASHAs not listed in S1 Table or ongoing provision of commodities after initial training.

The district- and state-level government officers actively participated in proposal development workshops, provided feedback about the software, participated in regular review meetings, and led implementation jointly with project management team. The IT partner carried out regular software maintenance and ensured timely resolution of any related issues on an ongoing basis.

Outcomes

The primary and secondary outcomes of interest were coverage of various health services to be delivered or facilitated by ASHAs, which were expected to change as a result of ImTeCHO intervention, along with process indicators to measure fidelity to the intervention. The two primary outcomes of interest were (i) the proportion of neonates/mothers visited at home by ASHA at least two times within one week after delivery, and (ii) modified ASHA-centric composite coverage index (MACCI). The details of MACCI have been published elsewhere [23]. See S2 Text for the definition and rationale for the choice of primary outcomes of interests.

The ImTeCHO program data were used to obtain results of the process indicators to assess adherence to the intervention. The definitions of process indicators and secondary outcomes of interest were included in the published protocol [23]. There were no changes in trial outcomes after its commencement.

Recruitment of respondents, and measurement of outcomes

An endline household survey was conducted from February, 2017 to July, 2017 to measure the primary as well as secondary outcomes of interest. The household survey tools used standard methodology for the district-level health and facility survey (DLFHS-4) [31]. There were two kinds of respondents.

1. For maternal and newborn health outcomes (respondent type A): All women who were natives of the study villages and were mothers of a one- to four-month–old infant at the time of survey were eligible. Women whose infants died before the survey were excluded.
2. For child health outcomes (respondent type B): All women who were natives of study villages and were mothers of a six- to eight-month–old infant at the time of survey were considered eligible. Women whose infants died before the survey were excluded.

All pregnancies, their outcomes, and infant and maternal deaths from the entire study area were counted prospectively from 2016 onwards as part of ongoing pregnancy and mortality surveillance. The information from pregnancy registration and mortality surveillance system was used to identify and enroll type A and type B respondents for the endline survey.

The data were entered into an in-house–developed data entry software, different from the ImTeCHO application, through smart phones that had inbuilt provisions to avoid illogical data entries and missing data. A data administrator checked for any inconsistencies, which were clarified with data collectors and corrected. The endline survey was conducted jointly by an independent evaluation team comprising trained data collectors, who worked separately and were not involved in the implementation activities, and an independent research organization, Population Research Centre (PRC), Vadodara, Gujarat. PRC validated data on a randomly selected 3% of respondents through household survey using a truncated questionnaire.

Sample size

We calculated sample size for both the primary outcomes of interest and used the larger one. Eleven PHCs each in the intervention and control clusters were included to address both questions [23]. Based on the results of an evaluation of an earlier pilot conducted prior to the trial, we assumed that ASHAs would visit 46% of neonates/mothers at least twice at home within first week of delivery and MACCI would be 36% in control area [32]. We assumed there would be 25 ASHAs associated with one PHC. We assumed intraclass correlation (ICC) to be 0.02 in light of no existing information available regarding ICC. We estimated that there might be loss of one cluster per arm and three ASHAs per PHC over the course of the study period. For a power of 80% and 5% two-sided significance level, we estimated that six PHCs/clusters (150 ASHAs) per arm were required to detect 20% absolute improvement in the proportion of neonates/mothers who received at least two postnatal home visits within first week of delivery by an ASHA in the intervention arm compared to the control arm at endline survey. Similarly, we estimated that 11 PHCs/clusters (275 ASHAs) per arm were required for detecting 15% absolute improvement in MACCI in the intervention arm compared to the control arm at endline survey.

Randomization

An independent senior statistician not involved in study implementation carried out randomization and assigned 11 PHCs to the intervention and 11 to the control arm. MACCI was calculated for each PHC after the baseline survey; the mean value of MACCI was calculated, which was then used as a cutoff to allocate each PHC into one of the two strata. The PHCs in each of the two strata were randomly allocated to the intervention and control groups in a 1:1 ratio to ensure that MACCI was similar at the baseline for the intervention and control arms using the software nQuery. Blinding was not possible because this was a community-based intervention. All pregnant women and infants identified through complete enumeration and identification of new pregnant women along with their newborn babies throughout the study period in the study area were eligible to participate.

Ethical review

The Multi-institutional Ethics Committee (Mumbai), Institutional Ethics Committee of SEWA Rural, and Ethics Review Committee of WHO approved the study. All respondents provided written informed consent to trained data collectors prior to the household survey. Every respondent who participated in the endline survey was assigned a unique identification number, and personally identifiable information was removed. Standard security and data encryption practices were used by the ImTeCHO application system for accepting, transmitting, processing, and storing data.

The results were reported in accordance with CONSORT statement for reporting cluster-randomized trials (see S2 Table).

Statistical methods

Primary analysis was intention to treat (ITT) and secondary analysis was per protocol (PP) at the ASHA level. A Generalized Estimating Equation (GEE) was used to account for clustering and to adjust for maternal age, caste, parity, poverty status, and education. For ITT analysis, all enrolled respondents were included in the analysis irrespective of whether they received the treatment/intervention or not. However, there is a common local custom for a large proportion of pregnant women to move out of their own homes and spend the last trimester and first few weeks after delivery at their maternal village, which might be out of the study cluster and might not have been completely exposed to the treatment arm. Therefore, PP analysis was done for indicators related to antenatal and postnatal services (respondent type A).

Two-level random-effects regression models were applied for each of the primary and secondary outcomes. For continuous and categorical outcomes, cluster-level means and proportion were used, respectively. In the case of proportion, the mean of proportion by cluster was calculated and reported. Analysis was done at the ASHA level. The differences with effect size and p-value for each outcome were also reported. The effect size was adjusted for maternal age, caste, parity, below poverty line (BPL) card status, and women’s education level. The denominator for outcomes related to pregnancy and postnatal services included respondents of type A. The denominator for outcomes related to postneonatal services included respondents of type B. STATA 13.0 was used for data analysis.

Trial registration

This trial was registered with the Clinical Trial Registry of India (www.ctri.nic.in, registration number CTRI/2015/06/005847, date of registration 3 June, 2015).

Data safety monitoring board

The data safety and monitoring board, comprising one scientist, one ethicist, one community representative, and one statistician, monitored the trial during annual meetings.

Contribution of study to existing literature

To our knowledge, this is one of the first randomized controlled trials of its kind that assessed effectiveness of multiple mHealth strategies implemented in a low- and middle-income country (LMIC) through frontline health workers in an existing public health system throughout the continuum of care to improve a wide range of MNCH outcomes among hard-to-reach populations using a robust research methodology. The ImTeCHO intervention targeted the entire scope of potential users of mHealth-based solutions, including health providers, beneficiaries, and health system managers. Published data have shown improvement in outcomes related to only one phase of RMNCH life cycle or use of only one mHealth strategy (most commonly Short Message Service [SMS]-based) or in controlled settings outside a government-supported public health system and targeted either beneficiary or health providers [10,33]. A study in rural Ethiopia, which did not involve random allocation of treatment, concluded that mHealth strategies, in the form of reminders through SMS to frontline health workers, resulted in an increase in the coverage of antenatal visits and facility delivery [34]. A randomized trial in Tanzania, conducted in a controlled setting by a nongovernmental organization, reported that mHealth strategies implemented during the antenatal period resulted in an increase in facility delivery [8]. Another randomized trial conducted in Bihar, India using similar mHealth strategies reported significant improvement in the coverage of a wide range of MNCH services delivered by frontline health workers; however, there were intensive efforts to strengthen the health system in addition to the deployment of the mHealth intervention [35]. Systematic reviews have stated a low quality of evidence and advocated randomized trials using robust research methodologies in uncontrolled settings [10,36–38]. Randomized trials in Ethiopia and Democratic Republic of Congo reported that a safe delivery mobile phone application as a training tool for health workers resulted in significant improvement in health workers’ knowledge and skills about neonatal resuscitation [39, 40].

ASHAs and PHC staff faced some challenges to adopt ImTeCHO on a consistent basis: first, to balance competing priorities in the context of various other programs. The uptake of ImTeCHO web application among PHC staff was less than expected, especially in light of ImTeCHO not being mainstreamed in the public health system in the state at the time of study and other competing priority public health programs; second, to deal with the perception of increase in workload due to introduction of ImTeCHO; third, to overcome technology problems involving internet coverage, hardware issues with mobile phones, and resolving bugs in the software in a timely manner. ASHAs and PHC staff from eight intervention PHCs did not use ImTeCHO’s incentive management system because they were concerned that digitization of incentive calculation might take away the flexibility of a paper-based system and might reduce the amount of incentive they might get. We established systems and processes to overcome these challenges by ensuring health system preparedness, effective change management, training, supportive supervision, motivation, and timely technology support. In addition to the existing components of ImTeCHO, the use of automated voice calls for targeted client communication, verification of data quality, digitization of birth and death registration, and expansion of the user base to involve other cadres of frontline health workers may help us further improve coverage of health services [41].

Study strengths and limitations

Our study had a few strengths. First, we aligned programmatic objectives (i.e., increasing coverage of MNCH services) with the personal priorities of ASHAs (e.g., calculation of performance-based incentives using ImTeCHO for timely disbursement). Second, we took health workers’ legitimate concerns into consideration about the reasons for their suboptimal performance and used technology as a job aid (not job add) to make their tasks more efficient, effective, and easy. These approaches resulted in a higher uptake of technology-based interventions and improved coverage of MNCH services. Finally, we identified critical operational requirements that improved adherence to the intervention; this included supportive supervision, timely resolution of technology problems, and change management, including monetary and/or nonmonetary incentives depending on the context. Such operational requirements could be an integral part for an mHealth program to succeed at scale.

Our study also had some potential limitations. First, the outcomes were reported by the mothers, and there was a risk of recall bias. The mean number of days between date of birth and date of questionnaires completed during the endline survey was 116 days and 112 days for the intervention and control areas, respectively. However, standard data collection methodologies were used, and the methodology was the same across the study arms to negate any problems with recall. There was high concordance (89%) between the data collected by the data collectors and the team of independent quality assessors. Second, there was a higher proportion of women from the scheduled tribe and from the lowest standard of living index in the intervention arm. These differences were adjusted in the final model. Finally, the total duration of the intervention was 12 months, which might be considered short. The choice of duration of the intervention was affected by experiences of the earlier pilot, logistic concerns related to expected events in future such as upcoming state elections making survey activities difficult, and possible reorganization of PHCs resulting in reorganization of clusters. Some of the indicators showed improvement over the course of study period even in the control clusters (at least two home visits by ASHA during first week of delivery improved from 14.0% at baseline to 22.9% at endline). This might indicate overall strengthening of ASHA performance, and additional benefits from the intervention could be viewed in this context.

Conclusions

In conclusion, our data show that mHealth strategies can improve the coverage of proven MNCH services, especially in hard-to-reach populations, if there is adequate supportive supervision, change management, and ongoing technology assistance to ensure satisfactory adherence to the intervention. Encouraged by the results of the study and the demonstrated feasibility in the existing public health system, the Gujarat state government is now leading the scale-up of the modified ImTeCHO intervention in the entire state. The scope of the intervention is being expanded to include other health domains and health cadres to prepare one integrated health IT platform. Many of the activities conducted by the project team are being taken over by another nonprofit organization, GVK EMRI, which is currently providing free emergency ambulance services across the state as part of a public–private partnership, thus improving the chances of a successful scale-up while minimizing the dilution of effectiveness of the intervention [42]. We recommend that it would be very useful to evaluate the scale-up after a longer period of implementation.