Figures
Abstract
Introduction
The current surge of mobile phone use in many African countries creates the opportunity to provide caregivers with limited access to the health care system with vital health recommendations. At the same time such communication system can be utilised to collect tempero-spatial data on disease symptoms.
Objective
We assessed the feasibility of an mHealth system among caregivers with children under-five years, designed as a health information and surveillance tool in a rural district of Ghana.
Methods
A mobile phone-based electronic health information and surveillance system was piloted from February to December 2015. Toll-free numbers were provided to 1446 caregivers, which they could call to receive health advice in case their children showed disease symptoms. The system was setup to evaluate the illness of a sick child. Symptoms reported via the system were evaluated and compared to clinician’s report after follow-up. Cogency of the reported symptoms was assessed using Cohen’s kappa coefficient.
Results
A total of 169 children with disease symptoms were identified based on phone calls from caregivers. The predominant reported symptoms were fever (64%; n = 108), cough (55%; n = 93) and diarrhoea (33%; n = 55). Temporal pattern of symptomatic cases revealed a peak saturation in the month of September, with fever registering the highest number of symptoms observed. Reported symptoms and clinician’s report revealed a very good agreement for fever (95%, kappa = 0.89); good for diarrhoea (87%, kappa = 0.73) and moderate for cough (76%, kappa = 0.49).
Conclusion
This pilot concept, has demonstrated the practicality of using mobile phones for assessing childhood disease symptoms and encouraging caregivers to seek early treatment for their children if needed. The strategy to use mobile phones in disease surveillance and treatment support is a promising strategy especially for areas with limited access to the health care system.
Citation: Mohammed A, Franke K, Boakye Okyere P, Brinkel J, Bonačić Marinovic A, Kreuels B, et al. (2018) Feasibility of Electronic Health Information and Surveillance System (eHISS) for disease symptom monitoring: A case of rural Ghana. PLoS ONE 13(5): e0197756. https://doi.org/10.1371/journal.pone.0197756
Editor: Delmiro Fernandez-Reyes, University College London, UNITED KINGDOM
Received: September 20, 2017; Accepted: May 8, 2018; Published: May 24, 2018
Copyright: © 2018 Mohammed et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data underlying the study is available from the Inter-university Consortium for Political and Social Research (ICPSR) database (http://doi.org/10.3886/E103280V1).
Funding: We received financial support from German Federal Ministry of Education and Research (https://www.bmbf.de/en/index.html), with Grant number 01DG13019A. The funders did not play any role in the design of the study, collection, management, analysis, or interpretation of the data; or preparation of the manuscript. JM received the funding for the study.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Worldwide, 5.9 million deaths (43 per 1,000 live births) in children under-five years are due to preventable diseases such as malaria, pneumonia and diarrhoea, with the African region alone accounting for 47% of these deaths [1]. In Ghana, progress in reducing child mortality has not been sufficient, decreasing from 80 per 1,000 live births in 2010 to 60 per 1,000 live births [2], which was beyond the unachievable Millennium Development Goal target of 40 deaths per 1,000 live births and still remains a far cry from the Newborn Strategy and Action Plan (2014–18) goal of reducing newborn deaths to 21 per 1,000 live births [3]. Malaria remain a major public health concern, with 36% of children (6–59 months) affected in 2014 [2].
However, early detection and effective treatment of malaria is known to save lives by seeking early treatment [4]. These efforts are highly constrained by weak health systems in rural areas, as well as factors such as travel-distance to health facilities, inadequate drug stocks at health facilities and lack of money to pay for services particularly in sub-Saharan African countries [5]. Failure to recognize signs and symptoms of childhood illness by caregivers have been attributed to delay in seeking care [6]. In an effort to reduce the burden of disease and provide appropriate care, Ghana adopted the Integrated Management of Childhood Illness (IMCI) policy. The IMCI guidelines require that sick children who show general danger signs such as inability to feed, incessant vomiting, unconsciousness and convulsions, be managed quickly and referred immediately after pre-referral treatment [7].
In the meantime, during the last two decades, there has been a surge of over 27 million mobile phone users in Ghana [8]. Mobile health (mHealth), the use of mobile electronic devices for public health purposes, holds the potential to overcome several challenges that plague the healthcare system. It has been used extensively in many health programmes to improve interaction between patients and the health system [9]. mHealth has been exploited in health programmes including chronic disease management [10], medication compliance [11], substance abuse [12], mental illness [13] and post-discharge monitoring of patients [14]. A consortium of medical, public health and mHealth experts from Ghana, Netherlands and Germany, in collaboration with the Ghana Health Service, piloted an Electronic Health Information and Surveillance System (eHISS), to evaluate the feasibility of using a mobile phone-based technology for monitoring symptoms of childhood illnesses and to encourage caregivers to seek early and appropriate healthcare. Based on the IMCI guidelines [7] the system is able to interact with caregivers via an Interactive Voice Response (IVR) algorithm and provide first treatment recommendations. The IVR is an automated mobile phone-based technology deigned to allow a caller to dial into a computer system over a telephone line and access a service running on the computer. The caller may then interact with and receive voice information from the service. So far, little is known about the feasibility and utilisation of the IVR-based application among caregivers of children under-five years of age in a malaria endemic country like Ghana. This paper therefore aims to assess the feasibility of a symptom-oriented mHealth system among caregivers of children under-five in a rural district of Ghana.
Materials and methods
Study area and participants
This pilot study was conducted from February 2015 to December 2015 in the Agogo sub-district of the Asante Akim North Municipal District, located in the Ashanti Region of Ghana. It has a population of approximately 45,870, who mainly engage in small-scale farming. The Agogo Presbyterian Hospital (APH), a local district hospital, provides health services to the entire population and also serves as a referral centre for other hospitals in the region. According to the district health annual report, the commonest childhood disease symptoms recorded among children under-five in the study area were fever, diarrhoea and cough. The eHISS system was piloted within the hospital catchment area. Caregivers of children under-five who could fluently speak the local language (Twi) were recruited to partake in the study.
mHealth application
Developed and operated by a Ghana-based private company called Viamo, this mHealth system was designed to assess symptoms of sick children and provide health advice to caregivers through mobile phones. Caregivers of sick children were provided with tailor-made health information after answering several sequential questions through an IVR. The automated system was based on a clinical algorithm (Fig 1), so that after answering a set of questions, data on current disease symptoms and geo-location were gathered. The clinical algorithm was based on the decision trees of the IMCI [7], and local assessment of clinician’s report and recommendation. The clinical algorithm and its translation was developed over a period of 2 years by experts consisting of clinicians, public health experts, epidemiologists/biostatisticians and communication researchers. After placing a call, the system automatically ends it and calls back. After a brief introduction, the system provided instructions for the caregivers to give responses by mostly pressing ‘1’ for a ‘yes’ response or ‘2’ for a “no” response on their mobile phones. The first set of questions were asked to assess whether the child had any ‘danger sign’ such as inability to breastfeed or drink or unconscious and had convulsions, taking into consideration the age of the child. The next set of questions were asked to identify the common symptoms of childhood illness including fever, cough and diarrhoea. Further questions were only asked to assess the severity of the child’s condition if the caregiver responded ‘yes’ to the previous question. A caregiver was required to answer all questions asked by the system, so that all symptoms could be captured, and at the end provided with advice after symptom assessment. This advice was based on the level of severity of a symptom or combinations of symptoms—where ‘A’ denoted severe, ‘B’ moderate, ‘C’ mild and ‘O’ none (see S1 Table for detail description of each category). Where a child fell in more than one category after answering all questions, priority was given to the highest category, and the appropriate advice given. As indicated in the IMCI guideline, all danger signs such as inability to breastfeed or drink, unconscious or convulsions and other combinations of symptoms were all classified as severe by the system. All calls that fell in category ‘A’ were deemed as requiring emergency treatment, and consequently, caregivers were advised by the system in their local dialect (‘Twi’) to take their sick child to the nearest health facility immediately. Similarly, symptoms that fell in the ‘B’ category were deemed to require causal treatment, hence advised by the system to take the sick child to the nearest hospital within 24 hours; and symptoms within the ‘C’ category advised to provide home treatment and take child to the nearest health facility if symptoms persist. The home treatment included the provision of Oral Rehydration Salt (ORS) or paracetamol syrup, depending on the symptom reported. On the other hand, symptom reports that fell in the ‘O’ category meant the system could not assess the underlying cause of illness. In such situations, caregivers were advised to take the sick child to the nearest health facility for further assessment in order to ascertain the condition of the child.
The arrows show the flow of the questions, with all questions asked irrespective of the response provided in the previous question. The system provides advice at the end of the call based on the combination of symptoms indicated.
Individuals who access the system for health advice are indirectly reporting disease symptoms, which are recorded by the system. These data, along with location and time information are automatically stored and available for immediate analyses. Due to the data protection regulations by the National Communications Authority the phone companies did not provide location data from phone calls. Thus, the study was conducted with registered users only, who provided their phone numbers and coordinates of their current residence.
Sample size
We estimated the sample size with an assumed ownership of mobile phone by caregivers of 50% and 5% case report of fever symptoms within the study population. Therefore, with a default alpha of 0.05 and a power of 80% and taking cognisance of non-response rate of approximately 20%, we arrived at a sample of 1446. This would enable us to oversample and detect at precision level of 5%.
Data collection procedures
This study had two major phases, namely (a) a three months testing phase and (b) an eight months implementation phase. During the testing phase, only caregivers who personally owned a mobile phone were registered (details such as phone number, name and location recorded into the system) and this was limited to caregivers who were attending Child Welfare Clinic (CWC) located in the communities. A total of 433 caregivers were recruited. The objective of the testing phase was to test the usability of the system by registered caregivers, system’s ability to provide appropriate advice and assess caregivers’ ability to correctly follow the advice of the system. In the implementation phase, a total of 1013 caregivers were registered.
Recruitment of study participants
Caregivers were recruited from 36 different communities with a functional mobile network located within the APH catchment area. Caregivers received a toll-free phone number and instructions on how and in which situation to access the system using their mobile phones. They were given the opportunity to use their phone or that of a relative. All registered caregivers at the point of recruitment were briefly trained by dummy demonstration about the use of the system. Using a structured questionnaire, socio-demographic characteristics including age, gender, marital status, education, religion, ethnicity and mobile phone ownership, as well as mobile phone numbers, experience with the use of IVR, name of community and Global Positioning System (GPS) coordinates of the communities were recorded at the point of recruitment.
After seeking permission from the administrative head and the health authority of the study area, eHISS was actively advertised for three months (twice in a week) during the implementation phase with the assistance of community health volunteers through public information centres and social gatherings.
Follow-up procedures
During the study period, all system-recorded calls were followed up within 2 days in order to evaluate the health status of the sick children and the action taken by caregivers. Records of children who were taken to the nearest hospital were retrieved to identify symptoms recorded by the clinicians. Intermittent contact was made with District Health Management Team, a team responsible for monitoring disease occurrence within communities and providing basic medical treatment. They were given weekly reports of emerging symptoms of childhood illness development following caregiver-IVR interaction including the spatial development of reported symptoms generated using google map (Fig 2).
Data management and statistical analysis
Data recorded during follow up were entered in Access database. Similarly, data obtained from the system were extracted in csv format and exported to STATA 12 (STATA Corp., College Station, Texas, United States) for analysis. Percentage agreement between data obtained from the IVR system and the clinicians was calculated for reported symptoms such as fever, cough and diarrhoea. This was done for 28 caregivers whose IVR reports were sufficient and medical record of reported sick child available. Cohen’s Kappa coefficient (k) was calculated to test for the percentage agreement and the corresponding 95% confidence interval. The values of p < 0.05 were considered significant. Kappa values were interpreted according to Altman’s guidelines [15]; values <0.20 = poor agreement, 0.21–0.40 = fair, 0.41–0.60 = moderate, 0.61–0.80 = good and 0.81–1.00 = very good.
Ethical considerations
Ethical clearance for the study was obtained from the Kwame Nkrumah University of Science and Technology (KNUST)/Komfo Anokye Teaching Hospital (KATH) Committee on Human Research, Publications and Ethics (CHRPE/AP/278/14). In addition, clearance and permission was obtained from the Director of Health Administration of the Asante Akim North Municipal District. Permission from the respective leaders of the communities was also sought. Written informed consent was obtained from the caregivers and assent from their children.
Results
Socio-demographic characteristics of registered respondents
The mean age of registered caregivers was 30 (SD = .20) years, 97% (n = 1401) were female, 65.2% (n = 942) of them were married. Furthermore, 49% (n = 708) had Junior High School education, 85% (n = 1242) were Christians, 68% (n = 980) belonged to the ethic group of Akans, 70% (n = 1008) owned a mobile phone, and 73% (n = 1047) had no prior experience with IVR applications (Table 1).
Of the 448 calls received from both registered and unregistered users, 229 calls were excluded mainly because the callers did not specify any symptom. Callers who reported at least one symptom were included in the analysis. All multiple calls were counted as one and added to the unique-call total, for both registered and unregistered callers, adding up to an overall 169 unique calls (Fig 3).
Based on the symptoms indicated by the caregiver, treatment recommendation provided by the system predominantly fell within the emergency category ‘A’ (55%, n = 93). Over forty percent (41%, n = 69) who fell within category ‘B’ and ‘C’ were advised to take their children to the nearest health facility after providing some level of care. Less than 5% of the calls which fell in the ‘O’ category could not be provided with useful advice, thus asked to seek advice for the unknown symptom. Of the 28 caregivers who were successfully followed-up by phone, 68% (n = 19) reported taking their children to the hospital and 11% (n = 3) providing home treatment as recommended by the system. One-fifth (21%, n = 6) however, did not take any action.
In total, 49% completed incoming calls and 51% unsuccessful calls were recorded by the system. The majority (85%, n = 144) of the caregivers selected Twi as their preferred language. The mean duration of time spent by caregivers interacting with the system was 4 minutes (SD = 1.9). Assessment of similarity of symptoms reported by caregivers and clinician’s report revealed a very good agreement for fever (95%, kappa = .89, P < 0.001) and good for diarrhoea (87%, kappa = .73, P < 0.01) and moderate for cough (76%, kappa = .49, P < 0.05) (Table 2).
The total number of symptoms accrued was estimated for communities involved in the study. The highest numbers of symptoms were recorded in communities such as Bontodiase (n = 151), Newtown (n = 138), Oboase/Mponisa (n = 205) and Gyedim (n = 97), with (n = 9) in Kusibo. The most reported symptoms were fever (64%; n = 108) and cough (55%; n = 93) while the least symptom was diarrhoea (33%; n = 55). Of the 169 children who were reported ill, more than half (52%; n = 88) were less than 1 year old, while less than 10% were 2–4 years (9%; n = 15) and at least 5 years old (8%; n = 14) as shown in Fig 4. Majority (86%, n = 145) of the caregivers also reported at least one danger sign.
Fig 5 shows the distribution of the 169 symptomatic cases identified throughout the study period, with the highest number of symptoms reported in the month of September. Fever was the predominant symptom reported in the peak period, with 27 cases observed. The temporal pattern of symptomatic cases identified among registered and unregistered users of the system.
Temporal pattern of symptoms identified by the system based on phone calls from caregivers. Symptom saturation was observed in May-October.
Discussion
Our study illustrates how a mobile phone-based electronic health information and surveillance system can be deployed to provide useful health advice, while at the same time providing information on potential disease clusters in a rural district. It was set up to test the potential to detect outbreaks by analysing spatiotemporal data on disease symptoms reported by service users.
Studies involving the use of IVR have reported varying access rates—ranging from 12.5% to 63.4% [16–18]. In the current study, the 11 month period yielded 169 calls with unique disease symptom reports of sick children from both registered and unregistered callers. Considering the number of registered caregivers and the universal access to mobile phone, the number of calls recorded by the system was generally low. This could be due to a number of reasons, including children not falling ill, lack of motivation and trust in a computer-based system, first-time use and lack of experience with the IVR system. Although the education level of the caregivers was generally low, this could not have accounted for the low report rate as some studies have found no association between IVR use and education level [18,19]. Nonetheless, a larger proportion of the calls recorded from the system came from unregistered community members who received no prior instructions on how to use the system. This clearly suggests that employing good marketing and advertisement strategy (e.g, at social gatherings or community information centres) for raising awareness could increase the number of people utilizing the system. As expected, most users preferred to use the system with the local language (Twi), which may have been easier for them to understand and report symptoms effectively as observed in an earlier study conducted by Piette and colleagues [19].
Other studies have documented the usefulness of using Short Message Service (SMS) to increase attendance to health care services [20] and adherence [21,22]. These studies have shown that, SMS reminders increase the likelihood of clinic appointments for general illnesses [23,24]. In our study, however, the system was designed to give advice depending on the severity of the symptom whether to provide home treatment or seek immediate treatment at the nearest health facility for caregivers who reported at least one symptom. This could help lay people care for their sick children and to triage them to the healthcare system, potentially reducing the likelihood relying on self-treatment and drug vendors which could result in misdiagnosis and incorrect choice of drugs [25] and subsequently, development of adverse drug reactions and dangerous drug interactions [26]. As one of the key measure of performance, the capacity of the system to encourage caregivers to seek early treatment was assessed. Although during the study period not all actions taken by all caregivers who accessed the system could be ascertained, caregivers who were successfully followed up by study team revealed that majority of them took their sick children to the nearest health facility for appropriate treatment. This suggests that the system could be deployed to tackle the current existing problem of about 44% of children who experience fever and not taken to a health facility for appropriate treatment in Ghana [2]. Caregivers who also reported at least one danger sign were recommended to seek immediate medical treatment at the appropriate source. This also suggests that, caregivers who may have been oblivious of the extent of the illness of the child and may not have otherwise acknowledged them as danger signs were accurately advised by the system to seek immediate and appropriate care at the nearest hospital, potentially reducing delayed treatment. In the future, the mobile phone-based system could be linked to a medical personnel at post in order to encourage timely reporting of symptoms. As a potential behaviour change tool, the system could also be used primarily to improve healthcare utilization and maternal and child health services such as antenatal care and immunization coverage [27].
Using a web-based mobile phone technology, Meankaew and others (2010) implemented a disease and treatment monitoring of malaria (DTMM) module in Thailand to assess its effectiveness. The follow-up rates were found to be fairly high, with the system capturing 534 patients with malaria [28]. Though similar to the study conducted in Thailand, our study solely relied on the unilateral and self-reported symptoms by caregivers, with system-user interaction monitored in real time. Temporal pattern of symptoms detected by the system employed in our study showed that symptoms were reported between May and October. Fever was found to be the most frequent symptom observed, steadily increasing from May to September. This finding is consistent with the temporal pattern of malaria prevalence within the study area, where malaria is endemic and known to be the number one cause of fever [29] during the month of June to August [30].
As an important system component, the validity of the IVR system reports provided by the caregivers were validated using Cohen’s Kappa coefficient. Although the study revealed varying concordance for all reported symptoms, percentage agreement between the IVR report and the medical record ranged from very good (0.89) to moderate agreement (0.48). This is consistent with a study conducted to validate the internal consistency of IVR and internet self-reports with clinician’s report among addiction patients using an Addiction Severity Index (ASI) [31] and earlier reliability and validity studies of IVR system [19,32]. This demonstrates the feasibility and reliability of using IVR for reporting symptoms in near-real time which also provides cues for diagnosis. However, additional work is needed to identify factors that affect IVR reporting quality in order to increase response rate for reported symptoms. The question-completion rate of 49% revealed in our study was generally low compared with about 90% completion rate achieved in the study involving addiction clients [31]. The difference in the completion rate is most likely because of the study population, as substance abuse persons may feel less judged conveying their substance use to a computer than a person. Comparatively, a major motivation factor for caregivers to access an IVR could be compelled by the child’s condition.
Limitations
Although the eHISS pilot study provided data for monitoring symptoms of childhood illnesses, a number of challenges were encountered during its implementation. Therefore, some limitations should be considered when interpreting the results of the study. Physical barriers such as unstable mobile network in some communities, low ownership of mobile phones, ineffective marketing strategy and hard-to-reach communities could have accounted for the low number of calls received during the study period. However, a major challenge was data quality- incompleteness. This implies that additional efforts are required to completely clean the data, adding to the time taken data extracted from the system to be ready for use. Additionally, all incoming calls that could not specify at least one symptom were excluded from the case analysis, indicating loss of potential cases. The absence of Geographic Information Systems (GIS) data also made it impractical to perform cluster analysis of diseases symptoms. Also, all reported symptoms could not be validated to ascertain whether caregivers were reporting “true” symptoms of their children.
Conclusion
As a pilot concept, this study has demonstrated the practicality of using mobile phones for monitoring childhood disease symptoms and encouraging caregivers to seek appropriate and early treatment for their children. However, further studies are needed to assess the impact of the system on health outcomes and its capacity to provide spatial monitoring of infectious diseases such as cholera, HIV, tuberculosis etc. as a surveillance tool for disease control efforts.
Supporting information
S1 Table. Level of severity of reported illness and recommendation provided by the system.
https://doi.org/10.1371/journal.pone.0197756.s001
(DOCX)
Acknowledgments
We would like to acknowledge Voto Mobile (now called Viamo) for providing the technological capacity of developing IMCI–based algorithm server and staff of the Agogo District Health Administration.
References
- 1.
Boerma T, Mathers C, AbouZahr C, Chatterji S, Hogan D, Stevens G et al. Health in 2015: From MDGs Millennium Development Goals to SDGs Sustainable Development Goals. 2015 [cited 2017 Feb 21]; http://www.popline.org/node/641266.
- 2.
GSS, GHS, ICF International. Ghana Demographic and Health Survey 2014. Ghana Stat Serv GSS Ghana Health Serv GHS ICF Int Rockv Md USA [Internet]. 2015 [cited 2017 Feb 21]; http://microdata.worldbank.org/index.php/catalog/2373
- 3.
Ministry of Health G. Ghana national newborn health strategy and action plan 2014–2018 [Internet]. 2014 [cited 2017 Feb 20]. http://www.gh.undp.org/content/dam/ghana/docs/Doc/Inclgro/UNDP_GH_IG_2010MDGreport_18102013.pdf
- 4.
Breman JG, Mills A, Snow RW, Mulligan J-A, Lengeler C, Mendis K, et al. Conquering Malaria. In: Jamison DT, Breman JG, Measham AR, Alleyne G, Claeson M, Evans DB, et al., editors. Disease Control Priorities in Developing Countries [Internet]. 2nd ed. Washington (DC): World Bank; 2006.
- 5. Williams HAHA, Jones COH. A critical review of behavioral issues related to malaria control in sub-Saharan Africa: what contributions have social scientists made? Soc Sci Med 1982. 2004 Aug;59(3):501–23. pmid:15144761
- 6. Chompolola A, Macwan’gi M. Caretakers’ Experiences with Sick Children in Luapula Province: Implications for Child Survival Interventions in Zambia. Med J Zambia 2008; 35(2).
- 7.
WHO. Integrated Management of Childhood Illness Chart Booklet. 1–76 [Internet]. WHO. 2014 [Accessed 2017 Jan 23]. http://www.who.int/maternal_child_adolescent/documents/IMCI_chartbooklet/en/
- 8.
National Communications Authority. Telecom Data Subscription [Internet]. 2016 [Accessed 2017 Feb 23]. http://www.nca.org.gh/industry-data-2/market-share-statistics-2/data-3/
- 9. Hurt K, Walker RJ, Campbell JA, Egede LE. mHealth interventions in low and middle-income countries: a systematic review. Glob J Health Sci. 2016;8(9):183. pmid:27157176
- 10. Goldman RE, Sanchez-Hernandez M, Ross-Degnan D, Piette JD, Trinacty CM, Simon SR. Developing an automated speech-recognition telephone diabetes intervention. Int J Qual Health Care J Int Soc Qual Health Care. 2008 Aug;20(4):264–70.
- 11. Oake N, van Walraven C, Rodger MA, Forster AJ. Effect of an interactive voice response system on oral anticoagulant management. CMAJ Can Med Assoc J J Assoc Medicale Can. 2009 Apr 28;180(9):927–33.
- 12. Kaminer Y, Litt MD, Burke RH, Burleson JA. An interactive voice response (IVR) system for adolescents with alcohol use disorders: a pilot study. Am J Addict. 2006;15 Suppl 1:122–5.
- 13. Greist JH, Osgood-Hynes DJ, Baer L, Marks IM. Technology-Based Advances in the Management of Depression. Dis Manag Health Outcomes. 2000 Apr 1;7(4):193–200.
- 14. Forster AJ, Boyle L, Shojania KG, Feasby TE, van Walraven C. Identifying Patients with Post-discharge Care Problems Using an Interactive Voice Response System. J Gen Intern Med. 2009 Apr 1;24(4):520–5. pmid:19156467
- 15.
Altman DG. Practical statistics for medical research. 1991: pp. 404. London, England: Chapman & Hall/CRC.
- 16. Haberer JE, Kiwanuka J, Nansera D, Wilson IB, Bangsberg DR. Challenges in using mobile phones for collection of antiretroviral therapy adherence data in a resource-limited setting. AIDS Behav. 2010 Dec;14(6):1294–301. pmid:20532605
- 17. Hettema JE, Hosseinbor S, Ingersoll KS. Feasibility and Reliability of Interactive Voice Response Assessment of HIV Medication Adherence: Research and Clinical Implications. HIV Clin Trials. 2012;13(5):271–7. pmid:23134627
- 18. Tucker JA, Simpson CA, Huang J, Roth DL, Stewart KE. Utility of an Interactive Voice Response System to Assess Antiretroviral Pharmacotherapy Adherence Among Substance Users Living with HIV/AIDS in the Rural South. AIDS Patient Care STDs. 2013 May;27(5):280–6. pmid:23651105
- 19. Piette JD, McPhee SJ, Weinberger M, Mah CA, Kraemer FB. Use of automated telephone disease management calls in an ethnically diverse sample of low-income patients with diabetes. Diabetes Care. 1999 Aug;22(8):1302–9. pmid:10480775
- 20. Lund S, Nielsen BB, Hemed M, Boas IM, Said A, Said K, et al. Mobile phones improve antenatal care attendance in Zanzibar: a cluster randomized controlled trial. BMC Pregnancy Childbirth. 2014;14:29. pmid:24438517
- 21. da Costa TM, Barbosa BJP, e Costa DAG, Sigulem D, de Fátima Marin H, Filho AC, et al. Results of a randomized controlled trial to assess the effects of a mobile SMS-based intervention on treatment adherence in HIV/AIDS-infected Brazilian women and impressions and satisfaction with respect to incoming messages. Int J Med Inf. 2012 Apr;81(4):257–69. pmid:22296762
- 22. Lester RT, Ritvo P, Mills EJ, Kariri A, Karanja S, Chung MH, et al. Effects of a mobile phone short message service on antiretroviral treatment adherence in Kenya (WelTel Kenya1): a randomised trial. The Lancet. 2010 Dec 3;376(9755):1838–45.
- 23. Gurol-Urganci I, de Jongh T, Vodopivec-Jamsek V, Atun R, Car J. Mobile phone messaging reminders for attendance at healthcare appointments. Cochrane Database Syst Rev. 2013 Dec 5;(12):CD007458. pmid:24310741
- 24. Guy R, Hocking J, Wand H, Stott S, Ali H, Kaldor J. How Effective Are Short Message Service Reminders at Increasing Clinic Attendance? A Meta-Analysis and Systematic Review. Health Serv Res. 2012 Apr 1;47(2):614–32. pmid:22091980
- 25. Hughes CM, McElnay JC, Fleming GF. Benefits and Risks of Self Medication. Drug Saf. 2012 Nov 18;24(14):1027–37.
- 26. Ruiz ME. Risks of Self-Medication Practices. Curr Drug Saf. 2010 Oct 1;5(4):315–23. pmid:20615179
- 27. Dini EF, Linkins RW, Sigafoos J. The impact of computer-generated messages on childhood immunization coverage. Am J Prev Med. 2000 Feb;18(2):132–9. pmid:10698243
- 28. Meankaew P, Kaewkungwal J, Khamsiriwatchara A, Khunthong P, Singhasivanon P, Satimai W. Application of mobile-technology for disease and treatment monitoring of malaria in the “Better Border Healthcare Programme”. Malar J. 2010;9:237. pmid:20723223
- 29. Rowe AK, Rowe SY, Snow RW, Korenromp EL, Schellenberg JRA, Stein C, et al. The burden of malaria mortality among African children in the year 2000. Int J Epidemiol. 2006 Jun 1;35(3):691–704. pmid:16507643
- 30. Appawu M, Owusu-Agyei S, Dadzie S, Asoala V, Anto F, Koram K, et al. Malaria transmission dynamics at a site in northern Ghana proposed for testing malaria vaccines. Trop Med Int Health. 2004 Jan 1;9(1):164–70. pmid:14728621
- 31. Brodey BB, Rosen CS, Brodey IS, Sheetz BM, Steinfeld RR, Gastfriend DR. Validation of the Addiction Severity Index (ASI) for internet and automated telephone self-report administration. J Subst Abuse Treat. 2004 Jun;26(4):253–9. pmid:15182889
- 32. Perrine MW, Mundt JC, Searles JS, Lester LS. Validation of daily self-reported alcohol consumption using interactive voice response (IVR) technology. J Stud Alcohol. 1995 Sep;56(5):487–90. pmid:7475027