Introduction

It is well-recognized that the promise of efficacious therapies to treat and prevent long-term conditions cannot be met unless patients consistently adhere to prescribed regimens [1]. Medication adherence influences individual health outcomes, the cost of healthcare and, in the case of communicable infections, can influence transmission and the emergence of resistant strains. In clinical settings, accurate measurement of adherence is necessary to inform interpretation of treatment response and identify individuals in need of enhanced adherence support. In research settings, accurate measurement of adherence is necessary to ensure that therapeutic outcomes can be attributed to the assigned treatment, especially when an intervention effect is determined based on adherence. Unfortunately, accurate adherence measurement and monitoring remain significant challenges in both clinical and research settings.

In lieu of a single “gold standard” for measuring adherence [1], several direct and indirect measurement methods were developed. The ideal methods for measuring adherence vary depending on the setting and purpose of measuring adherence [2, 3]. Direct methods are generally more objective, yielding more reliable adherence assessments, but all methods have limitations. Electronic monitoring devices have been recommended to collect detailed adherence information but their use is limited by cost and risk of malfunction [3]. Pill counting is considered more feasible, and when used to measure antiretroviral adherence, it has been associated with viral suppression [4–7]. Some studies, however, failed to demonstrate an association between pill counts and viral load change [8]. Inaccuracies in clinic-based pill counts primarily stem from patients failing to bring their pills to the clinic, as well as emptying pill containers ("pill dumping”) prior to the appointment [9]. Bangsberg et al developed a home-based unannounced pill counting procedure, which reduces the opportunity to “pill dump”, because patients do not know the time of pill counts in advance [10]. Home-based unannounced pill counts virtually eliminate the potential problems faced by clinic-based pill counts, correlate with electronic medication monitoring [11, 12], and predict changes in viral load over time [10, 13].

Logistics and cost are major limitations of home-based unannounced pill counts. Unannounced home visits are often met with unanswered doors and concerns of privacy and confidentiality due to stigma. Additionally, these visits are logistically challenging and time-consuming in rural areas. To increase utility and reduce costs, Kalichman and colleagues adapted and tested a phone-based unannounced pill count protocol [14]. Phone-based unannounced pill counts were shown to be feasible and reliable measures of adherence in developed settings [14–18]; and have been further used as part of medication adherence interventions [19]. However, it is not clear whether this method can be implemented successfully in resource-limited settings, where cellular network and mobile phone coverage may be low. As part of the START (Start TB patients on ART and Retain on Treatment) and ENRICH (ENhance initiation and Retention in IPT Care for HIV) implementation science studies, we employed phone-based monthly unannounced pill counts to measure anti-tuberculosis (TB) and antiretroviral medication adherence. Our objective is to describe operational issues surrounding the use of phone-based unannounced pill counts in resource-limited settings.

Materials and methods

START and ENRICH methods, including settings and facility characteristics, have been described elsewhere [20–22]. In brief, the START Study (ClinicalTrials.gov NCT01872390) is a cluster-randomized trial which evaluated the effectiveness, cost-effectiveness and acceptability of a combination intervention package (CIP) versus standard of care (SOC), to improve early antiretroviral therapy (ART) initiation, retention, and TB treatment success, among HIV-positive TB patients at 12 health facilities in Berea District, Lesotho, a nation with an HIV prevalence of 25% [23] and a TB incidence rate of 788 per 100,000 [24]. The ENRICH Study (ClinicalTrials.gov NCT01926379) is a cluster-randomized trial which evaluated the effectiveness of a CIP vs. SOC, to improve isoniazid preventive therapy (IPT) initiation, adherence, and completion among HIV-positive patients newly enrolled in HIV care at 10 health centers in Dire Dawa and Harari, Ethiopia, a nation with close to 800,000 people with HIV [25] and a TB incidence rate of 192 per 100,000 [24]. The follow-up period in both studies was six months.

Research Assistants (RA) received training on the unannounced pill count procedure and were provided with study phones to implement the pill counts. In both studies, monthly phone-based unannounced pill counts were conducted with all participants in both study arms. Pill counts were completed using an adaptation of a standardized protocol from previous studies [14], whereby participants were called at undisclosed times between monthly visits and asked to count their pills (S1 and S2 Files). Participants were not compensated for the pill count calls. At the baseline study visit, participants were trained to count their medications after answering the phone. In advance of the call, the RA recorded the name, dispense date, number of pills dispensed, number of doses per day, and number of pills per dose for each medication. Before proceeding with the pill counts, the RA first confirmed participants were in a private area and had access to their pills. The RA recorded the date, whether pills were taken before the call, and for each medication, how many pills were left. If a participant did not answer the call, the RA did not leave a message and attempted to reach the participant until the pill counts were completed, or five attempts have been made within the same month without answer or pill counts completion. The original study protocols called for three call attempts, but that was determined to be insufficient and after 15 and 12 months follow-up for START and ENRICH respectively, the number of attempts was increased to five. If the participant requested that the RA call back at another time, the RA did not schedule the callback at a specific time in order to maintain the pill count as ‘unannounced’.

We systematically collected quantitative and qualitative data on reasons for pill counts non-completion from study records, through weekly pill count completion reports submitted by RA (including attempt date, outcome, and if unsuccessful, the reason why), and conversations between RA and participants during study visits. Reasons for non-completed counts were recorded as open-ended responses that were later coded and categorized by the study investigators as network problems, numeracy issues, not having a phone, lack of privacy for the call, phone answered by someone other than the study participant, or not having pills at the time of the call. Qualitative data were analyzed using Microsoft Excel. Quantitative analyses were conducted using SAS 9.4 (SAS Institute Inc., Cary, NC). Participant characteristics and call outcomes were summarized with descriptive statistics using means, medians, proportions, and Pearson’s χ².

In Lesotho, mobile phone ownership was high (86 and 102 mobile cellular subscriptions per 100 people in 2013 and 2014 respectively [26]); therefore, we did not distribute mobile phones to participants. However in Ethiopia, mobile phone ownership was much lower (27 and 32 per 100 people in 2013 and 2014, respectively [26]), and as the ENRICH study intervention included provision of adherence support messages to patients on IPT using interactive voice response technology, we distributed inexpensive mobile phones to participants in the CIP arm. Participants received a study-issued mobile phone, SIM card, and six months’ airtime vouchers at an approximate total cost of USD 45 per patient. Landlines in the home are practically nonexistent in the study locations, thus mobile phones were used exclusively to reach participants for pill counts.

All procedures performed in these studies were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. The START Study was approved by the Columbia University Medical Center Institutional Review Board (CUMC-IRB) (Ref #IRB-AAAK7103) and Lesotho’s National Health Institutional Review Board and Ethics Research Committee (Ref #ID68-2012). The ENRICH Study was approved by the CUMC-IRB (Ref #IRB-AAAK3163) and the National Research Ethics Review Committee in Ethiopia (Ref #3.10/780/06). Informed consent was obtained from all individual participants included in the studies. Participants were informed that monthly phone-based unannounced pill counts will be conducted as part of the informed consent process.

Results

START enrolled 371 participants, who were followed up for a median of 4.8 months; 2831 calls to 334 participants were attempted and 554 (19.6%) calls were completed. Attempts were not made for 37 (10%) participants who didn’t have a phone (n = 25) or could be contacted after hours only (n = 12). Participant characteristics are shown in Table 1. Overall, 240/334 (71.9%) START participants were reached at least once. The mean (±standard deviation [SD]) number of attempts per participant each month was 1.9±1.0. Completeness of calls was slightly higher during months 1 and 2 of follow-up at 24.8% and 25.6%, respectively; calls attempted during months 3–6 were completed 14.6%-17.9% of the time. Success rates were similar across study arms (21.0% vs. 18.8%, p = 0.17) and significantly higher for females than males (23.2% vs. 16.6%, p<0.001).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Participant characteristics at baseline.

https://doi.org/10.1371/journal.pone.0185549.t001

ENRICH enrolled 316 participants, who were followed up for a median of 4.7 months; 3156 calls to 309 participants were attempted and 1058 (33.5%) calls were completed. Attempts were not made for 7 (2%) participants in the SOC study arm who didn’t have a phone (Table 1). Overall, 268/309 (86.7%) ENRICH participants were reached at least once. The mean (±SD) number of attempts per participant for each month was 2.1±1.4. Completeness of monthly calls was fairly consistent but declined from 36.6% in month 1 to 30.2% in month 6. Success rates were higher at CIP than SOC sites (38.4% vs. 27.2%, p<0.001); reasons differed by arm (p<0.001). SOC sites had a higher proportion of failed calls recorded as resulting from the patient having no phone (8.4% vs 1.5%, p < .0001) or being due to network problems (19.0% vs 7.7%, p < .0001). The most common reason for failure in the CIP arm was the phone was switched off (32.9%) followed by the phone was not answered (31.6%). Success rates did not differ between male and female participants.

As shown in Table 2, the majority of failed call attempts in START were due to the phone not being available (54.8%) or because the participant was away from the pills (32.7%). In ENRICH, the main reasons for failed call attempts were phone being switched off (31.5%), participants not answering the phone (27.3%), or participants’ discomfort with speaking on the phone (15.4%). Network problem was listed by the RA as the cause of failed calls 13.2% of the time.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Reasons for failed calls.

https://doi.org/10.1371/journal.pone.0185549.t002

Many challenges were encountered in implementation of phone-based unannounced pill counts in these settings (Fig 1). In response, the respective study teams explored reasons for these challenges through conversations with participants at monthly study visits, and developed solutions as challenges arose over the course of the studies.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Implementation challenges of phone-based unannounced pill counts in resource-limited settings.

https://doi.org/10.1371/journal.pone.0185549.g001

Discussion

To our knowledge, this is the first examination of the implementation of phone-based unannounced pill counts in resource-limited settings. In our implementation science studies in Lesotho and Ethiopia, phone-based unannounced pill counts were found to be challenging, with response rates varying between 20% and 34% in the two studies. Structural, facility-level, participant-level, and data collection challenges were encountered. While some challenges were specific to the local context (e.g., migration between Lesotho and South Africa; low cell phone ownership and literacy rates in Ethiopia), many are generalizable to resource-limited settings. It is important to note that the main contributors to failed calls were availability of phones, switched-off phones, and participants being away from the medicines. Many participants required as many as five calls to reach them, and many attempts did not result in successful completion of pill counts. It is important to consider the possibility that participants did not answer the phone when they recognized the study phone number because they could not, or did not want to, perform the pill count. This may have also contributed to the declining pill counts response rate over time. However, in these settings it’s not possible to have the caller number displayed as ‘private caller’.

While phone-based pill counts can be conducted in urban and rural areas, and missed phone contacts are inexpensive and allow staff to contact another participant without delay, they are also staff-intensive. It is crucial to carefully consider the practicalities of pill counts logistics, including the feasibility of making calls outside of normal work hours, and scheduling and ensuring attempts are actually made. In an effort to minimize implementation challenges, we consulted with experienced researchers who have used this procedure, albeit in developed settings, before study commencement. In a research study context, a possible solution to ease operational challenges may be to focus phone-based unannounced pill count efforts on a randomly selected sample from participants who are provided with study phones and rigorously ensure that up to five call attempts are made for these participants if they are not reached sooner. This approach may mitigate the potential for possible Hawthorne effects in two ways. First, the pill counts are unannounced and therefore participants do not know when they might be contacted; and second, participants will be randomly selected and therefore not know whether or not they will be contacted.

Earlier research experiences in the US found phone-based unannounced pill counts to be a feasible and reliable measure of medication adherence [14–17]. Among the greatest differences between early experiences and ours are structural challenges, including lack of mobile phone ownership, poor network availability, and participant mobility. Kalichman et al provided each participant with a dedicated mobile phone to maximize reachability [14]. Eligibility criteria in the study by Thompson et al included having a functioning phone [16]. As noted above, mobile phone ownership was very high in Lesotho and we supplied mobile phones to CIP participants in Ethiopia. Therefore, while phone ownership in our studies (except for SOC participants in Ethiopia) should have been comparable to prior studies, we suspect that unreliable network coverage and limited availability of electricity to charge phone batteries significantly impacted our participants’ ability to respond to calls. Kalichman et al found that 10/99 (10.1%) participants were unreachable by any phone, including the dedicated study phone given to them [15], compared to 94/334 (28.1%) of START participants and 41/309 (13.3%) of ENRICH participants. In a recent US study where participants were not provided with a mobile phone, Fredericksen et al found high concordance for phone-based and home-based unannounced pill counts [17]. The distribution of study mobile phones to CIP participants in Ethiopia had the potential to facilitate phone-based unannounced pill counts; however, phone use remained challenging in this setting. Surprisingly, when we compared response rates for study participants in the SOC arms, i.e. participants who did not receive a study phone, we found the response rate in Ethiopia to be higher than in Lesotho, which may be further indication that phone ownership does not guarantee good response. The relatively high level of population mobility in Lesotho, particularly among Basotho men, also posed a unique impediment to the completion of pill counts, which may account for a significantly higher response rate among females than males in Lesotho.

With respect to scheduling and management of the call attempts, we followed Kalichman [14] in having RA complete a baseline assessment with participants indicating the days and times which were most and least preferred to receive calls. Despite efforts to maximize the likelihood of successful calls by calling during acceptable times, the number of attempts required to reach our participants were similar to studies in developed settings. Kalichman et al reported that 57% of participants were reached on the first attempt, 23% on the second attempt, and 20% required ≥3 attempts [14], while Thompson et al found that nearly half of participants were contacted on the first attempt, with “the majority” being contacted by the second attempt [16]. These studies do not indicate common reasons why participants were not always reachable on the first or second attempt. Perhaps most notable is Fredericksen et al’s report that approximately four attempts were made to call participants before successfully reaching them [17]. Given the additional logistical and structural barriers present in resource-limited settings, the initial protocol of three attempts was perhaps overly optimistic. Authors of earlier studies do not provide details regarding human resources and call scheduling (e.g., whether a single staff member was dedicated to making calls, whether calls were attempted outside working hours), both of which would seem likely to increase pill count completion rates, but neither of which are likely to be feasible within public health systems in resource-limited settings. Integrating special software that helps manage the scheduling of these calls, such as maintaining a log of previous attempts and prompting additional attempts at given times, may have helped with their implementation. Having staff dedicated exclusively to the task of conducting unannounced pill counts might have alleviated some management and scheduling challenges, but such arrangements would be difficult to integrate into the structure of clinics in any future transition of implementation from research to clinic staff.

Participant confidence and competence in completing pill counts also offers a point of comparison. Kalichman et al report no participant was unable to count or required assistance, although they acknowledged this does not preclude the possibility of miscalculation or misidentification of medications [15]. This caveat is particularly important to bear in mind for populations with comorbidities, who are taking multiple medications. The major challenge reported by Kalichman and colleagues was that of participants struggling to manage and organize the pill count procedure, and they suggested repeated training and rehearsal with participants to minimize this problem; this additional training was undertaken in our studies by RA, village health workers (Lesotho), and peer educators (Ethiopia) to assist participants with the use of phones and with counting. While our studies in Lesotho and Ethiopia found that participants with low numeracy were in fact able to complete counts (contrary to prior expectations [15]), ultimately larger structural and logistical barriers overshadowed the overall feasibility of phone-based unannounced pill counts.

Conclusion

Phone-based unannounced pill counts have proven challenging in two implementation science studies in resource-limited settings, which may cast a shadow over their utility in assessing adherence in research studies. However, in a research study context, a sample-based ascertainment of adherence using phone-based unannounced pill counts may offer a feasible solution to collecting adherence data in these settings.

Supporting information

Acknowledgments

The authors would like to thank Wafaa El-Sadr, Blanche Pitt, Angela Campbell, Mary-Elizabeth Vachon, Mashale Shale, Moeketsi Ntoane, Limakatso Lebelo, Bereket Tefera, Mengisteab Kidane, Sweta Adhikari, and Katharine Yuengling for their contributions to study design and implementation. They are indebted to the study participants for their time and willingness to contribute to the understanding of service delivery for HIV-positive patients. They thank staff at the study sites; village health workers in the surrounding communities; the Berea District Health Management Team and the Lesotho Ministry of Health; and the Dire Dawa Administration and Harari Regional Health Bureaus in Ethiopia for their invaluable assistance in conducting this study.