Figures
Abstract
Background
World Health Organization (WHO) guidelines for monitoring HIV-infected individuals taking combination antiretroviral therapy (cART) in resource-limited settings recommend using CD4+ T cell (CD4) count changes to monitor treatment effectiveness. In practice, however, falling CD4 counts are a consequence, rather than a cause, of virologic failure. Adherence lapses precede virologic failure and, unlike CD4 counts, data on adherence are immediately available to all clinics dispensing cART. However, the accuracy of adherence assessments for predicting future or detecting current virologic failure has not been determined. The goal of this study therefore was to determine the accuracy of adherence assessments for predicting and detecting virologic failure and to compare the accuracy of adherence-based monitoring approaches with approaches monitoring CD4 count changes.
Methodology and Findings
We conducted an observational cohort study among 1,982 of 4,984 (40%) HIV-infected adults initiating non-nucleoside reverse transcriptase inhibitor-based cART in the Aid for AIDS Disease Management Program, which serves nine countries in southern Africa. Pharmacy refill adherence was calculated as the number of months of cART claims submitted divided by the number of complete months between cART initiation and the last refill prior to the endpoint of interest, expressed as a percentage. The main outcome measure was virologic failure defined as a viral load > 1,000 copies/ml (1) at an initial assessment either 6 or 12 mo after cART initiation and (2) after a previous undetectable (i.e., < 400 copies/ml) viral load (breakthrough viremia). Adherence levels outperformed CD4 count changes when used to detect current virologic failure in the first year after cART initiation (area under the receiver operating characteristic [ROC] curves [AUC] were 0.79 and 0.68 [difference = 0.11; 95% CI 0.06 to 0.16; χ2 = 20.1] respectively at 6 mo, and 0.85 and 0.75 [difference = 0.10; 95% CI 0.05 to 0.14; χ2 = 20.2] respectively at 12 mo; p < 0.001 for both comparisons). When used to detect current breakthrough viremia, adherence and CD4 counts were equally accurate (AUCs of 0.68 versus 0.67, respectively [difference = 0.01; 95% CI −0.06 to 0.07]; χ2 = 0.1, p > 0.5). In addition, adherence levels assessed 3 mo prior to viral load assessments were as accurate for virologic failure occurring approximately 3 mo later as were CD4 count changes calculated from cART initiation to the actual time of the viral load assessments, indicating the potential utility of adherence assessments for predicting future, rather than simply detecting current, virologic failure. Moreover, combinations of CD4 count and adherence data appeared useful in identifying patients at very low risk of virologic failure.
Conclusions
Pharmacy refill adherence assessments were as accurate as CD4 counts for detecting current virologic failure in this cohort of patients on cART and have the potential to predict virologic failure before it occurs. Approaches to cART scale-up in resource-limited settings should include an adherence-based monitoring approach.
Citation: Bisson GP, Gross R, Bellamy S, Chittams J, Hislop M, Regensberg L, et al. (2008) Pharmacy Refill Adherence Compared with CD4 Count Changes for Monitoring HIV-Infected Adults on Antiretroviral Therapy. PLoS Med 5(5): e109. https://doi.org/10.1371/journal.pmed.0050109
Academic Editor: David Bangsberg, San Francisco General Hospital, United States of America
Received: November 14, 2007; Accepted: April 3, 2008; Published: May 20, 2008
Copyright: © 2008 Bisson et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: We are grateful for the support of the National Institutes of Health, which supported this work through the University of Pennsylvania Center for AIDS Research Developmental and Clinical Cores (P30-AI45008); the PENN AIDS Clinical Trials Unit (U01-AI32783); Career Development Awards K23AI058881 (GB), K23AI068582–01 (JBN), and K08MH01584 (RG); and an Agency for Healthcare Research and Quality (AHRQ) Centers for Education and Research on Therapeutics cooperative agreement (HS10399). We also acknowledge support from the Doris Duke Charitable Foundation (GB and GM). The sponsors of this study had no role in the study design; in the collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the paper for publication.
Competing interests: The authors have declared that no competing interests exist.
Abbreviations: AUC, area(s) under the ROC curve; cART, combination antiretroviral therapy; CI, confidence interval; IQR, interquartile range; NNRTI, non-nucleoside reverse transcriptase inhibitor; NPV, negative predictive value; OR, odds ratio; PPV, positive predictive value; ROC, receiver operating characteristic; WHO, World Health Organization
Editors' Summary
Background.
Globally, more than 30 million people are infected with the human immunodeficiency virus (HIV), the cause of acquired immunodeficiency syndrome (AIDS). Combinations of antiretroviral drugs that hold HIV in check (viral suppression) have been available since 1996. Unfortunately, most of the people affected by HIV/AIDS live in developing countries and cannot afford these expensive drugs. As a result, life expectancy has plummeted and economic growth has reversed in these poor countries since the beginning of the AIDS pandemic. Faced with this humanitarian crisis, the lack of access to HIV treatment was declared a global health emergency in 2003. Today, through the concerted efforts of governments, international organizations, and funding bodies, about a quarter of the HIV-positive people in developing and transitional countries who are in immediate need of life-saving, combination antiretroviral therapy (cART) receive the drugs they need.
Why Was This Study Done?
To maximize the benefits of cART, health-care workers in developing countries need simple, affordable ways to monitor viral suppression in their patients—a poor virologic response to cART can lead to the selection of drug-resistant HIV, rapid disease progression, and death. In developed countries, virologic response is monitored by measuring the number of viral particles in patients' blood (viral load) but this technically demanding assay is unavailable in most developing countries. Instead, the World Health Organization recommends that CD4+ T cell (CD4) counts be used to monitor patient responses to cART in resource-limited settings. HIV results in loss of CD4 cells (a type of immune system cell), so a drop in a patient's CD4 count often indicates virologic failure (failure of treatment to suppress the virus). However, falling CD4 counts are often a result of virologic failure and therefore monitoring CD4 counts for drops is unlikely to prevent virologic failure from occurring. Rather, falling CD4 counts are often used only to guide a change to new medicines, which may be even more expensive or difficult to take. On the other hand “adherence lapses”—the failure to take cART regularly—often precede virologic failure, so detecting them early provides an opportunity for improvement in adherence that could prevent virologic failure. Because clinics that dispense cART routinely collect data that can be used to calculate adherence, in this study the researchers investigate whether assessing adherence might provide an alternative, low-cost way to monitor and predict virologic failure among HIV-infected adults on cART.
What Did the Researchers Do and Find?
The Aid for AIDS Disease Management Program provides cART to medical insurance fund subscribers in nine countries in southern Africa. Data on claims for antiretroviral drugs made through this program, plus CD4 counts assessed at about 6 or 12 months after initiating cART, and viral load measurements taken within 45 days of a CD4 count, were available for nearly 2,000 HIV-positive adults who had been prescribed a combination of HIV drugs including either efavirenz or nevirapine. The researchers defined adherence as the number of months of cART claims submitted divided by the number of complete months between cART initiation and the last pharmacy refill before a viral load assessment was performed. Virologic failure was defined in two ways: as a viral load of more than 1,000 copies per ml of blood 6 or 12 months after cART initiation, or as a rebound of viral load to similar levels after a previously very low reading (breakthrough viremia). The researchers' statistical analysis of these data shows that at 6 and 12 months after initiation of cART, adherence levels indicated virologic failure more accurately than CD4 count changes. For breakthrough viremia, both measurements were equally accurate. Adherence levels during the first 3 months of cART predicted virologic failure at 6 months as accurately as did CD4 count changes since cART initiation. Finally, the combination of adherence levels and CD4 count changes accurately identified patients at very low risk of virologic failure.
What Do These Findings Mean?
These findings suggest that adherence assessments (based in this study on insurance claims for pharmacy refills) can identify the patients on cART who are at high and low risk of virologic failure at least as accurately as CD4 counts. In addition, they suggest that adherence assessments could be used for early identification of patients at high risk of virologic failure, averting the health impact of treatment failure and the cost of changing to second-line drug regimens. Studies need to be done in other settings (in particular, in public clinics where cART is provided without charge) to confirm the generalizability of these findings. These finding do not change that fact that monitoring CD4 counts plays an important role in deciding when to start cART or indicating when cART is no longer protecting the immune system. But, write the researchers, systematic monitoring of adherence to cART should be considered as an alternative to CD4 count monitoring in patients who are receiving cART in resource-limited settings or as a way to direct the use of viral load testing where feasible.
Additional Information.
Please access these Web sites via the online version of this summary at http://dx.doi.org/10.1371/journal.pmed.0050109.
- This study is discussed further in a PLoS Medicine Perspective by David Bangsberg
- Information is available from the US National Institute of Allergy and Infectious Diseases on HIV infection and AIDS
- HIV InSite has comprehensive information on all aspects of HIV/AIDS, including an article about adherence to antiretroviral therapy
- Information is available from Avert, an international AIDS charity, on HIV and AIDS in Africa and on providing AIDS drug treatment for millions
- The World Health Organization provides information about universal access to HIV treatment (in several languages) and on its recommendations for antiretroviral therapy for HIV infection in adults and adolescents
- The US Centers for Disease Control and Prevention also provides information on global efforts to deal with the HIV/AIDS pandemic (in English and Spanish)
Introduction
As the number of patients on combination antiretroviral therapy (cART) grows worldwide, developing simple, affordable ways of monitoring patients after treatment initiation has become a major public health priority. Since the central paradigm of antiretroviral therapy is suppression of viral replication, and since costs of second-line cART are higher than first-line regimens [1], monitoring efforts should, as much as possible, focus on preserving the virologic effectiveness of first-line combinations. Failure to identify patients who are at high risk of future virologic failure or who are currently on partially suppressive regimens may result in selection of viral resistance mutations, which have been associated with more rapid disease progression and death [2–4].
In the developed world, the standard of care for monitoring virologic response involves measuring plasma HIV-1 RNA levels (“viral loads”) [5]. These assays are often unavailable in the developing world because of financial and technical constraints [6]. Since CD4+ T cell (CD4) counts are comparatively inexpensive, World Health Organization (WHO) guidelines for scaling up antiretroviral therapy in resource-limited settings advocate use of CD4 count criteria to identify patients on failing cART regimens [7]. Thus, CD4 counts are considered an essential tool for monitoring patients on cART [8], and there is a widespread movement to incorporate cheaper, less technologically demanding CD4 count assays into clinical care in the developing world [9].
Quantifying and monitoring adherence to cART is one potentially useful and low-cost method of identifying patients at high risk for virologic failure in resource-limited settings [10]. Adherence is strongly associated with virologic response in a dose-dependent manner [11–16]. Furthermore, among patients on first-line therapy, lapses in adherence usually precede immunologic declines and, unlike CD4 count data, adherence data available to all clinics that dispense cART may be simple to compile [10]. Furthermore, they directly measure the variable on which providers can intervene. Thus, adherence assessments focus on the cause rather than the consequence of virologic failure and may enable guided interventions capable of preventing virologic failure. Therefore, although CD4 count monitoring in patients on cART is deeply ingrained in HIV care [5,7], if adherence assessments are as accurate as CD4 count changes for identification of patients with virologic failure, sites currently performing or planning CD4 count measurements for this purpose could instead choose to monitor adherence, thereby preserving scarce resources for triaged virologic monitoring [17,18] or other treatment-related activities.
Despite the potential time and cost savings of this method, the diagnostic accuracy of various adherence levels for predicting virologic failure has not been determined, and current WHO guidelines for monitoring cART in resource-limited settings include only general recommendations for assessing adherence and no detail about the levels of adherence that should trigger interventions [7]. To address this information gap, we compared the diagnostic accuracy of CD4 cell count changes and adherence measurements for virologic failure on cART.
Methods
Study Design
The primary analysis of this observational cohort study evaluated the relative abilities of pharmacy refill adherence and CD4 counts to detect current viral loads indicative of treatment failure. Therefore the primary outcome of interest was virologic failure, defined as a viral load >1,000 copies/ml. Assessments of this outcome were considered in two ways: (1) lack of virologic response (defined as a viral load >1,000 copies/ml) either 6 or 12 mo after cART initiation, and (2) breakthrough viremia, as demonstrated by a follow-up viral load >1,000 copies/ml after achievement of an undetectable (i.e., < 400 copies/ml) viral load. Lack of response within the first year was chosen as an outcome because of the clinical relevance of initial response to cART on subsequent disease outcomes [19], and breakthrough viremia was chosen as an outcome because several studies have reported that a majority of patients initiating cART achieve viral loads < 400 copies/ml within the first year [19–22]. For the primary analyses focused on detecting current virologic failure, the available CD4 count and adherence data were analyzed up to the time the provider was assessing the likelihood of virologic failure. Sensitivity was defined as the proportion of patients with virologic failure who met certain CD4 count change or adherence level criteria, and specificity was defined as the proportion of patients without virologic failure who did not meet these criteria. Positive (negative) predictive value was defined as the probability that a patient meeting (not meeting) certain CD4 count change or adherence level criteria had (did not have) virologic failure at that time. Breakthrough viremia was defined as occurrence of virologic failure at any time > 30 d after a prior undetectable viral load.
Study Setting
We examined medical records from HIV-1–infected adults enrolled in Aid for AIDS, a private health care management program available to subscribers to medical insurance funds in nine countries in southern Africa. Patient demographic and clinical data and pharmacy drug information have been recorded by Aid for AIDS since June 1998 and have been described previously [12,20]. In brief, if HIV-infected patients consent, baseline demographic and clinical data are recorded in the electronic Aid for AIDS database at the time of patient enrollment. After enrollment, individuals with CD4 counts < 350 cells/μl or with an AIDS-defining condition are eligible to initiate cART. Patients submitting pharmacy claims are reimbursed by their medical insurance fund for the cost of drugs. All claims are processed through the coordinating center at the Aid for AIDS Cape Town office. Claims include the drug names and date of the prescription refill, and drugs are dispensed in uniform increments of 30 d of an entire cART regimen each time a prescription is refilled. Differential delays between countries or sites within countries with respect to returning prescriptions for processing are rare.
Study Participants
Patients who met the following general criteria were eligible: (1) age ≥ 18 y; (2) pretreatment plasma viral load level of > 2,000 copies/ml; (3) initiated non-nucleoside reverse transcriptase inhibitor (NNRTI)-based cART, a criterion chosen because of its relevance to resource-limited settings [23,24] and defined as an NNRTI plus two nucleoside reverse transcriptase inhibitors; and (4) CD4 count and viral load assessment within 90 d prior to or on the day of therapy initiation. In addition, for the evaluation of treatment response at 6 and 12 mo, patients needed: (1) a follow-up CD4 count within 3–9 or 9–15 mo after initiating cART, respectively, and (2) a follow-up viral load within 45 d of the corresponding follow-up CD4 count. The analysis of breakthrough viremia was limited to patients meeting general criteria who also had (1) at least one undetectable viral load obtained after cART initiation, (2) a subsequent follow-up viral load obtained at least 30 d after the first undetectable viral load, and (3) a CD4 count obtained within at least 45 d of this follow-up viral load. Patients could be included in one or more analyses.
Data Collection
Decisions to monitor patients in Aid for AIDS are left up to the patients' physicians, and patient specimens are sent to a variety of clinical laboratories, although physicians are instructed to use the same laboratory for each patient.
WHO-advocated criteria of a CD4 count drop to pretreatment levels or below, a CD4 count drop to 50% or less of maximum on-therapy levels, and a CD4 count persistently below 100 cells/μl as well as alternative CD4 count criteria, were examined [7]. The analysis of a CD4 count decrease to 50% of maximum on-therapy levels was limited to the analysis of breakthrough viremia, since this criterion implies that previous CD4 count monitoring on therapy has been done. The last viral load and corresponding CD4 count available in the database were chosen as the endpoint values for patients who did not have breakthrough viremia.
Pharmacy claims adherence data in this dataset have been validated for virologic response and for survival [12,20]. Adherence was calculated as the number of months with cART claims submitted, divided by the number of complete months from cART commencement to the date of the relevant study endpoint, and the result multiplied by 100, as described [12,16]. Since patients fill entire cART regimens with each refill, we tracked the entire regimen rather than using an “index drug” approach, which tracks only a single drug of a regimen.
Statistical Analysis
The primary goal of the analysis was to examine and compare the ability of CD4 counts and adherence data to detect current virologic failure, rather than to explore etiologic associations with these outcomes. However, in order first to understand risk factors for virologic failure in the patient population, we performed unadjusted and adjusted analyses for this outcome. Differences in baseline characteristics were assessed with two-sample t-tests or with Wilcoxon rank sum tests (continuous variables), depending on the distribution of the data, and Chi-square (χ2) tests were used to compare categorical variables. A relative risk for the primary exposures (e.g., CD4 count criteria and adherence level) with a 95% confidence interval (CI) was determined and then evaluated in multivariable logistic regression analysis (which produced odds ratios [ORs]) to assess possible confounding. Confounding was considered present if the unadjusted OR changed by 15% or more after adjustment. However, in order to evaluate whether the inclusion of possible confounders not meeting this criterion in the analysis affected the study's findings, we also evaluated the results after forcing variables plausibly associated with virologic failure in the multivariable model [25]. For uniformity, all presented ORs are adjusted values.
Overall diagnostic accuracy of adherence and CD4 count changes were expressed using receiver operating characteristic (ROC) curves and 95% CIs. Larger areas under the ROC curve (AUC) indicate greater overall ability to discriminate between patients with and without virologic failure. For tests with binary outcomes (e.g., presence or absence of virologic failure), the area under the ROC curve is equal to the c (for concordance) statistic [26]. For binary endpoints, c-statistics are the proportion of all pairs of patients, one with and one without the outcome, in which the patient with the event had a greater predicted probability of the outcome [27]. For example, a coin toss would have a c-statistic of 0.5, whereas a test with perfect discrimination would have a c-statistic of 1.0 [28]. The c-statistics were compared using Chi-square tests in Stata version 9.0 (Intercooled) (STATA, College Station, Texas). p-Values for the primary comparison of AUCs at 6 and 12 mo and for breakthrough viremia were not adjusted for multiple comparisons. Furthermore, we used bootstrap resampling to evaluate the robustness of our findings by resampling with replacement observations from the original dataset 999 times, which produced 95% CIs for the mean difference in AUCs [29]. The sensitivity of our findings to the definition of virologic failure was analyzed using a level of ≥ 10,000 copies/ml as the outcome in secondary analyses. In addition, AUCs for ROC curves derived from rules created using specific criteria are presented to enable rapid evaluation of overall diagnostic accuracy of each criterion with respect to others, and to permit comparison of these ROC areas with a reference test of a CD4 count decrease to pretreatment levels or below. Since this process involved multiple comparisons, these p-values were corrected for multiple comparisons using Sidak's method. Test characteristics of each specific criterion were determined using logistic regression.
Sub-analyses.
We performed subanalyses to evaluate the possibility that adherence assessments could be performed months prior to the date of viral load measurements in order to predict future, rather than detect current, virologic failure. Specifically, we determined the accuracy of adherence assessments performed approximately 3 mo prior to a viral load assessment for virologic failure and compared this estimate of accuracy with that resulting from assessments of CD4 count changes calculated from the time of cART initiation to the time the viral load was done (e.g., 3 mo later). This was done for analyses of virologic failure at 6 and 12 mo, and statistically was performed by comparing AUCs as in the primary analysis. A schematic of this approach is given in Figure 1. We also examined whether use of days, rather than months, to calculate adherence altered the discriminatory ability of adherence assessments in these shorter intervals.
Adherence in the first 3 mo of cART was compared with the CD4 count change from cART initiation to the time of the 6-mo viral load. Adherence in the first 3 mo after the 6-mo follow-up date was compared with the CD4 count change measured from baseline to the time of the 12-mo viral load.
In addition, combinations of CD4 count and adherence data were assessed to determine whether combined approaches could be created that resulted in positive or negative predictive values that were sufficiently high (e.g., 95%) that providers could avoid viral load testing altogether. The proportion of patients with or without current virologic failure at the time of assessment who met a specific criterion was also determined to evaluate how applicable the criterion would be in clinical practice. These results were analyzed by computing the exact binomial 95% CIs for these test characteristics and proportions and comparing these to characteristics of adherence or CD4 count changes alone.
Regulatory Approvals
This study was approved by the University of Cape Town Research Ethics Committee, by the Aid for AIDS Clinical Advisory Committee and Board, Cape Town, South Africa, by the Johns Hopkins Bloomberg School of Public Health's Committee on Human Research and by the University of Pennsylvania Institutional Review Board. Data were analyzed anonymously, and a waiver of informed consent was obtained for the study.
Results
There were 5,723 adults who initiated cART and had registration information included in the Aid for AIDS database used in this study. Of these, 739 patients (13%) initiated non-NNRTI-based regimens and were therefore excluded. Of the remaining 4,984, 1,982 (40%) initiating NNRTI-based cART between 20 December 2000 and 28 February 2003 had sufficient paired CD4 count and viral load data both at baseline and at follow-up to be included in at least one of the analyses below. The pretreatment median (interquartile range [IQR]) CD4 counts were slightly lower among those who did not have sufficient follow-up data (144 [61 to 223] versus 165 [75 to 241] cells/μl), and the median (IQR) viral loads were similar (5.12 [4.6 to 5.6] among those included versus 5.16 [4.7 to 5.6] log10 copies/ml among those not included). All patients meeting inclusion criteria described above were analyzed. Of 1,982 patients, 890 (45%) initiated zidovudine, lamivudine, and efavirenz; 538 (27%) initiated zidovudine, lamivudine, and nevirapine; 206 (10%) initiated didanosine, stavudine, and an NNRTI; the remaining 348 (18%) initiated other three-drug, NNRTI-based regimens.
Detecting Virologic Failure at 6 and 12 Months
The analyses of response at 6 and 12 mo included 958 and 872 individual patients, respectively; 293 patients who met inclusion criteria for both endpoints were included in both analyses. Virologic failure was experienced by 235 of 958 (25%) patients at 6 mo and 229 of 872 (26%) patients at 12 mo according to our definition (Table 1). Factors associated with virologic failure are shown in Table 1. The median (IQR) number of days between cART initiation and follow-up CD4 count was 201 (158 to 241) at 6 mo and 353 (296 to 399) at 12 mo, and more than 95% of follow-up CD4 counts and viral loads were done on the same day. CD4 cell count increases were significantly smaller for those experiencing virologic failure (median [IQR] increase in cells/μl of 70 [6 to 145] versus 142 [72 to 251] at 6 mo [z = 8.4; p < 0.001, rank sum test] and 51 [−5 to 153] versus 184 [95 to 316] at 12 mo [z = 11.4; p < 0.001, rank sum test]). Adherence levels and CD4 count changes according to virologic response at 6 and 12 mo are shown in Figures 2–5.
Characteristics of Patients with and without Virologic Failure at 6 and 12 Months
Adherence values had greater overall accuracy for detecting current virologic failure at 6 and 12 mo compared to CD4 count changes (AUCs [95% CI] of 0.79 [0.76 to 0.83] versus 0.68 [0.64 to 0.72] at 6 mo and 0.85 [0.82 to 0.88] versus 0.75 [0.72 to 0.79] at 12 mo; χ2 = 20.1 and 20.2, respectively, p < 0.001 for both comparisons] (Figures 6 and 7). The bootstrapped 95% CIs for the differences between adherence and CD4 count AUCs were 0.11 (0.06 to 0.16) at 6 mo and 0.10 (0.05 to 0.14) at 12 mo. The adherence level that would result in the fewest unnecessary treatment changes if used at either time point (i.e., < 50%) had greater overall ability to identify patients with virologic failure than the CD4 count change level with the highest specificity at 6 mo (i.e., a CD4 count drop to pretreatment levels or below; χ2 = 6.6, p = 0.01 comparing AUCs) and at 12 mo (i.e., CD4 cell counts < 100 cells/μl at baseline and follow-up; χ2 = 8.0, p < 0.001). The superiority of adherence persisted if virologic failure was defined as a viral load > 10,000 copies/ml (6-mo AUC of 0.80 [0.75 to 0.84] versus 0.71 [0.66 to 0.76]; χ2 = 7.6, p = 0.005; 12-mo AUC of 0.87 [0.84 to 0.90] versus 0.78 [0.75 to 0.82]; χ2 = 14.6, p < 0.001). Test characteristics for CD4 count changes and adherence levels when used as tests for virologic failure in the first year of cART are shown in Tables 2 and 3, respectively.
Green and orange dots are observed adherence and CD4 + T cell count change values, respectively. Thus, the graphs represent the sensitivity and specificity that would result if each observed adherence or CD4 + T cell count change value were used as a diagnostic test for current virologic failure. The AUC was 0.79 (95% CI 0.76–0.83) for adherence and 0.68 (95% CI 0.64–0.72) for CD4 count change (p< 0.001, Chi-square test).
Green and orange dots are observed adherence and CD4 + T cell count change values, respectively. Thus, the graphs represent the sensitivity and specificity that would result if each observed adherence or CD4 + T cell count change value were used as a diagnostic test for current virologic failure. The AUC was 0.85 (95% CI 0.82–0.88) for adherence and 0.75 (95% CI 0.72–0.79) for CD4 count change (p< 0.001, Chi-square test).
Test Characteristics for CD4 Cell Count Changes for Identifying Patients with Virologic Failure after Initiating cART when Assessed at 6 or 12 Months
Test Characteristics for Adherence Levels for Identifying Patients with Virologic Failure after Initiating cART when Assessed at 6 or 12 Months
Detecting Breakthrough Viremia after Initial Virologic Suppression
A total of 1,101 patients met inclusion criteria for the analysis of breakthrough viremia. Of these patients, 151 (14%) had breakthrough viremia after an initial undetectable viral load (Table 4). The median (IQR) duration of follow-up for these patients was 648 d (533 to 721 d), or approximately 1.75 y.
Characteristics of Patients with and without Breakthrough Viremia after Initial Virologic Suppression on NNRTI-Based cART (n =1,101)
Only nine of 1,101 patients (1%) had CD4 cell count values that were persistently below 100 cells/μl, and this criterion was therefore not evaluated further. 54 (5%) and 42 (4%) of patients experienced a CD4 cell count drop to pretreatment levels or below, or to levels 50% or less than the maximum on treatment value, respectively. Both criteria were strongly associated with breakthrough viremia, as was adherence (Table 4).
There was no significant difference between adherence values and CD4 count changes from maximum on-treatment values to follow-up with respect to identification of patients with breakthrough viremia (AUCs [95% CIs], 0.68 [0.64 to 0.73] for CD4 counts versus 0.67 [0.62 to 0.72]; χ2 = 0.1, p > 0.5). The bootstrapped 95% CI for the difference between adherence and CD4 count AUCs was 0.01 (−0.06 to 0.07). The CD4 count criterion with the largest AUC was not significantly different from the adherence criteria with the largest AUC (a CD4 count drop of 20% or more from maximum on-treatment values versus an adherence level of < 90% [χ2 = 0.0, p > 0.5]) (Table 5). Similar to above, altering the definition of virologic failure did not significantly change the results (AUC [95% CI] 0.65 [0.58 to 0.72] versus 0.70 [0.65 to 0.76]; χ2 = 1.6, p = 0.21).
Test Characteristics for CD4 Cell Count Changes and Adherence Levels for Identifying Patients with Breakthrough Virologic Failure after Initial Response to cART (n =1,101)
Ability of Early Adherence Assessments to Identify Patients with Virologic Failure
The AUC for pharmacy refill adherence measured during the initial 3 mo after cART initiation for virologic failure at 6 mo was 0.72 (0.68 to 0.75), which was smaller than that resulting from the adherence assessment at 6 mo (χ2 = 31.8, p < 0.01), but similar (0.72 for adherence versus 0.68 for CD4 count change; χ2 = 2.0, p = 0.15) to that resulting from evaluation of the change in the CD4 count over the first 6 mo of cART. The AUC for pharmacy refill adherence measured over a 3 mo period ending approximately 3 mo prior to the 12 mo viral load assessment was 0.76 (0.73 to 0.80). Similar to above, although this early adherence assessment was statistically significantly less able to discriminate between those with and without virologic failure at 12 mo when compared to the adherence assessment using all 12 mo of adherence data (χ2 = 46.0, p < 0.001), the AUC was similar to that resulting from evaluation of the change in the CD4 count over the first 12 mo of cART (0.76 for adherence measured over a 3 mo period ending approximately 3 months prior to the 12 mo viral load assessment versus 0.75 for CD4 change calculated over the entire 12 months; χ2 = 0.39, p > 0.5). Use of days (rather than months) did not significantly alter the results (unpublished data).
Adherence and CD4 Counts Combined
Positive predictive values (PPVs) and the proportions of patients with virologic failure meeting each combined criterion are shown in Table 6. At 6 mo, combined criteria did not significantly increase PPVs above that resulting from adherence alone. For example, although a CD4 count decrease from pretreatment values and adherence < 50% at 6 mo increased the point estimate for PPV compared to the adherence level with the highest PPV at 6 mo (< 50%, PPV = 87% [77%–93%], Table 3), the confidence intervals overlapped and the sensitivity for virologic failure decreased (from 28% [22%–34%] to 12% [8%–17%]). Similar results were seen at 12 mo (Table 6). For example, an adherence level with the highest PPV at 6 mo (< 50%, Table 3) at 12 mo resulted in a PPV of 88% (79%–93%), which was increased by addition of a CD4 count decrease, but the confidence intervals overlapped and the sensitivity for virologic failure significantly decreased (from 37% [30%–43%] to 18% [14%–24%]).
PPVs of CD4 Count and Adherence Data Combined for Virologic Failure at 6 and 12 Months
Negative predictive values (NPVs) and the proportions of patients without virologic failure meeting each combined criterion are shown in Table 7. Use of an adherence threshold alone of 100% at both 6 and 12 mo resulted in NPVs of 91% (88%–93%) and 94% (91%–97%), respectively. Furthermore, 66% (61%–70%) and 42% (38%–46%) of all patients without virologic failure met this criterion at the 6- and 12-mo visits, respectively. Two criteria, adherence ≥ 70% or ≥ 80% and a CD4 count increase of ≥ 50 cells/μl at 6 mo maintained the point estimate for NPV above 90% while increasing the proportion of patients meeting that criterion at 6 mo (Table 7). At 12 mo, several criteria that maintained NPVs at or above 94% while significantly increasing the proportion of patients meeting that criterion were identified (indicated by footnotes in the “12 Mo” column in Table 7).
NPVs of CD4 Count and Adherence Data Combined for Virologic Failure at 6 and 12 Months
Discussion
These results demonstrate that adherence levels, as estimated by pharmacy claims data, can be at least as accurate as CD4 count changes for detection of virologic failure among patients receiving cART. This finding was consistent when evaluating patients at two time points during the first year and after initial virologic suppression, and was not dependent on the level of viremia used to define virologic failure. Because cART scale-up guidelines for resource-limited settings suggest use of CD4 count monitoring after cART initiation [7], these findings are relevant to ongoing antiretroviral treatment efforts in resource-limited settings.
A clinical implication of these results is that systematic monitoring of pharmacy refill adherence should be considered as an alternative to CD4 count monitoring for identification of patients with a high probability of virologic failure. As shown, the ROC curve analyses indicate that given a fixed level of sensitivity, specificity resulting from monitoring adherence will tend to be higher than that seen with WHO-advocated CD4 count changes in the first year and approximately equivalent thereafter. This translates into fewer patients with unnecessary regimen changes and fewer virologic failures missed. For example, based on the analysis of this cohort, if an adherence level of < 50% was used in place of a CD4 count decrease to pretreatment levels or below to identify virologic failure at 12 mo, the proportion of unnecessary regimen changes would decrease (from 6% to 2%), and the proportion of virologic failures identified would increase (from 28% to 37%). Furthermore, the fact that the ROC areas for adherence measured months prior to a viral load were relatively consistent with adherence assessments and CD4 count changes performed later indicates that adherence monitoring may be a useful approach for early identification of patients at high risk of future virologic failure. This result is intuitive given the causal pathway wherein poor adherence leads to virologic failure, which leads in turn to immunologic decline [30]. Importantly, however, poor adherence does not invariably result in virologic failure [12–14]. Therefore, although detection of a CD4 count decline would be unlikely to lead to a treatment change in the first year of cART, detection of adherence lapses early may enable targeted adherence interventions capable of preventing virologic failure [31,32] and prolonging time on less-expensive first-line cART. This contrasts with CD4 count monitoring, which inherently identifies virologic failure after it has already occurred.
How should these data be used clinically? For clinics that do not have CD4 counts or viral load monitoring capabilities, pharmacy-based adherence monitoring should be adopted in order to identify patients in need of adherence interventions. Alternatively, adherence monitoring could be used to pursue focused virologic or genotypic testing in settings where these assays are available to some but not all, or after cheaper assays become more widely available. The high NPVs resulting from adherence monitoring indicate that adherence data can be used to identify a relatively large group of patients who have a low probability of virologic failure. Moreover, combining adherence data with a simple CD4 count threshold (e.g., adherence < 90% and no CD4 count decrease at 12 mo), could identify a subset of patients at such low risk of virologic failure that virologic monitoring might be postponed. For example, clinics able to perform viral load assessments in all patients routinely could use adherence monitoring to guide decision-making on timing of these tests. For example, patients with perfect adherence and a CD4 count increase of more than 100 cells/μl at 6 mo could have their viral load assessed at 12 mo, or patients with adherence < 90% at 3 mo could have their viral load assessed at 6 mo (e.g., after an early adherence intervention). Although pharmacy refill adherence monitoring appears useful in these types of approaches, ultimately monitoring strategies must consider the resources available to the setting and should be informed by formal cost-effectiveness studies.
Our inclusion criteria were strict, yet comparison of patients included and excluded indicated that there were no clinically meaningful differences in the baseline CD4 counts or viral loads. Moreover, the estimates of the accuracy of CD4 count changes for virologic failure documented in this study concur with results presented from two other settings [17,33]. We do not therefore feel that significant bias resulted from patient exclusions. However, although the NNRTI-based cART regimens used and the immunologic responses were similar to those found in several other cohorts [17,21–23], this study was performed in a private-care setting, so the potential generalizability of the results should also be considered. For example, the adherence calculation was based on claims, and patients receiving cART at free clinics do not generally submit pharmacy claims. However, dates of actual refills were used to calculate intervals between dispensations. For this reason, an adherence calculation such as that performed here should be feasible in large public clinics where medications are often free. Furthermore, pharmacy claims data may overestimate true adherence because patients may not take all claimed medications. However, the goal of this study was to determine if pharmacy refill data, as observed in clinical practice, could be used as a test for virologic failure, and to compare this approach with CD4 count levels, which also suffer from measurement error and biological variability. In addition, alternative measures of adherence, including subjective measures that were not available to us [34,35], should be examined, as should accuracy of this approach in patients on non-NNRTI-based cART. Finally, the AUC for adherence assessments for virologic failure was lower in the analysis of breakthrough viremia compared to the AUCs for adherence assessments for virologic failure done at 6 and 12 months after HAART initiation. One possible explanation of this finding is that selection of resistance mutations, caused by subtle lapses in adherence not captured by pharmacy claims data, caused a decrease in the diagnostic accuracy of adherence over time. Patterns of adherence may influence the risk of virologic failure differently when patients initiate therapy compared to when their viral loads are suppressed below quantifiable levels. Since nearly 25% of all drug resistance occurs in patients with high levels of adherence [36], it would have been interesting to explore the relationship between virologic failure, resistance, and diagnostic accuracy of adherence in this setting, but genotyping data were unavailable.
Although CD4 counts inform when to start antiretroviral therapy and when to stop prophylaxis for opportunistic infections, these data indicate that guidelines such as those produced by the WHO [7] for monitoring patients receiving cART in resource-limited settings should include consideration of an adherence-based monitoring approach.
Furthermore, future research should examine this approach in other settings with variable testing capabilities and should identify and address possible barriers to operationalizing systematic monitoring of adherence. For example, a requirement of adherence monitoring based on pharmacy records is ready access to drug refill information as well as conversion of these data into an adherence metric at the time a patient is seen. Provision of adherence data to patients at the point of care, however, could be either simple or more complex. In Botswana, for example, patients present to providers with a paper pharmacy card on which dates of cART dispensation (and pill counts) are noted. Providers conceptually can calculate adherence directly using these cards. A more complex approach for clinics with computers would be to link pharmacy and patient care records electronically, so that a program would automatically supply the pharmacy refill adherence. The ability of adherence to identify patients at high risk of virologic failure early and to provide data on the behavior on which providers often wish to intervene should be considered a reason for clinics to organize these data in a way that can be used in simple algorithmic approaches to patient care.
Author Contributions
GPB, RG, IF, GM, and JBN designed the experiments and the study. MH collected data for the study. MH was responsible for data collection, preparation, extraction, and reporting for the Aid for AIDS Disease Management Programme. GPB, SB, and JC performed the statistical analyses. GPB wrote the first draft of the paper. All authors contributed to the analysis and interpretation of the data summarized in the manuscript, and all authors contributed to revising and finalizing the paper.
References
- 1.
[No author listed] (2005) Sources and Prices of Selected Medicines and Diagnostics for People Living with HIV/AIDS. World Health Organization, United Nations Children's Fund, Joint United Nations Programme on HIV/AIDS, Medecins Sans Frontieres.
- 2. D'Aquila RT, Johnson VA, Welles SL, Japour AJ, Kuritzkes DR, et al. (1995) Zidovudine resistance and HIV-1 disease progression during antiretroviral therapy. AIDS Clinical Trials Group Protocol 116B/117 Team and the Virology Committee Resistance Working Group. Ann Intern Med 122: 401–408.
- 3. Kozal MJ, Kroodsma K, Winters MA, Shafer RW, Efron B, et al. (1994) Didanosine resistance in HIV-infected patients switched from zidovudine to didanosine monotherapy. Ann Intern Med 121: 263–268.
- 4. Zaccarelli M, Tozzi V, Lorenzini P, Trotta MP, Forbici F, et al. (2005) Multiple drug class-wide resistance associated with poorer survival after treatment failure in a cohort of HIV-infected patients. AIDS 19: 1081–1089.
- 5. Hammer SM, Saag MS, Schechter M, Montaner JS, Schooley RT, et al. (2006) Treatment for adult HIV infection: 2006 recommendations of the International AIDS Society-USA panel. JAMA 296: 827–843.
- 6. Calmy A, Ford N, Hirschel B, Reynolds SJ, Lynen L, et al. (2007) HIV viral load monitoring in resource-limited regions: optional or necessary. Clin Infect Dis 44: 128–134.
- 7.
WHO (2006) Scaling up antiretroviral therapy in resource-limited settings: treatment guidelines for a public health approach. 2006 Revision. Geneva: WHO. Available: http://www.who.int/hiv/pub/guidelines/artadultguidelines.pdf. Accessed 23 April 2008.
- 8.
WHO (2005) CD4+ T-cell enumeration technologies: technical information. Geneva: WHO and UNAIDS. Available: http://www.who.int/diagnostics_laboratory/CD4_Technical_Advice_ENG.pdf. Accessed 14 April 2008.
- 9. Cohen J (2004) Monitoring treatment: at what cost. Science 304: 1936.
- 10. Montaner JS, Hogg RS (2007) Hungering for HAART. Ann Intern Med 146: 609–610.
- 11. Paterson DL, Swindells S, Mohr J, Brester M, Vergis EN, et al. (2000) Adherence to protease inhibitor therapy and outcomes in patients with HIV infection. Ann Intern Med 133: 21–30.
- 12. Nachega JB, Hislop M, Dowdy D, Chaisson R, Regensberg L, et al. (2007) Adherence to non-nucleoside reverse transcriptase-based hiv therapy and virologic outcomes. Ann Intern Med 146: 564–573.
- 13. Gross R, Bilker WB, Friedman HM, Strom BL (2001) Effect of adherence to newly initiated antiretroviral therapy on plasma viral load. AIDS 15: 2109–2117.
- 14. Bangsberg DR (2006) Less than 95% adherence to nonnucleoside reverse-transcriptase inhibitor therapy can lead to viral suppression. Clin Infect Dis 43: 939–941.
- 15. Low-Beer S, Yip B, O'Shaughnessy MV, Hogg RS, Montaner JS (2000) Adherence to triple therapy and viral load response. J Acquir Immune Defic Syndr 23: 360–361.
- 16. Grossberg R, Zhang Y, Gross R (2004) A time-to-prescription-refill measure of antiretroviral adherence predicted changes in viral load in HIV. J Clin Epidemiol 57: 1107–1110.
- 17. Bisson GP, Gross R, Strom JB, Rollins C, Bellamy S, et al. (2006) Diagnostic accuracy of CD4 cell count increase for virologic response after initiating highly active antiretroviral therapy. AIDS 20: 1613–1619.
- 18. Colebunders R, Moses KR, Laurence J, Shihab HM, Semitala F, et al. (2006) A new model to monitor the virological efficacy of antiretroviral treatment in resource-poor countries. Lancet Infect Dis 6: 53–59.
- 19. Chene G, Sterne JA, May M, Costagliola D, Ledergerber B, et al. (2003) Prognostic importance of initial response in HIV-1 infected patients starting potent antiretroviral therapy: analysis of prospective studies. Lancet 362: 679–686.
- 20. Nachega JB, Hislop M, Dowdy DW, Lo M, Omer SB, et al. (2006) Adherence to highly active antiretroviral therapy assessed by pharmacy claims predicts survival in HIV-infected South African adults. J Acquir Immune Defic Syndr 43: 78–84.
- 21. Ferradini L, Jeannin A, Pinoges L, Izopet J, Odhiambo D, et al. (2006) Scaling up of highly active antiretroviral therapy in a rural district of Malawi: an effectiveness assessment. Lancet 367: 1335–1342.
- 22. Wester CW, Kim S, Bussmann H, Avalos A, Ndwapi N, et al. (2005) Initial response to highly active antiretroviral therapy in HIV-1C-infected adults in a public sector treatment program in Botswana. J Acquir Immune Defic Syndr 40: 336–343.
- 23. Stringer JS, Zulu I, Levy J, Stringer EM, Mwango A, et al. (2006) Rapid scale-up of antiretroviral therapy at primary care sites in Zambia: feasibility and early outcomes. Jama 296: 782–793.
- 24. Severe P, Leger P, Charles M, Noel F, Bonhomme G, et al. (2005) Antiretroviral therapy in a thousand patients with AIDS in Haiti. N Engl J Med 353: 2325–2334.
- 25.
Hosmer DW, Lemeshow S (2000) Applied logistic regression. New York: Wiley. 373 p.
- 26. Hanley JA, McNeil BJ (1982) The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 143: 29–36.
- 27. Ohman EM, Granger CB, Harrington RA, Lee KL (2000) Risk stratification and therapeutic decision making in acute coronary syndromes. JAMA 284: 876–878.
- 28. Harrell FE Jr., Califf RM, Pryor DB, Lee KL, Rosati RA (1982) Evaluating the yield of medical tests. JAMA 247: 2543–2546.
- 29. Efron B, Tibshirani R (1986) Bootstrap methods for standard errors, confidence intervals, and other measures of statistical accuracy. Stat Sci 1: 54–77.
- 30. Dragsted UB, Mocroft A, Vella S, Viard JP, Hansen AB, et al. (2004) Predictors of immunological failure after initial response to highly active antiretroviral therapy in HIV-1-infected adults: a EuroSIDA study. J Infect Dis 190: 148–155.
- 31. Levy RW, Rayner CR, Fairley CK, Kong DC, Mijch A, et al. (2004) Multidisciplinary HIV adherence intervention: a randomized study. AIDS Patient Care STDS 18: 728–735.
- 32. Dieckhaus KD, Odesina V (2007) Outcomes of a multifaceted medication adherence intervention for HIV-positive patients. AIDS Patient Care STDS 21: 81–91.
- 33. Moore DM, Mermin J, Awor A, Yip B, Hogg RS, et al. (2006) Performance of immunologic responses in predicting viral load suppression: implications for monitoring patients in resource-limited settings. J Acquir Immune Defic Syndr 43: 436–439.
- 34. Nieuwkerk PT, Oort FJ (2005) Self-reported adherence to antiretroviral therapy for HIV-1 infection and virologic treatment response: a meta-analysis. J Acquir Immune Defic Syndr 38: 445–448.
- 35. Simoni JM, Kurth AE, Pearson CR, Pantalone DW, Merrill JO, et al. (2006) Self-report measures of antiretroviral therapy adherence: A review with recommendations for HIV research and clinical management. AIDS Behav 10: 227–245.
- 36. Bangsberg DR, Charlebois ED, Grant RM, Holodniy M, Deeks SG, et al. (2003) High levels of adherence do not prevent accumulation of HIV drug resistance mutations. AIDS 17: 1925–1932.