http://orcid.org/0000-0002-6590-8728
Figures
Abstract
Background
Dynamic HIV transmission models can provide evidence-based guidance on optimal combination implementation strategies to treat and prevent HIV/AIDS. However, these models can be extremely data intensive, and the availability of good-quality data characterizing regional microepidemics varies substantially within and across countries. We aim to provide a comprehensive and transparent description of an evidence synthesis process and reporting framework employed to populate and calibrate a dynamic, compartmental HIV transmission model for six US cities.
Methods
We executed a mixed-method evidence synthesis strategy to populate model parameters in six categories: (i) initial HIV-negative and HIV-infected populations; (ii) parameters used to calculate the probability of HIV transmission; (iii) screening, diagnosis, treatment and HIV disease progression; (iv) HIV prevention programs; (v) the costs of medical care; and (vi) health utility weights for each stage of HIV disease progression. We identified parameters that required city-specific data and stratification by gender, risk group and race/ethnicity a priori and sought out databases for primary analysis to augment our evidence synthesis. We ranked the quality of each parameter using context- and domain-specific criteria and verified sources and assumptions with our scientific advisory committee.
Findings
To inform the 1,667 parameters needed to populate our model, we synthesized evidence from 59 peer-reviewed publications and 24 public health and surveillance reports and executed primary analyses using 11 data sets. Of these 1,667 parameters, 1,517 (91%) were city-specific and 150 (9%) were common for all cities. Notably, 1,074 (64%), 201 (12%) and 312 (19%) parameters corresponded to categories (i), (ii) and (iii), respectively. Parameters ranked as best- to moderate-quality evidence comprised 39% of the common parameters and ranged from 56%-60% across cities for the city-specific parameters. We identified variation in parameter values across cities as well as within cities across risk and race/ethnic groups.
Conclusions
Better integration of modelling in decision making can be achieved by systematically reporting on the evidence synthesis process that is used to populate models, and by explicitly assessing the quality of data entered into the model. The effective communication of this process can help prioritize data collection of the most informative components of local HIV prevention and care services in order to reduce decision uncertainty and strengthen model conclusions.
Citation: Krebs E, Enns B, Wang L, Zang X, Panagiotoglou D, Del Rio C, et al. (2019) Developing a dynamic HIV transmission model for 6 U.S. cities: An evidence synthesis. PLoS ONE 14(5): e0217559. https://doi.org/10.1371/journal.pone.0217559
Editor: Lauren Cipriano, Western University, CANADA
Received: December 19, 2018; Accepted: May 14, 2019; Published: May 30, 2019
Copyright: © 2019 Krebs et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript and its Supporting Information files.
Funding: This study was funded by the National Institutes on Drug Abuse (NIDA), grant no. R01-DA041747 (BN), and received additional support from NIDA grants DA12568 and DA036297. BRS received additional support from the Center for Health Economics of Treatment Interventions for Substance Use Disorder, HCV, and HIV (P30DA040500). SAS is supported by a NIDA Method to Extend Research in Time (MERIT) award (R37-DA019829). The funders had no direct role in the conduct of the analysis or the decision to submit the manuscript. NIDA - https://www.drugabuse.gov/. CHERISH - https://cherishresearch.org/.
Competing interests: The authors have declared that no competing interests exist.
Introduction
In the United States, more than 1.1 million people were estimated to be living with HIV in 2015, including 162,500 (15%) people who had not been diagnosed [1]. Although the number of people living with HIV is increasing and access to antiretroviral medications is extending life expectancy [2], current political uncertainty related to health financing is straining resources and challenging public health departments to use available funding efficiently [3]. Further complicating these decisions is the fact that HIV epidemics tend to be heterogeneous across geographic regions [4–6]. In the United States, the majority of people living with HIV/AIDS (PLHIV) reside in large urban centers that have unique underlying epidemiological and structural features [7]. This heterogeneity across regional microepidemics necessitates prioritizing resources according to the greatest public health benefit, accounting for the local epidemiological and structural context [6, 8, 9].
Increasingly, mathematical models are being used to help set priorities to address HIV microepidemics internationally [10–13]. Dynamic HIV transmission models can estimate, within a causal framework, all relevant costs and benefits attributable to HIV care interventions over an extended time horizon [14]. Such models can be adapted for multiple settings, capturing the heterogeneity across settings and also estimating the potentially synergistic effects of combinations of public health interventions to treat and prevent HIV [15]. However, these models are often data intensive because they require context-specific information about the demographics of HIV-negative and infected populations, heterogeneous HIV risk behaviors and access to health services such as HIV testing and antiretroviral treatment (ART), among other factors. While efforts to collect and compile population-based health administrative and surveillance data are rapidly increasing, the availability of representative, high-quality data still varies substantially within and across countries [16–18].
Comprehensively reporting the evidence synthesis process and sources of data used in a model can help readers assess its validity and the credibility of its inferences. In addition, calibrating a model to match a jurisdiction’s microepidemic over a stated period is a necessary condition for ensuring a model’s external validity [19]. Despite the importance of the quality of evidence entered into a model, there are no explicit guidelines for reporting the evidence synthesis process for models used in health economic evaluation [20–23]. While the efficiency and appropriateness of systematic searching for every model parameter has been questioned, it has been suggested that search approaches should reflect the complexity of the evidence [23, 24]. A recent modeling study for Vietnam [25] described the estimation techniques and triangulation methods that were used to approximate parameter values. The study drew on national surveillance data and behavioral surveys and provided a greater level of transparency in reporting on input data sources than had been seen elsewhere. Comprehensively and transparently reporting the evidence used in mathematical models improves reproducibility and allows it to be updated more easily as newer and higher-quality data become available. More importantly, this reporting process can reveal the areas of greatest uncertainty for sensitivity analysis, and, through value of information analysis [26], identify areas where additional surveillance data are worth collecting.
We aim to provide a comprehensive description of an evidence synthesis process and reporting framework that can be used to populate and calibrate a dynamic, compartmental HIV transmission model for six US cities. We hope to maximize the transparency of our model so that interested parties can review and evaluate its structure and equations, as well as the generating process and assumptions for all parameters (25), in order to promote the use of modeling recommendations in decision making to address city-level HIV microepidemics.
Methods
Model structure
We adapted a previously published dynamic, compartmental HIV transmission model [27–30] to simulate the HIV microepidemics in six US cities: Atlanta, Georgia; Baltimore, Maryland; Los Angeles, California; Miami, Florida; New York City, New York; and Seattle, Washington (boundaries defined in S1 Supplement). We selected these six cities because they represent nearly one-quarter of the US population of PLHIV [31] and because they have extensive epidemiological and structural differences in their public health responses to HIV [7]. For each city, the adult population aged 15–64 was stratified on the basis of gender (male or female), race/ethnicity (black/African American, Hispanic/Latino, and non-Hispanic white/others), and HIV risk behavior type (men who have sex with men (MSM), people who inject drugs (PWID), MSM who inject drugs (MWID), and heterosexual (HET)). MSM, MWID, and HET were further stratified into subgroups based on HIV sexual risk behavior intensity (high vs. low), and PWID and MWID were categorized based on whether they were receiving opioid agonist treatment (OAT).
Individuals within each of these 42 strata (MSM: 6 groups; MWID: 12 groups; PWID: 12 groups; HET: 12 groups) progress through the model according to the health states outlined in Fig 1. Prior to HIV infection, HIV-negative individuals can be screened for HIV (screened in the past 12 months), and screened MSM or MWID can take pre-exposure prophylaxis (PrEP). HIV transmission can occur through three modes: heterosexual contact, homosexual contact and needle sharing. We specified sexual mixing assortativity between risk groups and race/ethnicity to determine the proportion of sexual contacts within the same group, and varied the level of assortativity across cities [32, 33]. Following HIV infection, individuals transition through various stages beginning with acute infection (three months). They are then classified as infected but not diagnosed, diagnosed but ART-naïve, and on- or off-ART, and partitioned according to CD4 cell count (CD4 ≥ 500, 200–499, and <200). Health state transitions occur at monthly intervals and transition to death is a possibility from each of the health states depicted, with varying probabilities.
For each city, the adult population aged 15–64 was stratified into compartments on the basis of (1) gender (male or female), (2) race/ethnicity (black/African American, Hispanic/Latino, and non-Hispanic white/others), and (3) HIV risk behavior type (men who have sex with men (MSM), people who inject drugs (PWID), MWID, and heterosexual (HET)). MSM, MWID, and HET were further stratified into subgroups based on HIV sexual risk behavior intensity (high vs low), and PWID and MWID were categorized based on whether they were receiving opioid agonist treatment (OAT). Individuals within each of these 42 strata (MSM: 6 groups, MWID: 12 groups; PWID: 12 groups; HET: 12 groups) progress through the model according to the 19 health states illustrated above. Prior to HIV infection, HIV-negative individuals can be screened for HIV (screened in past 12 months), and screened MSM or MWID can take pre-exposure prophylaxis (PrEP). HIV transmission can occur through three modes: heterosexual contact, homosexual contact, and needle-sharing. We specified the pattern of sexual mixing between risk groups and race/ethnicity, where assortativity determines the proportion of sexual contacts within the same group, and we varied the level of assortativity across cities (28). Following HIV infection, individuals transition through acute infection (3 months), then are classified as infected but not diagnosed, diagnosed but ART-naïve, and on- or off-ART, and partitioned according to CD4 cell count (CD4 ≥ 500, 200–499, and <200). Health state transitions occur at monthly intervals, with transition to death a possibility from each of the health states depicted, with varying probabilities.
Data requirements
We organized the data needed for the model into six model parameter categories: (i) initial HIV-negative and HIV-infected population estimates; (ii) parameters used to calculate the probability of HIV transmission; (iii) screening, diagnosis, treatment and HIV disease progression; (iv) HIV prevention programs, including syringe service programs (SSP), OAT, and PrEP; (v) the costs of medical care for HIV-negative and HIV-infected individuals; and (vi) health utility weights for each stage of HIV disease progression.
Each parameter in the model required a point estimate and range to facilitate model calibration and probabilistic sensitivity analysis according to best practice guidelines in model-based cost-effectiveness analysis [34, 35].
In addition to evidence informing model parameters, we required annual city-specific data for at least two time points to be used as calibration and validation targets for comparison of model projections [36, 37]. We chose the model calibration period (2012–2015) according to the availability of city-level surveillance data (stratified by gender, race/ethnicity, and risk group) for critical clinical and epidemiological endpoints characterizing the course of each HIV microepidemic [34].
Evidence synthesis strategy
We first identified parameter estimates that we determined to be common across cities and generalizable across city-level microepidemics. The remaining parameters required context-specific data to adequately characterize the population mix, HIV risk behaviors and health care utilization patterns for a given city. We divided our search strategy into two parts: (i) identifying a rank order of a priori potential data sources for each model parameter category; and (ii) selecting the best data to use, given additional factors and constraints (S1 Supplement).
The best possible data source for each parameter depended on factors unique to each parameter category. For example, the most accurate and reliable source for total population numbers was city-level census data, while the best source for ART effectiveness estimates came from randomized controlled trials. For each non-city-specific (common) parameter, we selected source data based on study quality, how well a study matched the ideal study type for a given model parameter, and recency of the evidence. For city-specific parameters, we selected source data based on geographic representativeness and stratification level relative to our model requirements and recency of the evidence. We assessed recency according to evidence type as we required more up-to-date surveillance data for initial populations and calibration targets in comparison to other non-city-specific evidence such as efficacy data from RCTs or untreated HIV disease progression. When necessary, parameter estimates and ranges were derived from triangulation, defined as using numbers from multiple sources and/or from the same source but requiring additional assumptions to match our model’s level of stratification.
We used several search methods to identify evidence sources for the disparate data types, including searches in bibliographic databases (PubMed searches for (ii) parameters used to calculate the probability of HIV transmission conducted for all articles published prior to May 31, 2017; searches for (iii) Screening, diagnosis, treatment and HIV disease progression conducted for all articles published prior to February 8, 2018; searches for (iv) HIV Prevention Programs conducted for all articles published prior to October 26, 2018; and searches for (vi) Health utility weights conducted for all articles published prior to August 30, 2017), non-database searches, “snowballing” (such as searching references from key sources to identify further sources) (Google Scholar snowball searches for (ii) Parameters used to calculate the probability of HIV transmission conducted for all articles published prior to May 31, 2017), and local surveillance reports [23, 24, 38]. Where necessary due to a paucity of available published data, we sought out large and representative databases that could be used for primary analysis to further augment our evidence synthesis.
Ranking data quality
The quality of each parameter was determined using context- and category-specific criteria, incorporating an adapted version of the Oxford Centre for Evidence-based Medicine–Levels of Evidence scale for common parameters (S1 Supplement) [39]. We ranked common parameter inputs according to the best quality of evidence that could be used to inform a given model parameter category. Best quality indicated a perfect match, moderate quality indicated that the evidence did not match perfectly or required some triangulation, and lowest-quality indicated that we derived parameter inputs by assumption or by another low quality evidence source.
For city-specific model parameters, we ranked the inputs according to how closely the evidence mapped onto the model stratification. Best quality indicated that the evidence data mapped onto the model parameter inputs by city perfectly (e.g., population-level data acquired since 2010 at the city level and stratified by risk group, gender, and race/ethnicity), moderate quality indicated that the evidence was stratified by city or region, with some level of population stratification or other triangulation, and lowest quality indicated that the evidence was at the national level, aggregated across population strata, or derived from expert opinion/assumption (e.g., aggregate data acquired prior to 2000 at the national-level).
All quality rankings were independently assessed by at least two team members and discrepancies were resolved through discussion and consensus among team members. Finally, missing city-level parameter values were assigned using a standardized algorithm to prioritize best-available data in surveillance and peer-reviewed literature at the (i) state, (ii) regional, or (iii) national level.
Data verification
Where the available data was less than ideal in at least two ways (e.g., potentially outdated according to the parameter category, geographically non-specific, or lacking stratification by gender, risk group or race/ethnicity), we posed explicit questions to our scientific advisory committee, a collection of city-specific experts, to confirm use of the best-available data or gain access to data otherwise unavailable publicly (e.g., current studies underway, disaggregated data from surveillance and other regularly produced reports). We prepared a web-based survey specific to each of the six cities. For each parameter in question, we provided the specific definition of the parameter and the best publicly available data to populate it. Scientific advisory committee members were asked to (i) identify additional sources that we had overlooked or that were not in the public domain but could be made available to the study team, (ii) rate their confidence in proposed triangulation methods to estimate parameter values, (iii) explain their rated responses, and (iv) where possible, share the survey with other knowledgeable public health experts in their professional networks (S1 Supplement).
Results
We identified 1,667 parameters needed to populate our dynamic, compartmental HIV transmission model (Table 1). Of these, 1,517 (91%) were unique to each city and the other 150 (9%) were common for all cities. The proportion of model parameters that composed each of the six model parameter categories varied extensively (Fig 2).
The boxes are proportionally scaled to the corresponding model parameter category sizes. Model parameter category labels: Population estimates ‒ 1. Initial HIV-negative and HIV-infected population estimates; HIV transmission ‒ 2. Parameters used to calculate the probability of HIV transmission; Treatment and HIV disease progression ‒ 3. Screening, diagnosis, treatment and HIV disease progression; Prevention ‒ 4. HIV prevention programs, including syringe service programs (SSP), OAT, and PrEP; Costs ‒ 5. The costs of medical care for HIV-negative and HIV-infected individuals; and QALYs ‒ 6. Health utility weights for each stage of HIV disease progression. ART: Antiretroviral treatment; All Pop.: Census population estimates; QALYs: Quality-adjusted life-years; Mixing: Sexual mixing patterns.
To inform each parameter’s point estimate and range, we synthesized evidence from 59 peer-reviewed publications and 24 public health and surveillance reports (Table 2) and executed primary analyses using 11 data sets (Table 3). Parameters ranked as best- to moderate-quality evidence comprised 39% of the 150 common parameters (Table 1) and 13% were directly estimated in the literature (Table 4). City-specific parameters that were ranked as best- to moderate-quality evidence ranged from 56% of the parameters for Baltimore to 60% for New York City (Table 1).
Results of data verification
The city-specific surveys that were sent to our scientific advisory committee contained questions about four of the six model parameter categories, including questions regarding population size estimates for HIV risk groups, parameters used to calculate the probability of HIV transmission, ART dropout rates and HIV prevention programs (S1 Supplement). Each city had at least one scientific advisory committee representative respond, and two cities had multiple respondents participate. Responses helped guide triangulation methods, and updated parameter estimates were re-sent to scientific advisory committee members for final review so that they could see where their responses were incorporated and how parameter values were used in calibration and/or sensitivity analysis.
Evidence synthesis for model parameter categories
Key results of our evidence synthesis are highlighted by model parameter category in this section. A detailed description of the derivation of each individual parameter is provided in the supplementary material (S1 Supplement), and descriptions of all datasets used in primary analysis can be found in Table 3 and in the supplementary material (S1 Supplement).
Initial population estimates and population dynamics.
A majority of the model’s parameters (1,075; 64%) were for population estimates and population dynamics. We derived the necessary evidence from 13 public health and surveillance reports [85–98] and 9 peer-reviewed publications [40, 99–106] and from primary analyses of 5 datasets [42, 107–110] (Table 2). More than two-thirds of the city-specific evidence used for the 558 parameters (38% of all parameters) that established the size of PLHIV populations in the model were of best- or moderate-quality (Table 1). However, the limited evidence available to determine the proportion of HIV-infected individuals in the acute stage of HIV disease progression was of low-quality (Table 4), and these parameters (n = 84) represented half of all parameters common across cities (Table 1). Approximately one-quarter of the city-specific parameters that determined the probabilities of mortality (n = 372) from each health state were of best- or moderate-quality, including primary analyses of data from The HIV Research Network (HIVRN) that were used to derive mortality rates for PLHIV receiving ART [107, 111].
Parameters used to calculate the probability of HIV transmission.
We synthesized evidence from 29 peer-reviewed publications [41, 43, 45–60, 65–69, 112–117] (Table 2) and conducted primary analyses of 6 datasets [42, 108, 110, 118–120] (Table 3) to derive the parameters that determined the probability of HIV transmission (n = 224; 13% of total parameters). The probabilities for HIV transmission per shared injection or sexual act and the effectiveness of HIV-related interventions (i.e., condom use, OAT, SSP and ART)–all common across cities–were derived from the peer-reviewed literature. All common parameters (n = 41) were of best- or moderate-quality (Table 1), and approximately a third (n = 7, 30%) were directly estimated (Table 4). In contrast, best- or moderate-quality evidence that informed city-specific sexual risk behavior parameters (n = 157) ranged from 8% to 19%, and evidence for sexual mixing pattern parameters (n = 12) was of low-quality across all cities (Table 1). Estimates of sexual risk behavior were obtained from National HIV Behavioral Surveillance (NHBS) data for MSM and PWID [42, 110] and we used region-specific population-based data from the National Survey of Family Growth (NSFG) for heterosexuals [108]. To determine ranges used in sensitivity analyses and calibration, we supplemented this evidence with primary analyses of data from the AIDS Linked to IntraVenous Experience (ALIVE) PWID cohort study [120] and from Project AWARE [119]. Lastly, we estimated injection risk behavior using NHBS data [42].
Screening, diagnosis, treatment and HIV disease progression.
Screening, diagnosis, treatment and HIV disease progression parameters represented 18% (n = 312) of all model parameters and were derived from 5 peer-reviewed publications [71, 72, 111, 121–123] and 6 public health and surveillance reports [86, 88, 90, 96–98] (Table 2) and from primary analyses using 6 data sets [42, 107, 109, 110, 124, 125] (Table 3). HIV testing rates (n = 42) were derived from primary analyses of sample data from NHBS [42, 110], the US Centers for Disease Control and Prevention’s (CDC) Behavioral Risk Factor Surveillance System [109], and the New York City Community Health Survey [125]. Notably, best- or moderate-quality evidence for stratified population-level testing rates was sparse (Table 1). In the absence of city-specific ART data, we used corresponding regional HIVRN data to obtain rates of ART initiation (n = 84) and re-initiation (n = 18) [107]. We also used HIVRN data with continuous-time multi-state Markov models to populate parameters specific to HIV disease progression rates while PLHIV are on ART (n = 108) and in relation to ART dropout rates (n = 54). The rates of HIV testing and ART dropout varied extensively across cities and across risk groups and races/ethnicities within cities (Fig 3). Rates of ART initiation were supplemented using analyses of Medical Monitoring Project (MMP) data [124], and disease progression off-ART was estimated using peer-reviewed literature [71] (Table 4).
MSM: Men who have sex with men; PWID: People who inject drugs; HET: Heterosexuals; ART: Antiretroviral treatment; F: Female; M: Male.
HIV prevention programs.
Parameters for HIV prevention programs (n = 23) were derived by combining evidence from 10 local, state and national sources [77, 126–134], 7 peer-reviewed articles [73, 74, 135–139], 2 publicly accessible data sources [126, 127] (Table 2). Availability of local data sources to populate syringe distribution parameters varied greatly, and we found extensive variation across cities in relation to the availability of syringes per 1,000 PWID (Fig 4). We used state-level data from the Substance Abuse and Mental Health Services Administration combined with evidence from the peer-reviewed literature to derive the number of PWID and MWID receiving OAT with either methadone or buprenorphine [126, 127, 135, 137]. Common parameters for the protective effects of HIV prevention programs were of good quality and often directly estimated in the peer-reviewed literature (Table 4). Lastly, we used AIDSvu data to determine pre-exposure prophylaxis (PrEP) uptake [134].
Costs of medical care.
To quantify health resource use cost parameters for infected (n = 24) and HIV-negative (n = 2) individuals, we included evidence from 4 public health and surveillance reports [140–143] and 4 peer-reviewed publications [28, 144–146] and conducted primary analysis of 1 dataset[147] (Table 2). For each city, we used corresponding regional HIVRN patient level utilization data and corresponding unit costs to estimate quarterly health care costs for HIV-infected individuals (Table 3). For HIV-negative individuals, we stratified cost estimates for MSM and HET risk groups from regional Medical Expenditure Panel Survey (MEPS) data [140] and used a multiplier to estimate costs for PWID [28].
Health utility weights.
We used common health utility weights across all six cities to calculate quality adjusted life years (QALYs) parameters (n = 30), synthesizing evidence from 4 peer-reviewed articles (Table 4). Estimates were derived from a nationally representative sample of QALY estimates [82], a meta-analysis [83], and a study that used a sample of Veterans Affairs members to estimate the change in health-related quality of life when patients were diagnosed and became aware of their HIV status [79, 82, 83]. We also incorporated evidence from the literature to establish a framework for relevant health states in HIV infection and disease progression [78]. Evidence from two additional studies was used to update the weights used for individuals receiving treatment in the modern era of highly active ART [80, 81]. Lastly, we used QALY weight multipliers for PWID based on whether or not they were receiving OAT [69, 84]. While these sources were the best-available evidence for health utility weights, the majority were published prior to 2007. Notably, health utility weights among PLHIV may have changed over time as a result of modern advances in treatment and HIV care.
Calibration and validation targets.
We identified 3 sets of calibration targets (17 targets in total), including stratified indicators of the annual number of new HIV diagnoses per year, the total number of diagnosed PLHIV and the annual number of all-cause deaths among PLHIV (S1 Supplement). These were representative of some of the best-quality data available and were also important to long-term clinical and epidemiological projections of city-level microepidemics, consistent with guidelines on selecting calibration and validation targets in cost-effectiveness analysis [19]. Furthermore, point estimates and ranges of HIV prevalence were used as validation targets to ensure external validity.
Discussion
We have provided a comprehensive description of an extensive evidence synthesis process that is required to populate a dynamic, compartmental HIV transmission model for six US cities. We identified differences across cities in the quality and representativeness of evidence available to inform our model. However, we identified consistency in the lack of availability of best-quality local administrative data that are critical to assess health system performance, particularly in relation to population-level rates of HIV testing and ART engagement. Nonetheless, our findings, which used the best-available evidence, highlight fundamental differences across settings related to rates of health system engagement and access to HIV prevention programs. The modeling of targeted, locally-oriented combination implementation strategies is necessary to determine how scarce resources should be allocated to interventions that can provide the greatest value for money in a given microepidemic. Our findings emphasize the need for increased public health efforts to measure and monitor the most informative components of local HIV prevention and care services, including the delivery, uptake and effect of localized HIV programs.
Reviews of health economic models in specific disease areas typically focus on differences in model structures and projected outcomes, with limited discussion of how the differences in the quality of input data can function as an explanation for variations across outputs [148–152]. Failure to report the sources and quality of model parameters, or reporting evidence directly from other modelling exercises without assessing the quality and representativeness of the inputs, can limit the interpretability of a model thereby eroding the confidence of its recommendations [153]. Cooper et al. (2007) discussed three practical issues and methodological challenges related to the use of evidence in health decision models: (i) defining and identifying ‘relevant’ evidence, (ii) assessing the quality and relevance of different sources of evidence, and (iii) synthesizing the evidence for use in modeling exercises [24]. Their paper also provided practical recommendations to address these challenges. Namely, a) describe the search method and selection process used to identify ‘relevant’ evidence per parameter; b) evaluate the quality and representativeness of the data retrieved; and c) pool evidence using explicit criteria where applicable [24]. Decision makers must be able to interpret mathematical models to use them to develop and evaluate effective HIV responses [154]. We documented our evidence synthesis process for six different cities as comprehensively as possible for transparency and reproducibility. We hope this effort promotes the use of modeling recommendations in decision making processes that address city-level HIV microepidemics.
The sustainability of an effective and efficient HIV response is critical to the control of local microepidemics [155–157]. For instance, the benefits of a treatment intervention that increases ART engagement might only be maximized with a sufficient level of HIV testing. Modeling recommendations promoting locally-oriented combination implementation strategies depend on evidence from local health systems. Reliable evidence of interactions between PLHIV and local health systems provided by surveillance and administrative data would greatly enhance the validity of modeling recommendations. Furthermore, and despite a paucity of behavioral data available describing sexual risk behavior and race/ethnicity mixing, recent evidence suggests that racial assortativity alone cannot adequately explain observed disparities in HIV incidence [158]. However, a meaningful share of this disparity can be explained by differential ART engagement by race [158]. This discrepancy further highlights the need for an improved use of routinely collected surveillance data (e.g., laboratory viral load monitoring can be used as a reasonable proxy of ART engagement) to allow for a better understanding of how to improve HIV care. Similarly important for locally-oriented modeling recommendation, city-level estimates of population sizes for PWID and MWID risk groups are either completely lacking or in critical need of updating [102]. Varying assumptions about risk groups in the modeling of epidemic dynamics can alter cost-effectiveness conclusions and intervention recommendations despite good model calibration [159], suggesting the need for a careful assessment of the potential value of collecting data about subpopulations that can have a disproportionate impact on local microepidemics. These examples of imperfect or missing data underline the importance of the data collection efforts of the US CDC’s NHBS and MMP, which provide behavioral information about people at risk of HIV and disease and treatment status of PLHIV [42, 110, 160].
Systematically conducting one-way sensitivity analyses and probabilistic sensitivity analysis to quantify the uncertainty in model recommendations resulting from parameters derived from evidence of poorer quality or representativeness is critical for further information gathering [34]. Value of information analysis [161] should guide the identification of influential parameters requiring additional research to reduce uncertainty in the decision making process. This exercise can also help define the collection of city-specific HIV surveillance data [162]. Data source identification and parameter estimate derivation should therefore be reported in sufficient detail to allow readers with the necessary expertise to perform a detailed evaluation of the model and possibly replicate it [19]. Ultimately, the development of formal guidelines pertaining to the evidence synthesis process and how it informs decision models should result in increased guidance for those engaged in the reporting process. These guidelines would include the creation of a standardized checklist that emulates the Consolidated Health Economic Evaluation Reporting Standards [163]. Publishers now routinely require this kind of checklist in cost-effectiveness studies.
This comprehensive evidence synthesis process had several potential limitations. First, the search for evidence sources was not systematic; however, we used a systematic structured identification strategy [24]. Furthermore, we included best-quality evidence sources from a narrative review of high-impact, current and diverse HIV models to further inform our initial identification process [164] in order to mitigate the potential correlation between sources. Second, we have not assessed the impact that the uncertainty in poorer quality parameters could have on model recommendations since this was beyond the scope of this study but rather report these findings in work elsewhere [32]. Third, despite a data verification process involving a scientific advisory committee composed of city-specific experts that helped to resolve instances where evidence was of poor representativeness, the number of respondents involved was low. Future evidence syntheses could benefit from a broad inclusion of public health officials. Lastly, given the relative wealth of surveillance data sources in the United States, the extent to which this evidence synthesis process could be replicated needs to be assessed on a case-by-case basis. As the possibilities for exercises of similar scope continue to grow in other regions and disease areas, our reporting framework can bolster future efforts.
Better integration of modelling in decision making can be achieved by systematically reporting on the evidence synthesis process that is used to populate models and by explicitly assessing the quality of data. The effective communication of this process can help prioritize data collection of the most informative components of local HIV prevention and care services in order to reduce decision uncertainty and strengthen model conclusions.
Supporting information
S1 Supplement.
Supporting information includes descriptions of city boundaries (Supplement A), search strategy for model inputs (Supplement B), derivation of all model parameters (Supplement C), description of data sets used for primary analysis (Supplement D), derivation of model calibration/validation targets and PSA probability distributions (Supplement E), and the data verification survey/results from our scientific advisory committee (SAC) (Supplement F).
https://doi.org/10.1371/journal.pone.0217559.s001
(PDF)
S2 Supplement.
Supplement Tables Supporting information includes (i) excel file of values, ranges and PSA distributions for all model parameters, (ii) excel file of values and ranges for calibration/validation targets for all cities, and (iii) pdf of full survey given to SAC members for data verification.
https://doi.org/10.1371/journal.pone.0217559.s002
(ZIP)
Acknowledgments
The authors thank Qian An, Linda Beer, Sarah Braunstein, Dita Broz, Isaedmarie Febo-Vázquez, Teresa Finlayson, Emma L. Frazier, Sara Glick, Sarah Jefferies, Ricky Rosales, Luke Shouse and Cyprian Wejnert for support in obtaining access to data.
The Localized HIV Modeling Study Group is composed of:
Carlos Del Rio, MD, Hubert Department of Global Health, Emory Center for AIDS Research, Rollins School of Public Health, Emory University
Julia Dombrowski, MD, primary with Department of Medicine, Division of Allergy & Infectious Disease, adjunct in Epidemiology, University of Washington
Daniel J Feaster, PhD, Center for Family Studies, Department of Epidemiology and Public Health, Leonard M. Miller School of Medicine, University of Miami
Kelly Gebo, MD, Bloomberg School of Public Health, Johns Hopkins University
Matthew Golden, MD, primary with Department of Medicine, Division of Allergy & Infectious Disease, adjunct in Epidemiology, University of Washington
Reuben Granich, MD, International Association of Providers of AIDS Care
Thomas Kerr, PhD, British Columbia Center on Substance Use; Faculty of Medicine, University of British Columbia
Gregory Kirk, PhD, Bloomberg School of Public Health, Johns Hopkins University
Brandon Marshall, PhD, Department of Epidemiology, Brown School of Public Health, Rhode Island, United States
Shruti H Mehta, PhD, Bloomberg School of Public Health, Johns Hopkins University
Lisa Metsch, PhD, Department of Sociomedical Sciences, Mailman School of Public Health, Columbia University
Julio Montaner, MD, BC Centre for Excellence in HIV/AIDS; Faculty of Medicine, University of British Columbia
Bohdan Nosyk, PhD, BC Centre for Excellence in HIV/AIDS; Faculty of Health Sciences, Simon Fraser University
Bruce R Schackman, PhD, Department of Healthcare Policy and Research, Weill Cornell Medical College
Steven Shoptaw, PhD, Centre for HIV Identification, Prevention and Treatment Services, School of Medicine, University of California Los Angeles
William Small, PhD, BC Centre for Excellence in HIV/AIDS; Faculty of Health Sciences, Simon Fraser University
Steffanie A Strathdee, PhD, School of Medicine, University of California San Diego
References
- 1. Centers for Disease Control and Prevention (CDC). Estimated HIV incidence and prevalence in the United States, 2010–2015. HIV Surveillance Supplemental Report. 2018;23(1).
- 2. Katz I, Maughan-Brown B. Improved life expectancy of people living with HIV: who is left behind? The Lancet HIV. 2017;4(8):e324–e6. pmid:28501496
- 3. Emanuel EJ. How can the United States spend its health care dollars better? Journal of the American Medical Association. 2016;316(24):2604–6. pmid:28027348
- 4. Wand H, Whitaker C, Ramjee G. Geoadditive models to assess spatial variation of HIV infections among women in local communities of Durban, South Africa. International Journal of Health Geographics. 2011;10(28).
- 5. Rebeiro P, Gange S, Horberg M, Abraham A, Napravnik S, Samji H, et al. Geographic variations in retention in care among HIV-infected adults in the United States. PLoS One. 2016;11(1):e0146119. pmid:26752637
- 6. Cuadros D, Li J, Branscum A, Akullian A, Jia P, Mziray E, et al. Mapping the spatial variability of HIV infection in Sub-Saharan Africa: Effective information for localized HIV prevention and control. Sci Rep. 2017;7:9093. pmid:28831171
- 7. Panagiotoglou D, Olding M, Enns B, Feaster D, del Rio C, Metsch L, et al. Building the case for localized approaches to HIV: structural conditions and health system capacity to address the HIV/AIDS epidemic in six US cities. AIDS and Behavior. 2017.
- 8. Anderson S, Cherutich P, Kilonzo N, Cremin I, Fecht D, Kimanga D, et al. Maximising the effect of combination HIV prevention through prioritisation of the people and places in greatest need: a modelling study. Lancet. 2014;384(9939):249–56. pmid:25042235
- 9.
Office of National AIDS Policy. National HIV/AIDS Strategy for the United States: Updated to 2020. 2015.
- 10. Wiseman V, Mitton C, Doyle-Waters M, Drake T, Conteh L, Newall A, et al. Using economic evidence to set healthcare priorities in low-income and lower-middle-income countries: A systematic review of methodological frameworkds. Health Economics. 2016;25(S1):140–61.
- 11. Cleary S, Mooney G, McIntyre D. Equity and efficiency in HIV-treatment in South Africa: the contribution of mathematical programming to priority setting. Health Economics. 2009;19(10):1166–80.
- 12. Granich R, Gilks C, Dye C, De Cock K, Williams B. Universal voluntary HIV testing with immediate antiretroviral therapy as a strategy for elimination of HIV transmission: a mathematical model. Lancet. 2009 373(9657):1077–8.
- 13. Long EF, Brandeau ML, Owens DK. The Cost-Effectiveness and Population Outcomes of Expanded HIV Screening and Antiretroviral Treatment in the United States. Ann Intern Med. 2010;153(12):778–+. ISI:000285453700027. pmid:21173412
- 14. Garnett G, Cousens S, Hallett T, Steketee R, Walker N. Mathematical models in the evaluation of health programmes. Lancet. 2011;378(9790):515–25. pmid:21481448
- 15. Delva W, Wilson DP, Abu-Raddad L, Gorgens M, Wilson D, Hallett TB, et al. HIV treatment as prevention: principles of good HIV epidemiology modelling for public health decision-making in all modes of prevention and evaluation. PLoS Medicine. 2012;9(7):e1001239. pmid:22802729
- 16. Frieden T. Evidence for health decision making—beyond randomized, controlled trials. N Engl J Med. 2017;377:465–75. pmid:28767357
- 17. Garrison LJ, Neumann P, Erickson P, Marshall D, Mullins C. Using real-world data for coverage and payment decisions: The ISPOR real-world data task force report. Value in Health. 2007;10(5):326–35. pmid:17888097
- 18. Jit M, Brisson M, Portnoy A, Hutubessy R. Cost-effectiveness of female human papillomavirus vaccination in 179 countries: a PRIME modelling study. The Lancet Global Health. 2014;2(7):e406–e14. pmid:25103394
- 19. Eddy D, Hollingworth W, Caro J, Tsevat J, McDonald K, Wong J, et al. Model transparency and validation: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-7. Med Decis Making. 2012;32(5):733–43. pmid:22990088
- 20. Cooper N, Coyle D, Abrams K, Mugford M, Sutton A. Use of evidence in decision models: an appraisal of health technology assessments in the UK since 1997. J Health Serv Res Policy. 2005;10(4):245–50. pmid:16259692
- 21. Zechmeister-Koss I, Schnell-Inderst P, Zauner G. Appropriate evidence sources for populating decision analytic models within health technology assessment (HTA): a systematic review of HTA manuals and health economic guidelines. Med Decis Making. 2014;34(3):288–99. pmid:24135150
- 22. Paisley S. Classification of evidence in decision-analytic models of cost-effectiveness: a content analysis of published reports. International Journal of Technology Assessment in Health Care. 2010;26(4):458–62. pmid:20923588
- 23. Paisley S. Identification of Evidence for Key Parameters in Decision-Analytic Models of Cost Effectiveness: A Description of Sources and a Recommended Minimum Search Requirement. PharmacoEconomics. 2016;34(6):597–608. pmid:26861793
- 24. Cooper N, Sutton A, Ades A, Paisley S, Jones D, Working Group on the Use of Evidence in Economic Decision Models. Use of evidence in economic decision models: practical issues and methodological challenges. Health Economics. 2007;16(12):1277–86. pmid:18034447
- 25. Kato M, Granich R, Bui D, Tran H, Nadol P, Jacka D, et al. The potential impact of expanding antiretroviral therapy and combination prevention in Vietnam: towards elimination of HIV transmission. JAIDS. 2013;63(15):e142–9.
- 26. Claxton K, Sculpher M, Drummond M. A rational framework for decision making by the National Institute for Clinical Excellence (NICE). Lancet. 2002;360(9334):711–5. pmid:12241891
- 27. Long E, Brandeau M, Owens D. The cost-effectiveness and population outcomes of expanded HIV screening and antiretroviral treatment in the United States. Ann Intern Med. 2010 153(12):778–89. pmid:21173412
- 28. Nosyk B, Min JE, Lima VD, Hogg RS, Montaner JS. Cost-effectiveness of population-level expansion of highly active antiretroviral treatment for HIV in British Columbia, Canada: a modelling study. The lancet HIV. 2015;2(9):00127–7.
- 29. Nosyk B, Zang X, Min JE, Krebs E, Lima VD, Milloy MJ, et al. Relative effects of antiretroviral therapy and harm reduction initiatives on HIV incidence in British Columbia, Canada, 1996–2013: a modelling study. The Lancet HIV. 2017;4(7):E303–E10. ISI:000404131200010. pmid:28366707
- 30. Nosyk B, Min JE, Krebs E, Zang X, Compton M, Gustafson R, et al. The Cost-Effectiveness of Human Immunodeficiency Virus Testing and Treatment Engagement Initiatives in British Columbia, Canada: 2011–2013. Clinical Infectious Diseases. 2017;66(5):765–77.
- 31.
Centers for Disease Control and Prevention (CDC). HIV Surveillance Report, 2017; vol. 29. Atlanta, GA: U.S. Department of Health and Human Services, 2018.
- 32. Zang X, Krebs E, Min J, Marshall B, Granich R, Shackman B, et al. Development and calibration of a dynamic HIV transmission model for 6 US cities. Submitted. 2019.
- 33. Sutton AJ, House T, Hope VD, Ncube F, Wiessing L, Kretzschmar M. Modelling HIV in the injecting drug user population and the male homosexual population in a developed country context. Epidemics-Neth. 2012;4(1):48–56. ISI:000300706600006. pmid:22325014
- 34. Briggs AH, Weinstein MC, Fenwick EA, Karnon J, Sculpher MJ, Paltiel AD. Model parameter estimation and uncertainty analysis a report of the ISPOR-SMDM Modeling Good Research Practices Task Force Working Group–6. Medical Decision Making. 2012;32(5):722–32. pmid:22990087
- 35. Sanders GD, Neumann PJ, Basu A, Brock DW, Feeny D, Krahn M, et al. Recommendations for conduct, methodological practices, and reporting of cost-effectiveness analyses: Second Panel on Cost-Effectiveness in Health and Medicine. Journal of the American Medical Association. 2016;316(10):1093. pmid:27623463
- 36. Karnon J, Vanni T. Calibrating models in economic evaluation. PharmacoEconomics. 2011;29(1):51–62. pmid:21142278
- 37. Taylor DC, Pawar V, Kruzikas D, Gilmore KE, Pandya A, Iskandar R, et al. Methods of model calibration: observations from a mathematical model of cervical cancer. PharmacoEconomics. 2010;28(11):995–1000. pmid:20936883
- 38. Greenhalgh T, Peacock R. Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources. BMJ. 2005;331(7524):1064–5. pmid:16230312
- 39.
Phillips B, Ball C, Sackett D, Badenoch D, Straus S, Haynes B, et al. Oxford Centre for Evidence-based Medicine—Levels of Evidence (March 2009) Oxford, UK: Centre for Evidence-Based Medicine; 2009 [24 October 2017]. Available from: http://www.cebm.net/oxford-centre-evidence-based-medicine-levels-evidence-march-2009/.
- 40. Long E, Mandalia R, Mandalia S, Alistar S, Beck E, Brandeau M. Expanded HIV testing in low-prevalence, high-income countries: A cost-effectiveness analysis for the United Kingdom. PLoS One. 2014;9(4):e95735. pmid:24763373
- 41. Marks G, Crepaz N, Senterfitt JW, Janssen RS. Meta-analysis of high-risk sexual behavior in persons aware and unaware they are infected with HIV in the United States—Implications for HIV prevention programs. Jaids-J Acq Imm Def. 2005;39(4):446–53. ISI:000230572400013.
- 42.
Centers for Disease Control and Prevention. HIV Infection, Risk, Prevention, and Testing Behaviors among Persons Who Inject Drugs—National HIV Behavioral Surveillance: Injection Drug Use, 20 U.S. Cities, 2012. HIV Surveillance Special Report 11. Revised edition. 2015 August 2015. Report No.
- 43. Attia S, Egger M, Muller M, Zwahlen M, Low N. Sexual transmission of HIV according to viral load and antiretroviral therapy: systematic review and meta-analysis. Aids. 2009;23(11):1397–404. ISI:000267962900013. pmid:19381076
- 44. Baggaley R, White R, Hollingsworth T, Boily M. Heterosexual HIV-1 infectiousness and antiretroviral use: Systematic review of prosective studies of discordant couples. Epidemiology. 2013;24(1):110–21. pmid:23222513
- 45. Boily MC, Baggaley RF, Wang L, Masse B, White RG, Hayes RJ, et al. Heterosexual risk of HIV-1 infection per sexual act: systematic review and meta-analysis of observational studies. Lancet Infect Dis. 2009;9(2):118–29. ISI:000262990900023. pmid:19179227
- 46. de Vincenzi I. A longitudinal study of human immunodeficiency virus transmission by heterosexual partners. European Study Group on Heterosexual Transmission of HIV. The New England Journal of Medicine. 1994;331(6):341–6. Epub 1994/08/11. pmid:8028613.
- 47. Downs AM, DeVincenzi I, Costigliola P, Ricchi E, Chiodo F, Roumeliotou A, et al. Probability of heterosexual transmission of HIV: Relationship to the number of unprotected sexual contacts. J Acq Immun Def Synd. 1996;11(4):388–95. ISI:A1996UC83300010.
- 48. Leynaert B, Downs AM, de Vincenzi I, H ESGHT. Heterosexual transmission of human immunodeficiency virus—Variability of infectivity throughout the course of infection. Am J Epidemiol. 1998;148(1):88–96. ISI:000074710300013. pmid:9663408
- 49. Nicolosi A, Leite MLC, Musicco M, Arici C, Gavazzeni G, Lazzarin A, et al. The Efficiency of Male-to-Female and Female-to-Male Sexual Transmission of the Human-Immunodeficiency-Virus—a Study of 730 Stable Couples. Epidemiology. 1994;5(6):570–5. ISI:A1994PP46800003. pmid:7841237
- 50. Obrien TR, Busch MP, Donegan E, Ward JW, Wong LY, Samson SM, et al. Heterosexual Transmission of Human-Immunodeficiency-Virus Type-1 from Transfusion Recipients to Their Sex Partners. J Acq Immun Def Synd. 1994;7(7):705–10. ISI:A1994NT68800010.
- 51. Padian NS, Shiboski SC, Glass SO, Vittinghoff E. Heterosexual transmission of human immunodeficiency virus (HIV) in Northern California: Results from a ten-year study. Am J Epidemiol. 1997;146(4):350–7. ISI:A1997XR27300008. pmid:9270414
- 52. Peterman TA, Stoneburner RL, Allen JR, Jaffe HW, Curran JW. Risk of Human Immunodeficiency Virus Transmission from Heterosexual Adults with Transfusion-Associated Infections. Jama-J Am Med Assoc. 1988;259(1):55–8. ISI:A1988L408900030.
- 53. Powers KA, Poole C, Pettifor AE, Cohen MS. Rethinking the heterosexual infectivity of HIV-1: a systematic review and meta-analysis. Lancet Infect Dis. 2008;8(9):553–63. ISI:000258752200015. pmid:18684670
- 54. Shiboski SC, Padian NS. Epidemiologic evidence for time variation in HIV infectivity. J Acq Immun Def Synd. 1998;19(5):527–35. ISI:000077424500013.
- 55. Baggaley RF, White RG, Boily MC. HIV transmission risk through anal intercourse: systematic review, meta-analysis and implications for HIV prevention. Int J Epidemiol. 2010;39(4):1048–63. ISI:000281532300023. pmid:20406794
- 56. Caceres CF, Vangriensven GJP. Male-Homosexual Transmission of Hiv-1. Aids. 1994;8(8):1051–61. ISI:A1994NZ83300003. pmid:7986400
- 57. Degruttola V, Seage GR, Mayer KH, Horsburgh CR. Infectiousness of Hiv between Male-Homosexual Partners. J Clin Epidemiol. 1989;42(9):849–56. ISI:A1989AQ73400009. pmid:2789269
- 58. Grant RM, Wiley JA, Winkelstein W. Infectivity of the Human-Immunodeficiency-Virus—Estimates from a Prospective-Study of Homosexual Men. J Infect Dis. 1987;156(1):189–93. ISI:A1987H772900026. pmid:3474298
- 59. Kaplan EH. Modeling Hiv Infectivity—Must Sex Acts Be Counted. J Acq Immun Def Synd. 1990;3(1):55–61. ISI:A1990CD74900009.
- 60. Weller S, Davis K. Condom effectiveness in reducing heterosexual HIV transmission. Cochrane Database Syst Rev. 2002;(1):CD003255. Epub 2002/03/01. pmid:11869658.
- 61.
World Health Organization. Prevention and treatment of HIV and other sexually transmitted infections among men who have sex with men and transgender people: recommendations for a public health approach, 2011. Geneva: World Health Organization, 2011 Feb 13, 2011. Report No.
- 62. Cohen M, Chen Y, McCauley M, Gamble T, Hosseinipour M, Kumarasamy N, et al. Prevention of HIV-1 infection with early antiretroviral therapy. N Engl J Med. 2011;365(6):493–505. pmid:21767103
- 63.
Rodger A, Bruun T, Cambiano V, Vernazza P, Estrada V, van Lunzen J, et al., editors. 153LB: HIV transmission risk through condomless sex if HIV+ partner on suppressive ART: PARTNER Study. Conference on Retroviruses and Opportunistic Infections 2014 March 3–6, 2014; Boston, Massachusetts.
- 64. Rodger A, Cambiano V, Bruun T, Vernazza P, Collins S, van Lunzen J, et al. Sexual activity without condoms and risk of HIV transmission in serodifferent couples when the HIV-positive partner is using suppressive antiretroviral therapy. Journal of the American Medical Association. 2016;316(2):171–81. pmid:27404185
- 65. Baggaley RF, Boily MC, White RG, Alary M. Risk of HIV-1 transmission for parenteral exposure and blood transfusion: a systematic review and meta-analysis. Aids. 2006;20(6):805–12. ISI:000236713500002. pmid:16549963
- 66. Hudgens MG, Longini IM, Vanichseni S, Hu DJ, Kitayaporn D, Mock PA, et al. Subtype-specific transmission probabilities for human immunodeficiency virus type 1 among injecting drug users in Bangkok, Thailand. Am J Epidemiol. 2002;155(2):159–68. ISI:000173389100009. pmid:11790680
- 67. Kaplan EH, Heimer R. A Model-Based Estimate of Hiv Infectivity Via Needle Sharing. J Acq Immun Def Synd. 1992;5(11):1116–8. ISI:A1992JV26500009.
- 68. Patel P, Borkowf CB, Brooks JT, Lasry A, Lansky A, Mermin J. Estimating per-act HIV transmission risk: a systematic review. Aids. 2014;28(10):1509–19. ISI:000337705400014. pmid:24809629
- 69. Long EF, Brandeau ML, Galvin CM, Vinichenko T, Tole SP, Schwartz A, et al. Effectiveness and cost-effectiveness of strategies to expand antiretroviral therapy in St. Petersburg, Russia. Aids. 2006;20(17):2207–15. ISI:000242676900010. pmid:17086061
- 70. Mellors JW, Muñoz A, Giorgi JV, Margolick JB, Tassoni CJ, Gupta P, et al. Plasma viral load and CD4+ lymphocytes as prognostic markers of HIV-1 infection. Ann Intern Med. 1997;126(12):946–54. pmid:9182471
- 71. Sanders GD, Bayoumi AM, Sundaram V, Bilir SP, Neukermans CP, Rydzak CE, et al. Cost-effectiveness of screening for HIV in the era of highly active antiretroviral therapy. New Engl J Med. 2005;352(6):570–85. ISI:000226862100007. pmid:15703422
- 72. Bellan S, Dushoff J, Galvani A, Meyers L. Reassessment of HIV-1 Acute Phase Infectivity: Accounting for heterogeneity and study design with simulated cohorts. PLoS Med. 2015;12(3).
- 73. MacArthur GJ, Minozzi S, Martin N, Vickerman P, Deren S, Bruneau J, et al. Opiate substitution treatment and HIV transmission in people who inject drugs: systematic review and meta-analysis. BMJ. 2012;345:e5945. pmid:23038795
- 74. Timko C, Schultz NR, Cucciare MA, Vittorio L, Garrison-Diehn C. Retention in medication-assisted treatment for opiate dependence: A systematic review. Journal of Addictive Diseases. 2016;35(1):22–35. pmid:26467975
- 75. Low AJ, Mburu G, Welton NJ, May MT, Davies CF, French C, et al. Impact of Opioid Substitution Therapy on Antiretroviral Therapy Outcomes: A Systematic Review and Meta-Analysis. Clin Infect Dis. 2016;63(8):1094–104. pmid:27343545
- 76. Choopanya K, Martin M, Suntharasamai P, Sangkum U, Mock PA, Leethochawalit M, et al. Antiretroviral prophylaxis for HIV infection in injecting drug users in Bangkok, Thailand (the Bangkok Tenofovir Study): a randomised, double-blind, placebo-controlled phase 3 trial. Lancet. 2013;381(9883):2083–90. pmid:23769234
- 77.
Centers for Disease Control and Prevention. Preexposure Prophylaxis for the prevention of HIV infection in the United States. 2014.
- 78. Holtgrave DR, Pinkerton SD. Updates of cost of illness and quality of life estimates for use in economic evaluations of HIV prevention programs. JAIDS Journal of Acquired Immune Deficiency Syndromes. 1997;16(1):54–62.
- 79. Honiden S, Sundaram V, Nease RF, Holodniy M, Lazzeroni LC, Zolopa A, et al. The effect of diagnosis with HIV infection on health-related quality of Life. Quality of Life Research. 2006;15(1):69–82. pmid:16411032
- 80. Joyce VR, Barnett PG, Bayoumi AM, Griffin SC, Kyriakides TC, Yu W, et al. Health-related quality of life in a randomized trial of antiretroviral therapy for advanced HIV disease. Journal of Acquired Immune Deficiency Syndromes. 2009;50(1):27–36. pmid:19295332
- 81. Kauf TL, Roskell N, Shearer A, Gazzard B, Mauskopf J, Davis EA, et al. A predictive model of health state utilities for HIV patients in the modern era of highly active antiretroviral therapy. Value in Health. 2008;11(7):1144–53. pmid:18494750
- 82. Schackman BR, Goldie SJ, Freedberg KA, Losina E, Brazier J, Weinstein MC. Comparison of health state utilities using community and patient preference weights derived from a survey of patients with HIV/AIDS. Med Decis Making. 2002;22(1):27–38. pmid:11833663
- 83. Tengs TO, Lin TH. A meta-analysis of utility estimates for HIV/AIDS. Med Decis Making. 2002;22(6):475–81. pmid:12458977
- 84. Song DL, Altice FL, Copenhaver MM, Long EF. Cost-effectiveness analysis of brief and expanded evidence-based risk reduction interventions for HIV-infected people who inject drugs in the United States. PloS one. 2015;10(2):e0116694. pmid:25658949
- 85.
Administration. DoHaMHVS. Maryland Vital Statistics Annual Report 2010 2010.
- 86.
Center for HIV Surveillance—Epidemiology and Evaluation—Department of Health and Mental Hygiene—Baltimore—MD. Baltimore City Annual HIV Epidemiological Profile 2013 2015. Available from: http://health.baltimorecity.gov/sites/default/files/Baltimore_City_HIV_Epidemiological_Profile%202013.pdf, [Accessed: February 2, 2018].
- 87.
Centers for Disease Control and Prevention. Prevalence of Diagnosed and Undiagnosed HIV infection—United States, 2008–2012. Morbidity and mortality weekly report (MMWR). 2015;64(24):657–62.
- 88.
County of Los Angeles. 2010 Annual HIV Surveillance Report. Los Angeles: Public Health LA County, 2011.
- 89.
Florida Department of Health. All Causes Crude Death Rate 2010 2018. Available from: http://www.flhealthcharts.com/charts/DataViewer/DeathViewer/DeathViewer.aspx?indNumber=0084, [Accessed February 8, 2018].
- 90.
Georgia Department of Public Health. HIV/AIDS Epidemiology Program HIV Care Continuum Surveillance Report 2012. Available from: https://dph.georgia.gov/hiv-care-continuum, published February 2014 [Accessed, February 2, 2018].
- 91.
Georgia Department of Public Health. Age-Adjusted Death Rate 2010. 2018.
- 92.
Los Angeles County Department of Public Health. Mortality in Los Angeles County 2010. 2013.
- 93.
New York City Department of Health and Mental Hygiene. Summary of Vital Statistics 2010: The City of New York. 2011.
- 94.
United States Census Bureau. April 1 2010 to July 1 2015 –United States–Metropolitan and micropolitan statistical area; and Puerto Rico United States2016 [cited 2016]. Available from: https://www.census.gov/popest/data/metro/totals/2015/.
- 95.
Washington State Department of Health. Age-Adjusted Mortality Rates. 2011.
- 96.
Florida Department of Health. HIV Surveillance Miami, FL2015 [18 April 2016]. Available from: http://miamidade.floridahealth.gov/programs-and-services/infectious-disease-services/hiv-aids-services/hiv-surveillance.html.
- 97.
New York City Department of Health and Mental Hygiene. New York City HIV/AIDS Annual Surveillance Statistics 2010. New York City: New York City Department of Health and Mental Hygiene, 2011.
- 98.
Public Health—Seattle and King County. HIV/AIDS annual and quarterly reports 2018. Available from: https://www.kingcounty.gov/depts/health/communicable-diseases/hiv-std/patients/epidemiology/annual-reports.aspx, [Accessed: February 21, 2018].
- 99. Grey JA, Bernstein KT, Sullivan PS, Purcell DW, Chesson HW, Gift TL, et al. Estimating the Population Sizes of Men Who Have Sex With Men in US States and Counties Using Data From the American Community Survey. JMIR Public Health and Surveillance. 2016;2(1):e14. PMC4873305. pmid:27227149
- 100. Mangal TD. Joint estimation of CD4+ cell progression and survival in untreated individuals with HIV-1 infection. Aids. 2017;31(8):1073–82. pmid:28301424
- 101. Nosyk B, Min J, Lima V, Hogg R, Montaner J, STOP HIV/AIDS. Cost-effectiveness of population-level expansion of highly active antiretroviral treatment for HIV in British Columbia, Canada: a modelling study. Lancet HIV. 2015;2(9):e393–e400. pmid:26423553
- 102. Tempalski B, Pouget E, Cleland C, Brady J, Cooper H, Hall H, et al. Trends in the population prevalence of people who inject drugs in US metropolitan areas 1992–2007. PLoS One. 2013;8(6):e64789. pmid:23755143
- 103. Vlahov D, Graham N, Hoover D, Flynn C, Bartlett JG, Margolick JB, et al. Prognostic indicators for AIDS and infectious disease death in HIV-infected injection drug users: plasma viral load and CD4+ cell count. Journal of the American Medical Association. 1998;279(1):35–40. pmid:9424041
- 104. Bernard CL, Brandeau ML, Humphreys K, Bendavid E, Holodniy M, Weyant C, et al. Cost-Effectiveness of HIV Preexposure Prophylaxis for People Who Inject Drugs in the United States. Ann Intern Med. 2016;26(2517406):M15–2634.
- 105. Evans E, Li L, Min J, Huang D, Urada D, Liu L, et al. Mortality among individuals accessing pharmacological treatment for opioid dependence in California, 2006–10. Addiction. 2015;110(6):996–1005. pmid:25644938
- 106. Song R, Hall HI, Green TA, Szwarcwald CL, Pantazis N. Using CD4 Data to Estimate HIV Incidence, Prevalence, and Percent of Undiagnosed Infections in the United States. Journal of Acquired Immune Deficiency Syndromes. 2017;74(1):3–9. pmid:27509244
- 107.
The HIV Research Network (HIVRN). Goals of the HIV Research Network. [cited 2017 Aug 15]. Available from: https://cds.johnshopkins.edu/hivrn/
- 108.
Centers for Disease Control and Prevention. Public use data file documentation. 2011–2013. National Survey of Family Growth. User's guide. Hyattsville, Maryland: Centers for Disease Control and Prevention, National Center for Health Science, 2014 December, 2014. Report No.
- 109.
Centers for Disease Control and Prevention. Behavioral Risk Factor Surveillance System Survey Data. Atlanta, GA: Centers for Disease Control and Prevention; 2014.
- 110.
Centers for Disease Control and Prevention. HIV Infection Risk, Prevention, and Testing Behaviors among Men Who Have Sex With Men—National HIV Behavioral Surveillance, 20 U.S. Cities, 2014. HIV Surveillance Special Report 15. 2016.
- 111. Wang L, Krebs E, Min JE, Mathews W, Nijhawan A, Somboonwit C, et al. Combined estimation of disease progression and antiretroviral therapy retention among treated individuals with HIV in the United States. The Lancet HIV. 2019; In Press.
- 112. Baggaley RF, White RG, Hollingsworth TD, Boily MC. Heterosexual HIV-1 Infectiousness and Antiretroviral Use Systematic Review of Prospective Studies of Discordant Couples. Epidemiology. 2013;24(1):110–21. ISI:000312498600018. pmid:23222513
- 113. Fujimoto K, Wang P, Ross MW, Williams ML. Venue-Mediated Weak Ties in Multiplex HIV Transmission Risk Networks Among Drug-Using Male Sex Workers and Associates. American Journal of Public Health. 2015;105(6):1128–35. pmid:25880956
- 114. Nosyk B, Zang X, Min JE, Krebs E, Lima VD, Milloy MJ, et al. Relative effects of antiretroviral therapy and harm reduction initiatives on HIV incidence in British Columbia, Canada, 1996–2013: a modelling study. Lancet Hiv. 2017;4(7):E303–E10. ISI:000404131200010. pmid:28366707
- 115. Raymond HF, McFarland W. Racial Mixing and HIV Risk Among Men Who Have Sex with Men. Aids and Behavior. 2009;13(4):630–7. ISI:000268478000002. pmid:19479369
- 116. Marmor M, Sheppard HW, Donnell D, Bozeman S, Celum C, Buchbinder S, et al. Homozygous and heterozygous CCR5-Delta32 genotypes are associated with resistance to HIV infection. Journal of Acquired Immune Deficiency Syndromes. 2001;27(5):472–81. pmid:11511825
- 117. Smith DK, Herbst JH, Zhang XJ, Rose CE. Condom Effectiveness for HIV Prevention by Consistency of Use Among Men Who Have Sex With Men in the United States. Jaids-J Acq Imm Def. 2015;68(3):337–44. ISI:000350245400020. pmid:25469526
- 118.
Centers for Disease Control and Prevention. HIV Infection Risk, Prevention, and Testing Behaviors among Heterosexuals at Increased Risk of HIV Infection—National HIV Behavioral Surveillance, 20 U.S. Cities, 2013. HIV Surveillance Special Report 13. 2015.
- 119. Metsch LR, Feaster DJ, Gooden L, Schackman BR, Matheson T, Das M, et al. Effect of risk-reduction counseling with rapid HIV testing on risk of acquiring sexually transmitted infections: the AWARE randomized clinical trial. Journal of the American Medical Association. 2013;310(16):1701–10. pmid:24150466
- 120. Vlahov D, Anthony JC, Munoz A, Margolick J, Nelson KE, Celentano DD, et al. The Alive Study—a Longitudinal-Study of Hiv-1 Infection in Intravenous-Drug-Users—Description of Methods. J Drug Issues. 1991;21(4):759–76. ISI:A1991GT12200006.
- 121. Dombrowski JC, Buskin SE, Bennett A, Thiede H, Golden MR. Use of multiple data sources and individual case investigation to refine surveillance-based estimates of the HIV care continuum. Journal of Acquired Immune Deficiency Syndromes. 2014;67(3):323–30. pmid:25140904
- 122. Gaines TL, Caldwell JT, Ford CL, Mulatu MS, Godette DC. Relationship between a Centers for Disease Control and Prevention expanded HIV testing initiative and past-year testing by race/ethnicity: a multilevel analysis of the Behavioral Risk Factor Surveillance System. AIDS Care. 2016;28(5):554–60. pmid:27045327
- 123. Mellors JW, Munoz A, Giorgi JV, Margolick JB, Tassoni CJ, Gupta P, et al. Plasma viral load and CD4+ lymphocytes as prognostic markers of HIV-1 infection. Ann Intern Med. 1997;126(12):946–54. pmid:9182471
- 124.
Centers for Disease Control and Prevention. Behavioral and Clinical Characteristics of Persons Receiving Medical Care for HIV Infection—Medical Monitoring Project, United States. 2016.
- 125.
New York City Department of Health and Mental Hygiene. Community Health Survey. 2014.
- 126.
SAMHSA. Number of DATA-Certified Physicians 2017. Available from: https://www.samhsa.gov/medication-assisted-treatment/physician-program-data/certified-physicians [Accessed: November 2, 2017].
- 127.
SAMHSA. Treatment Episode Data Set (TEDS) 2018. Available from: https://wwwdasis.samhsa.gov/dasis2/teds.htm [Accessed: January 24, 2018].
- 128.
Centers for Disease Control and Prevention. Syringe Service Programs for Persons Who Inject Drugs in Urban, Suburban, and Rural Areas—United States, 2013 2015. Available from: https://www.cdc.gov/mmwr/preview/mmwrhtml/mm6448a3.htm?s_cid=mm6448a3_w, [Accessed: February 4, 2018].
- 129.
City of Baltimore. Baltimore City Syringe Exchange Program 2016. Available from: http://www.aacounty.org/boards-and-commissions/HIV-AIDS-commission/presentations/BCHD%20Needle%20Exchange%20Presentation9.7.16.pdf [Accessed: January 24, 2018].
- 130.
City of Los Angeles. AIDS Coordinator's Office 2014. Available from: http://lacityaids.org/contact.html [Accessed: January 25, 2018].
- 131.
Georgia Health News. Injectable drugs can kill, but clean syringes can save lives 2017. Available from: http://www.georgiahealthnews.com/2017/01/injectable-drugs-kill-clean-syringes-save-lives/ [Accessed: January 24, 2018].
- 132.
New York State Department of Health AIDS Institute. Comprehensive Harm Reduction Reverses the Trend in New HIV Infections 2014. Available from: https://www.health.ny.gov/diseases/aids/providers/reports/docs/sep_report.pdf, [Accessed: February 4, 2018].
- 133.
Public Health—Seattle & King County. HIV/STD Program 2017. Available from: http://www.kingcounty.gov/depts/health/communicable-diseases/hiv-std.aspx [Accessed: January 24, 2018].
- 134.
Emory University Rollins School of Public Health. AIDSvu. Atlanta, GA. 2018. Available from: www.aidsvu.org; Accessed: June 6, 2018].
- 135. Hansen HB, Siegel CE, Case BG, Bertollo DN, DiRocco D, Galanter M. Variation in use of buprenorphine and methadone treatment by racial, ethnic, and income characteristics of residential social areas in New York City. The journal of behavioral health services & research. 2013;40(3):367–77.
- 136. Nosyk B, Min J, Colley G, Lima VD, Yip B, Milloy M-J, et al. The causal effect of opioid substitution treatment on highly active antiretroviral treatment adherence. Drug Alcohol Depend. 2015;146:e53–e4.
- 137. Saloner B, Daubresse M, Alexander GC. Patterns of Buprenorphine-Naloxone Treatment for Opioid Use Disorder in a Multistate Population. Medical care. 2017;55(7):669–76. pmid:28410339
- 138. Anderson PL, Glidden DV, Liu A, Buchbinder S, Lama JR, Guanira JV, et al. Emtricitabine-Tenofovir Concentrations and Pre-Exposure Prophylaxis Efficacy in Men Who Have Sex with Men. Science Translational Medicine. 2012;4(151):151ra25.
- 139. Liu AY, Cohen SE, Vittinghoff E, Anderson PL, Doblecki-Lewis S, Bacon O, et al. Preexposure Prophylaxis for HIV Infection Integrated With Municipal- and Community-Based Sexual Health Services. JAMA Intern Med. 2016;176(1):75–84. pmid:26571482
- 140.
Agency for Healthcare Research and Quality. Total Health Services-Mean and Median Expenses per Person With Expense and Distribution of Expenses by Source of Payment: United States, 2014 2014. Available from: https://meps.ahrq.gov/mepsweb/data_stats/quick_tables.jsp, Generated interactively. [Accessed August 11, 2017].
- 141.
Centers for Medicare and Medicaid Services. Physician fee schedule search 2017. Available from: https://www.cms.gov/apps/physician-fee-schedule/search/search-criteria.aspx [Accessed July 25, 2017].
- 142.
Centers for Medicare and Medicaid Services. Clinical laboratory fee schedule 2017. Available from: https://www.cms.gov/Medicare/Medicare-Fee-for-Service-Payment/ClinicalLabFeeSched/Clinical-Laboratory-Fee-Schedule-Files.html [Accessed July 25, 2017].
- 143.
Truven Health Analytics. RED BOOK Average Wholesale Price 2018. Available from: http://truvenhealth.com/products/micromedex/product-suites/clinical-knowledge/red-book, [Accessed: February 8, 2018].
- 144. Baser O, Chalk M, Fiellin DA, Gastfriend DR. Cost and utilization outcomes of opioid-dependence treatments. American Journal of Managed Care. 2011;17(8):S235–48.
- 145. McCollister K, Yang X, Sayed B, French MT, Leff JA, Schackman BR. Monetary conversion factors for economic evaluations of substance use disorders. J Subst Abuse Treat. 2017;81:25–34. pmid:28847452
- 146. Tookes H, Diaz C, Li H, Khalid R, Doblecki-Lewis S. A cost analysis of hospitalizations for infections related to injection drug use at a county safety-net hospital in Miami, Florida. PloS one. 2015;10(6):e0129360. pmid:26075888
- 147. Enns B, Krebs E, Mathews WC, Moore RD, Gebo KA, Nosyk B. Heterogeneity in the costs of medical care among people living with HIV/AIDS in the United States. Aids. 2019;3(10):0000000000002220.
- 148. Critchlow S, Hirst M, Akehurst R, Phillips C, Philips Z, Sullivan W, et al. A systematic review of cost-effectiveness modeling of pharmaceutical therapies in neuropathic pain: variation in practice, key challenges, and recommendations for the future. Journal of Medical Economics. 2016;201(2):129–39.
- 149. Ganz M, Hansen B, Valencia X, Strandberg-Larsen M. Key data elements for use in cost-utility modeling of biological treatments for rheumatoid arthritis. J Med Econ. 2015;18(5):366–75. pmid:25530467
- 150. Hoang V, Shanahan M, Shukla N, Perez P, Farrell MF, Ritter A. A systematic review of modelling approaches in economic evaluations of health interventions for drug and alcohol problems. BMC Health Serv Res. 2016;16(127).
- 151. Praditsitthikorn N, Kotirum S, Mohara A, Dumrongprat K, Velasco R, Teerawattananon Y. Assessing key model parameters for economic evaluation of pandemic influenze interventions: the data source matters. Influenza Other Respi Viruses. 2013;7(Suppl 2):59–63.
- 152. Schiller-Frühwirth I, Jahn B, Arvandi M, Siebert U. Cost-effectiveness models in breast cancer screening in the general population: A systematic review. Appl Health Econ Health Policy. 2017;15(3):333–51. pmid:28185134
- 153.
Neumann PJ, Sanders GD, Russell LB, Siegel JE, Ganiats TG. Cost-effectiveness in health and medicine: Oxford University Press; 2016.
- 154. Jones A, Cremin I, Abdullah F, Idoko J, Cherutich P, Kilonzo N, et al. Transformation of HIV from pandemic to low-endemic levels: a public health approach to combination prevention. Lancet. 2014;384(9939):272–9. pmid:24740087
- 155. Greenberg AE, Purcell DW, Gordon CM, Barasky RJ, Del Rio C. Addressing the challenges of the HIV continuum of care in high-prevalence cities in the United States. JAIDS Journal of Acquired Immune Deficiency Syndromes. 2015;69:S1–S7. pmid:25867773
- 156. Mugavero MJ, Amico KR, Horn T, Thompson MA. The state of engagement in HIV care in the United States: from cascade to continuum to control. Clinical Infectious Diseases. 2013;57(8):1164–71. pmid:23797289
- 157. Nachega JB, Uthman OA, del Rio, Mugavero MJ, Rees H, Mills EJ. Addressing the Achilles' heel in the HIV care continuum for the success of a test-and-treat strategy to achieve an AIDS-free generation. Clin Infect Dis. 2014;59(1):S21–7.
- 158. Goodreau SM, Rosenberg ES, Jenness SM, Luisi N, Stansfield SE, Millett GA, et al. Sources of racial disparities in HIV prevalence in men who have sex with men in Atlanta, GA, USA: a modelling study. The Lancet HIV. 2017.
- 159. Suen S-c Goldhaber-Fiebert JD, Brandeau ML. Risk Stratification in Compartmental Epidemic Models: Where to Draw the Line? Journal of Theoretical Biology. 2017.
- 160. Blair JM, Fagan JL, Frazier E, Do AN, Bradley H, Valverde EE, et al. Behavioral and clinical characteristics of persons receiving medical care for HIV infection—Medical Monitoring Project, United States, 2009. 2014. pmid:24941443
- 161. Claxton K. The irrelevance of inference: a decision-making approach to the stochastic evaluation of health care technologies. J Health Econ. 1999;8(3):341–64.
- 162. Shepherd K. Development goals should enable decision-making. Nature. 2015;523(7559):152. pmid:26156358
- 163. Husereau D, Drummond M, Petrou S, Carswell C, Moher D, Greenberg D, et al. Consolidated Health Economic Evaluation Reporting Standards (CHEERS)—explanation and alaboration: A report of the ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force. Value in Health. 2013;16:231–50. pmid:23538175
- 164. Zang X, Krebs E, Wang L, Marshall B, Granich R, Shackman B, et al. Structural design and data requirements for simulation modeling in HIV/AIDS: a narrative review. Second Review. 2019.