Figures
Abstract
Maternal syphilis results in an estimated 500,000 stillbirths and neonatal deaths annually in Sub-Saharan Africa. Despite the existence of national guidelines for antenatal syphilis screening, syphilis testing is often limited by inadequate laboratory and staff services. Recent availability of inexpensive rapid point-of-care syphilis tests (RST) can improve access to antenatal syphilis screening. A 2010 pilot in Zambia explored the feasibility of integrating RST within prevention of mother-to-child-transmission of HIV services. Following successful demonstration, the Zambian Ministry of Health adopted RSTs into national policy in 2011. Cost data from the pilot and 2012 preliminary national rollout were extracted from project records, antenatal registers, clinic staff interviews, and facility observations, with the aim of assessing the cost and quality implications of scaling up a successful pilot into a national rollout. Start-up, capital, and recurrent cost inputs were collected, including costs of extensive supervision and quality monitoring during the pilot. Costs were analysed from a provider’s perspective, incremental to existing antenatal services. Total and unit costs were calculated and a multivariate sensitivity analysis was performed. Our accompanying qualitative study by Ansbro et al. (2015) elucidated quality assurance and supervisory system challenges experienced during rollout, which helped explain key cost drivers. The average unit cost per woman screened during rollout ($11.16) was more than triple the pilot unit cost ($3.19). While quality assurance costs were much lower during rollout, the increased unit costs can be attributed to several factors, including higher RST prices and lower RST coverage during rollout, which reduced economies of scale. Pilot and rollout cost drivers differed due to implementation decisions related to training, supervision, and quality assurance. This study explored the cost of integrating RST into antenatal care in pilot and national rollout settings, and highlighted important differences in costs that may be observed when moving from pilot to scale-up.
Citation: Shelley KD, Ansbro ÉM, Ncube AT, Sweeney S, Fleischer C, Mumba GT, et al. (2015) Scaling Down to Scale Up: A Health Economic Analysis of Integrating Point-of-Care Syphilis Testing into Antenatal Care in Zambia during Pilot and National Rollout Implementation. PLoS ONE 10(5): e0125675. https://doi.org/10.1371/journal.pone.0125675
Academic Editor: Yukari C. Manabe, Johns Hopkins University, UNITED STATES
Received: September 24, 2014; Accepted: March 25, 2015; Published: May 13, 2015
Copyright: © 2015 Shelley et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited
Data Availability: All data are available in the Supporting Information files or within the mansuscript.
Funding: The pilot study was funded from UNICEF/UNDP/World Bank/World Health Organization Special Programme for Research and Training in Tropical Diseases through a grant from the Bill & Melinda Gates Foundation (grant no. OPP 47697). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Background
Syphilis is a sexually transmitted infection responsible for significant adult and perinatal mortality and morbidity, particularly in low income countries where the majority of the 12 million new cases occur each year [1]. Syphilis prevalence among pregnant women varies across countries in Sub-Saharan Africa (SSA), ranging from 0.1% in Benin up to 14.9% in Zambia [2]. A recent analysis of multinational antenatal surveillance data produced an approximate global estimate of 520,000 adverse outcomes due to maternal syphilis annually, including roughly 212,000 stillbirths or early foetal deaths, 92,000 neonatal deaths, 65,000 preterm or low birth weight infants, and over 151,000 syphilis-infected newborns [3]. Treatment guidelines recommend one to three doses of Benzathine penicillin (BP) in syphilis-positive pregnant women, depending on their stage of infection[4–6]. However, in resource-limited settings, testing and treatment of pregnant women with a single dose of BP has been shown to significantly reduce adverse pregnancy outcomes associated with syphilis [7]. Indeed, testing pregnant women for syphilis is part of the World Health Organization’s (WHO) recommended package of antenatal care (ANC) and has been integrated into ANC policies of most countries in SSA [8,9]. Despite the enormous health burden associated with maternal syphilis, antenatal syphilis testing is not routine in many countries in SSA, with coverage ranging from 0 to 100%. In Zambia, surveillance data from 2012 indicated 27.6% of pregnant women were tested for syphilis during ANC [2].
Until recently, the WHO-recommended syphilis screening tool was the Rapid Plasma Reagin (RPR) test. RPR requires laboratory capacity, electricity, refrigeration and specialist training, which has proved a barrier to consistent availability and accuracy of syphilis testing [9,10]. Research led by WHO/TDR Sexually Transmitted Diseases Diagnostics Initiative validated new point-of-care (POC) rapid syphilis tests (RST) in laboratory and in field settings [11,12]. POC tests have been defined as cheap, simple, hand-held tools which do not require extensive training or refrigeration, are performed near the patient or facility and deliver a rapid result [13,14]. These characteristics lend themselves to mass production, ease-of-use by minimally-trained healthcare workers (HCWs) and implementation in remote settings poorly served by electricity, supply and laboratory networks, thus greatly increasing access to diagnosis.
However, the intrinsic simplicity of POC tests poses challenges for quality maintenance. It is increasingly recognized that successful POC test implementation requires procurement of high quality test kits with good positive predictive value, effective training, quality monitoring systems and regular supervision of testers [15–18], all of which add to implementation costs. Failure to consider these programme aspects can negatively impact on cost-effectiveness of POC tests through over-diagnosis and treatment; yet, few studies assessing these programmatic costs are available and commentators note the unit cost of POC tests is repeatedly underestimated as economists fail to include the costs of implementation, training, human resources and maintenance in their estimates [15].
The cost-effectiveness of testing and treatment of maternal syphilis has been clearly demonstrated [19] and numerous cost models have highlighted that syphilis screening with POC tests is highly cost-effective at WHO-defined thresholds [20,21]. Furthermore, the cost-effectiveness of RSTs was shown to be comparable with RPR in ANC settings in several studies [22,23]. Recent studies demonstrated the feasibility of RST implementation in programmatic settings and highlighted the cost and health system benefits of integrating syphilis with HIV testing within ANC [24–26]. Evidence on the incremental increased costs associated with including a comprehensive quality assurance (QA) system for RST is limited. A recent study in Tanzania found QA costs ranged from $513 to $554 per clinic for a 9-month cost period; the high inter-clinic variability in QA costs was attributed to differences in staff salaries and the transportation costs to reach remote facilities [23].
To our knowledge, no published studies have yet compared the costs of implementing RST in research settings to those of a national scale-up. The primary aim of this study was to analyse the implementation costs of Zambia’s pilot programme, which explored feasibility of RST integration into ANC services, and secondarily, to present preliminary costs observed during Zambia’s national rollout of RST. We explored the key drivers of differences in cost between the two implementation stages, and examined how cost containment measures may have impacted on the quality of syphilis testing delivered to rollout ANC attendees.
Materials and Methods
Pilot Study Setting
From 2008–2010, the Elizabeth Glaser Pediatric AIDS Foundation (EGPAF) in partnership with the Centre for Infectious Disease Research in Zambia (CIDRZ) conducted a pilot that examined the feasibility and acceptability of introducing RSTs into prevention of mother-to-child-transmission of HIV programmes in ANC clinics in Zambia. A detailed description of the pilot and results is available elsewhere [25]; in brief, RST was introduced within a variety of ANC settings at 15 pilot facilities in two districts, comprising urban and rural locations, high and low ANC volume clinics and high and low syphilis prevalence (approximately 7% in Mongu and 2.5% in Lusaka). A centralized two-day RST training workshop took place before RST was integrated within existing clinic staffing patterns, patient flow, and clinic processes alongside other routine antenatal POC tests (HIV, malaria, and haemoglobin). The pilot utilised SD BIOLINE Syphilis 3.0, a rapid POC syphilis antibody test produced by Standard Diagnostics (Yongin-Si, South Korea), with a manufacturer reported 99.3% sensitivity and 99.5% specificity in serum versus the gold standard Treponema pallidum haemagglutination (TPHA) test [27]; of note, a recent meta-analysis of the diagnostic accuracy of SD Bioline 3.0 in field conditions reported a lower pooled sensitivity (87.9% serum; 83.8% whole blood) and specificity (96.0% serum; 98.4% whole blood)[28]. Pilot-specific QA and quality control (QC) measures were established to ensure high standards of quality management [25,29]. The pilot results showed increased syphilis testing among ANC attendees (79.9% versus 95.6%, p<0.0001) and treatment of syphilis positive pregnant women (51.1% versus 95.2%, p<0.0001), and demonstrated the feasibility of integrating RST within busy urban and rural ANC settings in Zambia [25].
National Rollout Setting
Following the successful pilot, Zambia adopted RST into national policy and recommended the use of rapid syphilis tests to offer same-day testing and treatment [30,31]. In March 2012, the Zambian Ministry of Health (MOH) launched the first phase of RST rollout in four underserved districts with high rates of maternal mortality: Mansa (Luapula Province), Kalomo (Southern Province), and Lundazi and Nyimba (Eastern Province). There were several key implementation differences between the pilot and rollout with regard to training, supervision, quality management mechanisms, and testing and treatment algorithms. Firstly, both pilot and rollout utilised a cascaded training approach whereby at least one staff member from each participating facility attended a district-level training workshop and, in-turn, trained facility colleagues in RST testing procedure. However, in practice, rollout HCWs received substantially less on-the-job supervision and guidance following initial training. Second, the rollout supervision process involved a 10-day MOH/EGPAF joint visit to rollout districts during May-July 2012, with unannounced visits to selected facilities; whereas, the pilot supervision process involved monthly EGPAF/CIDRZ monitoring visits at all pilot sites for data collection, quality checks and remedial on-site training for poor performers identified through proficiency testing. Third, the QA/QC mechanisms were centrally led during the pilot, with study staff providing materials and feedback; for the rollout, training was provided to district-level laboratory staff in order to assume this role. Fourth, at all pilot sites, treatment was initiated on the basis of a positive RST. RST detects antibodies specific to the causative bacterium Treponema pallidum, and does not distinguish between active and past, treated infection; whereas, the rollout introduced different testing algorithms utilising either RST alone, or RST as a screening test followed by RPR as a confirmatory test where this was available. The differences in treatment algorithm are described below. Our interpretation of differentials in the cost data and of the influence of implementation decisions on costs draws heavily on the qualitative findings reported in our accompanying paper by Ansbro et al. (2015, in review), which includes a detailed comparison of implementation differences in scale up from successful pilot to national rollout [32].
Costing Methods
Data collection.
Our cost methodology combined an ingredients-based approach, whereby a unit cost is multiplied by a resource quantity to generate a total cost, with a step-down cost accounting approach for facility level data in that joint costs are allocated to activities through cost centres [33,34]. A variety of sources were used for data on inputs, outputs and costs, including: inspection of facility records and registers, clinic observation, expert interviews with clinic and project staff, and review of project accounts. An Excel-based cost collection tool was used to collect the cost information during the pilot and rollout phases [34]. Costs were collected between March and July 2010 from five of the fifteen pilot facilities, including two urban health centres (UHC) in Lusaka plus one UHC and two rural health centres (RHC) in Mongu; facilities were purposively sampled to represent variation in facility size, target population, and location. For the national rollout, costs for the period March to July 2012 were collected from five facilities, including one UHC and three RHCs in Mansa District and one district hospital (DH) in Kalomo; facilities were sampled by convenience from among facilities visited by the MOH supervisory team during July and August 2012. Prevalence data from the urban facility (UHC4) diverged significantly from the average and was considered an outlier. We excluded UHC4 from the cost analysis, but the unique challenges experienced by this facility are discussed in the qualitative paper [32]. Table 1 illustrates key differences across the pilot and rollout facilities.
Both pilot and rollout examined the incremental cost of adding RST screening and treatment onto existing ANC services, i.e. any additional costs to execute syphilis screening and treatment were included but administrative costs to run the health facility were excluded. All research costs to collect data during the pilot and rollout were also excluded. Economic costs were collected retrospectively from the provider’s perspective. Financial and logistical constraints during the rollout evaluation prevented full replication of pilot cost methods, the key difference being that direct observation of rollout RST test performance and clinic flow was not possible, either due to RST stock outs or lack of ANC services scheduled on the day of site visits. Therefore, rollout recurrent staff time estimates are based on pilot costing and interviews with HCWs and experts.
All pilot and rollout costs are presented in 2012 United States Dollars (USD). Pilot costs collected in 2010 Zambian Kwacha (ZKW) were converted to USD using the average exchange rate for 2010 (ZKW 4,743.98 = 1 USD [35]), then adjusted to 2012 USD using a 2-year inflation rate of 5.29% (2010 CPI 218.056 / 2012 CPI 229.594 = 1.0529) from the US Consumer Price Index (CPI) [36,37]. During the rollout, non-supply costs were collected in 2012 ZKW or USD; supply costs were collected in 2011 ZKW and inflation adjusted to 2012 ZKW using Zambia’s CPI (2012 CPI 122.439 / 2011 CPI 115.091 = 1.0638) [38]. All rollout costs in 2012 ZKW were then converted to USD using the average exchange rate for 2012 (ZKW 5,219.83 = 1 USD [39]).
Cost Categorisation.
Start-up, capital and recurrent cost inputs were collected during pilot and rollout phases. Recurrent and capital costs were further subdivided into facility and central level categories to highlight the implementation costs of incorporating an extensive supervision and QA system during the pilot phase. Start-up is considered an input, rather than an activity, and therefore includes all resources required for training (e.g. personnel, per diems, conference hire, training equipment and supplies, and vehicle transport). For the pilot period only, the start-up also included costs for two district events to launch the RST pilot activities. All start-up costs during the pilot and rollout were annualised over an estimated project life of three years.
Capital costs are generally considered to have a life span of more than one year and cost greater than $100 USD per unit [40]. We included capital cost inputs for vehicles and computers. Annual financial costs were estimated using straightline depreciation, while economic costs were annualised with a 3% discount rate [41]. RPR-related equipment was not included, as this study focussed on incremental costs to existing programs. None of the pilot or rollout sites had an on-site vehicle dedicated to ANC services; thus, pilot vehicle costs included travel in the project, MOH, or district health vehicle for routine study monitoring, delivery of supplies and QA visits. During rollout, the only vehicle costs comprised return travel from Lusaka for supervisory monitoring visits. Vehicles were annualized over a five-year period. The economic costs of computers used for electronic medical records at two pilot facilities in Lusaka were annualised over a three-year period with an allocation factor of 20%. During rollout, the economic costs of computers used during RST training were annualised over three years with an allocation factor ranging from 3 to 12% based on the number of participants attending training from each facility.
Recurrent cost inputs comprised all operating costs throughout the project life, including: personnel, supplies, vehicle fuel and maintenance for supervisory visits, QA/QC, and supervision. Table 2 presents differences between pilot and rollout in terms of unit prices and quantity of healthcare resources consumed for testing commodities. Personnel time and some supplies were considered joint “shared” costs, with an allocation factor applied based on researcher time-motion observation during the pilot. Supply use was not directly observed but was modelled from monthly output data plus a 10% adjustment to account for estimated supply wastage. Supplies for testing and treating male partners were included for the pilot. Treatment with a single dose of BP was assumed during the pilot because follow up doses were not generally recorded in registers and could not be verified. Following the pilot and RST policy adoption, national treatment algorithms outlined a single injection of BP following a reactive RST; where RPR confirmation was unavailable, this was followed by two further weekly doses of BP; where RPR confirmation was available, two further BP doses were given if active syphilis infection was confirmed (a description of algorithms is available in Ansbro et al.) [32]. Two of the four rollout facilities (DH1 and RHC3) utilised confirmatory RPR; for these clinics, three doses of BP are assumed in the costing calculations for RPR-confirmed positives. Male partner testing was inconsistent during rollout; therefore, only male partner treatment costs were included. Shared supplies (e.g. biohazard bags, test tubes and needles for blood draw, sharps bins, gloves, cottonwool, and disinfectant) were given a 25% allocation factor to reflect that four blood tests were routinely conducted on ANC patients (HIV, syphilis, haemoglobin, and malaria). Whereas, supplies used only for syphlis testing (e.g. RST test kit, RST job aid, penicillin, water and needle for BP injection), were given an allocation factor of 100%. For facilities that used both venipuncture and finger prick methods of blood collection, we assumed each method was used 50% of the time.
QA/QC was considered a recurrent input that included personnel, supplies, and transporation costs for distributing and collecting known positive and negative samples for testing at the facility level during the pilot phase; a supplementary table is available with further details of the QA/QC cost calculations (S1 Table). During the rollout, a formal QA/QC system was not implemented and supervision activities occurred infrequently; therefore, the only central level costs for supervision included personnel salaries, accommodation, and fuel costs during the supervision trip. Recurent building utilities and waste management were excluded given the incremental costing approach. See the supplementary table of data inputs and assumptions for additional information (S2 Table).
Project Outputs
All project outputs were retrospectively collected from facility-level ANC patient registers during the five-month cost collection period for both pilot and rollout. Outputs included number of pregnant women attending first ANC visit, number of pregnant women and partners screened for syphilis with RST, number of syphilis-reactive tests, and number of women and partners treated for syphilis. Unit economic costs were calculated per patient tested and per patient treated at each facility, and a facility average was calculated.
Sensitivity Analysis
We performed univariate and multivariate sensitivity analyses to assess the influence of key inputs on overall unit costs. Cost inputs that were not directly observed, were highly uncertain or differed substantially between the pilot and rollout periods were varied in the sensitivity analysis, including project life years (1 to 5 years), supply wastage rate (0 to 50%), blood collection method (finger prick versus venepuncture), price of RST kits ($0.65 to $1.15), and coverage of RST among first ANC attendees (25% to 100%). Best and worst case scenarios were estimated by applying the minimum and maximum value for all parameters varied in the sensitivity analysis.
Ethical Review
The RST pilot protocol was approved by the University of Zambia Biomedical Research Ethics Committee (UNZAREC), University of Alabama at Birmingham’s Institutional Review Board, the WHO’s Research Ethics Review Committee, and MOH Zambia. The follow-up research to collect costs during the national RST rollout was approved by LSHTM University Ethics, UNZAREC and by the Permanent Secretary of the Zambian MOH. Additionally, written informed consent to conduct data collection related to national rollout was obtained from each Provincial and District Health Office. During both pilot and rollout, individual patient consent was not administered. De-identified patient outcome data were retrospectively abstracted from routinely collected health register data into aggregate monthly summaries.
Results and Discussion
During the March-July 2010 pilot study period, 4,949 pregnant women presented for a first ANC visit, of whom 97.3% (n = 4817) were screened for syphilis, 9.3% (n = 447) tested reactive, and of these, 89.3% (n = 399) were treated for maternal syphilis at five facilities in two districts (Table 3). The average prevalence based on reactive RST was notably different between the pilot facilities with a range from 6% to 23%, and a notable difference between urban Lusaka (8.1%) and rural Mongu (17.7%) districts [25]. During the same period in 2012, 1,210 pregnant women attended first ANC at four rollout facilities in two different districts. Of the 34.8% (n = 421) who were screened for syphilis, 6.2% (n = 26) had a reactive result (or tested positive), but facility-level prevalence ranged from 3.5% to 14.0%. Treatment coverage with the first dose of penicillin averaged 84.6%, and ranged from 50 to 100% across rollout sites.
Total Costs
Across pilot facilities, total economic costs ranged from $1,952 to $4,799 (Table 4). Central-level costs (supervision and QA/QC) comprised the majority of costs (53.4%), followed by facility-level costs (42.8%) and start-up costs (3.8%). Overall, supervision (34.0%), supplies (26.3%), clinic personnel (16.1%), and QA/QC (19.3%) were the major cost drivers during the pilot (Fig 1). Pilot QA/QC costs ranged from $204 to $883 and supervision costs (central-level personnel and vehicle costs) averaged around $1,500 per Lusaka facility and $770 for Mongu facilities which were visited less frequently. In comparison, during rollout, the total economic costs ranged from $882 to $1,719 (Table 4); health facility costs (31.2%) comprised a lower proportion of total costs compared to the pilot given the far lower RST coverage (Fig 1). During the rollout, supervision by central level personnel (38.1%) and the associated transport costs (18.6%) were major cost drivers. Clinic personnel (16.4%), supplies (13.2%), and start-up (12.1%) were also major cost components (Fig 1). In-depth supervision and QA/QC mechanisms put in place during the pilot phase were not similarly implemented during rollout, yet central-level pilot costs (53.4%) were similar to rollout costs (56.7%), discussed below.
Central-level supervision (including QA/QC costs during pilot) accounted for over half of costs. Supervision, start-up, and health facility costs (supplies, personnel) were also major cost drivers.
Unit Costs
During the pilot, the average unit cost was $3.19 per person screened and $30.34 per person treated, with substantial variation in unit cost across pilot facilities, which ranged from $2.02 to $13.75 per person screened and from $22.43 to $221.94 per person treated (Table 5). In the rollout phase, the average unit cost per woman screened ($11.16) was over triple that of the pilot, ranging from $9.34 to $17.64, and the average cost per woman treated was $167.85, ranging from $125.98 to $266.10. One key difference between implementation phases was the increase in RST kit costs from $0.65 during the pilot to $1.15 during the rollout. Economies of scale were noticeable at the high volume urban facilities of the pilot, where the relatively high fixed costs associated with start-up, ongoing supervision, and QA/QC were spread across a larger ANC patient population. When summarised by urban versus rural health centres, pilot unit costs were $2.53 per person screened versus $11.49, respectively. For the rollout, there was no difference between the average unit cost per person screened ($11.16) at RHCs and the DH1 facility.
Sensitivity Analysis
We noted differences in the impact of varying key parameters on the cost per person screened between pilot and rollout. Results of the univariate sensitivity analysis indicated that unit costs were particularly sensitive to coverage of RST services in both pilot and rollout. For the pilot, decreasing RST coverage from the base case (97%) to 25% resulted in an increase from $3.19 to $9.28 in average unit cost per person screened, which was much closer to the unit cost during the rollout period where testing coverage was only 35%. Similarly, for the rollout, varying RST coverage from the base case (35%) to 100% coverage decreased the cost per person screened from $11.16 to $5.87, again far closer to the pilot unit cost ($3.19) (Fig 2), and more reflective of the unit screening cost of $3.10 (minus QA/QC and supervisory costs) modelled by Larson et al. with RST coverage of 62% [42]. Unit costs were also sensitive to RST kit price and the wastage rate for supplies during both pilot and rollout phases, which reflects the substantial proportion of overall costs dedicated to supplies. During rollout, the unit cost estimate was sensitive to the length of the project life, reflecting the comparatively higher proportion of training start-up costs during rollout (12.1%) versus pilot (3.8%). The multivariate best/worst case scenario analysis suggests that the worst case scenario during the pilot ($11.43) would approximate the average unit cost of the rollout period. The rollout unit cost was much greater, ranging from $5.03 to $17.79 per person screened.
A range of uncertain parameters was varied in one-way sensitivity analyses; parameters are displayed along the vertical axis. The solid vertical line indicates the base case incremental cost per person screened for syphilis ($3.19 during pilot; $11.16 during rollout). The horizontal bars represent the range of cost per person screened when varying the associated parameter from the low to high values indicated in parentheses. The best versus worst case scenario is a multivariate representation when all parameters are set to the low versus high values. During pilot and rollout, the cost per person screened was most highly sensitive to RST coverage among ANC attendees. FP = Finger prick; VP = Venepuncture.
Cost Differentials in Moving from Pilot to Scale-Up
The average unit cost ($3.19) per person screened in the Zambian pilot study setting is consistent with the range of published data on RST in ANC settings from a variety of low- and middle-income countries, ranging from $1.02 to $6.96 [22,23,43,44]. However, our results highlight the substantial increase in unit cost ($11.16) per person screened observed during Zambia’s preliminary national rollout of RST. This is partially attributable to higher supply costs for RST kits during the rollout ($1.15 versus $0.65 during the pilot) but is primarily driven by far lower testing rates; 97.3% of women were screened at first ANC visit during the pilot, compared to 34.8% during rollout. The low coverage can in part be explained by supply constraints where three of four rollout sites reported repeated RST stock outs, discussed below, but may also be attributable to provider failure to test. For example, DH1 had low testing rates despite no reported RST stock outs (Table 3). The low rate of testing coverage meant far lower volumes of tests were conducted during the rollout cost period (N = 421) compared to pilot cost period (N = 4,817), also reducing economies of scale. Treatment rates were similarly high during pilot and rollout phases (85–90%). The average cost per person treated during rollout was nearly five times ($148) that of the pilot ($30), again driven by the far lower testing coverage during rollout.
Urban and rural cost differentials varied. During the pilot, the average cost per person screened was $2.53 in urban facilities and $11.49 in rural facilities. At urban sites, higher facility-level costs were mitigated by the economies of scale afforded by high patient volumes. Costs to implement QA/QC in facilities in rural districts were elevated due to long travel distances from Lusaka. During the rollout, low RST coverage at DH1 (24%) resulted in an elevated screening unit cost ($11.16), which was unexpected for a high volume ANC setting. The low coverage meant the average screening unit cost at DH1 was comparable to the lower volume, rural-based rollout facilities. A recent pre-post study of RST rollout in 18 ANC facilities in one of the Zambian districts included here, reported baseline antenatal syphilis testing (using RPR) of 10%, which increased to 62% during the 12 months after RST introduction [45]. Thus, testing rates are likely to have improved in the time since our data were collected.
Clinic personnel costs comprised approximately 16% of total costs for both pilot and rollout, while supplies contributed a higher proportion (26%) to costs during pilot versus rollout (13%) which was driven by higher RST coverage during the pilot. There were notable differences in cost drivers between the phases, in part due to the absence of comprehensive supervision and quality monitoring systems during the rollout (Fig 1).
Monitoring and Supervision Costs
During the pilot, QA/QC accounted for about 20% of costs and general supervision accounted for 34% of costs driven by transport (8.4%) and salaries of central-level supervisors (25.6%); whereas during rollout, central-level supervision and transport accounted for 38.1% and 18.6% of total costs, respectively. The higher proportion of costs devoted to transport during rollout is partly explained by the distance from Lusaka and the vast distances between rollout facilities which necessitated two vehicles and two supervisory teams from the central level to conduct supervision visits. Devolution of supervision and quality monitoring systems to district level (as intended by the MOH) should reduce duration of supervisory visits and the associated salary, vehicle and transport costs from Lusaka. As noted, the QA/QC implementation was substantially changed in the transition from pilot to rollout for perceived cost-saving reasons but was carried out inconsistently, if at all; these costs were therefore excluded from the cost analysis.
Quality
To our knowledge, this is the first study to compare RST QA/QC costs in a programmatic and a research setting. A recent modelling study supported the cost-effectiveness of scaling up syphilis screening within existing ANC programs in eight theoretical country models but excluded the costs associated with training, supervision and quality maintenance [21], which can contribute to both testing quality and cost. Larson et al. (2014) demonstrated the cost-effectiveness of the Zambian RST programme after the first year of implementation in one district. They included cost of testing commodities, HCW time and training but did not address costs associated with supervision and quality monitoring [42]. A Tanzanian pilot RST study addressed this issue [23], documenting the small incremental increase in unit costs incurred by adding a comprehensive quality monitoring system. The average cost per woman screened increased from only $1.92 to $2.74 and the average cost per woman treated from $21.40 to $30.57, reflecting our pilot cost findings, where inclusion of supervision and QA/QC costs augmented the average screening unit cost from $1.49 to $3.19 and treatment unit cost from $14.12 to $30.28. The Tanzanian analysis suggested that devolving supervision and monitoring to district level and reducing frequency of external QA/QC could potentially reduce programme costs, in keeping with the Zambian MOH’s implementation approach. However, the impact of this reduced QA/QC model needs evaluating.
Implementation Challenges
As noted, several aspects of the pilot study implementation were adapted or scaled down in order to scale-up for a national roll out: training, supervision, QA/QC, supply chain and diagnostic algorithms. Our accompanying qualitative study, Ansbro et al., provides details around actual testing activity and HCW behaviour, allowing examination of our costing data through a richer lens [32]. A number of key challenges were identified during this evaluation of the first phase of the national rollout:
First, cascaded training was utilised in both phases. However, pilot HCWs had much greater and more frequent exposure to central-level research staff who provided monthly supervision and remedial training as required. Second, the reduction in supervision frequency from monthly at all pilot locations to quarterly at selected rollout locations reduced average facility supervision cost by about one-third, from approximately $1,000 to $660. While it resulted in cost-savings, the reduction in supervision frequency afforded fewer opportunities to promote testing and treatment, identify gaps in test kit quality, and to identify and remediate poor provider proficiency or non-adherence to guidelines. Supervision and monitoring of HCWs’ performance has been shown to increase their confidence in test performance and accuracy of test results [46]. Third, while the Zambian MOH was progressive in including QA/QC activities in their national RST programme, devolution to district level introduced challenges during the first rollout phase (Table 6) [47]. On evaluation, rollout HCWs rarely performed incoming kit inspection; district laboratories performed external QC on their own samples but had not initiated an external QA/QC system for surrounding facilities. Several factors may have contributed to this: ineffective planning and communication, lack of nominated responsible personnel, lack of dedicated budget and logistics, lack of local expertise in this type of activity which is often undertaken by NGOs, or lack of hands-on training for district laboratory staff. Since external QA/QC and confirmatory testing were not performed, it is difficult to comment on the quality of testing provided in this preliminary phase of the rollout.
Fourth, introducing a new POC test with new diagnostic algorithms posed potential challenges and cost implications during the rollout. Our qualitative data shows that several facilities did not adhere to guideline diagnostic algorithms [32]. In sites without confirmatory RPR testing, inadequate HCW training and QA/QC systems may have contributed to inaccurate test result interpretation and unnecessary and costly repetition of RST. If DH1, a high ANC volume site with RPR capability and relatively high RST reactivity rate (7.1%), provided 100% testing coverage as per guidelines, large numbers of positive RST tests would require RPR confirmation; performing a second test incurs higher costs; and RPR-capable sites are still vulnerable to the pre-existing barriers to RPR testing, including supply chain weaknesses, absent or inadequately trained staff and patient loss-to-follow-up. The most cost-effective and usable testing algorithms for areas of varying syphilis prevalence have yet to be established.
Finally, RST supply was consistent during the pilot but less so during rollout. Pilot kits were delivered by study staff during supervisory visits. During the preliminary rollout, RSTs were delivered via the national pharmacy supply chain rather than the usual medical supply route; three of the five rollout sites evaluated (RHC3, RHC4, and RHC5) reported RST stock outs during the entire month preceding data collection.
Study Limitations
Rollout data were collected within five months of the rollout commencing and therefore reflect early implementation issues which may have been resolved in the interim. Our analysis was cross-sectional and therefore limited to the number of people screened and treated; unfortunately, we were unable to assess birth outcomes, including congenital syphilis, to assess programme impact and cost-effectiveness. We could not adequately quantify the additional costs for RPR confirmatory testing during the rollout, because sites were not selected according to diagnostic algorithm but rather by convenience sampling—therefore, costing sites included a variety of facility level laboratory and staffing capacity which employed different diagnostic algorithms. An assessment of the differential in costs between facilities with and without RPR confirmatory testing and with different local syphilis prevalence was not adequately captured by this study but is an area for future research. The cost methodologies were not completely replicated between pilot and rollout as RST stock outs and ANC scheduling prevented direct observation of RST performance during the rollout phase site visits. In addition, we compared pilot and rollout districts with different characteristics including: population densities, syphilis prevalence, and distance from the district health office—which reduces comparability of data between pilot and rollout phases, but reflects the district variations that exist within the country and is an important consideration in scaling up an intervention. Finally, secular trends during the two-year time gap between pilot and rollout data collection could not be accounted for in this analysis; rollout RST data were not available at the original pilot facilities during 2012.
Further Research
This study demonstrates the potential for further research pairing in-depth qualitative methods with cost analyses to better understand the challenges of POC test scale-up. Recent commentary has called for increased operations research into scale-up to identify the optimum mix of QA/QC which will be cost-effective yet maintain POC test reliability [28] and research to explore the cost-effectiveness of cascaded training. Dual antibody/antigen POC syphilis screening tests currently under investigation may address issues around the RST antibody test’s lack of specificity for active infection, which inevitably leads to over-diagnosis and treatment of pregnant women [48]. Dual HIV and syphilis antibody tests may potentially contribute to economies of scale in terms of start-up, training, QA/QC and supervision and monitoring, while also serving to increase syphilis screening to match existing high HIV testing coverage in ANC settings [49–51]; further implementation research on the cost and feasibility of dual test deployment is needed. In addition, the WHO is currently investigating the impact and cost-effectiveness of varying syphilis diagnostic algorithms using combinations of RST, dual tests and RPR, according to local prevalence. The results of these studies are awaited.
Conclusion
This study explored the cost of integrating RST into ANC clinics in both pilot study and national rollout settings in Zambia, reflecting findings from studies performed in other low- and middle-income countries. Cost differentials between pilot and rollout, including the significant influence of syphilis prevalence on unit cost per person screened and treated, and the considerably lower costs of supervision and QA/QC systems, were influenced by an increase in RST kit price, lower testing coverage and by challenges in implementation of training, regularity of testing, supervision, and QA/QC programme components. Planning for further scale-up of the Zambian RST programme must take heed of the ongoing budgetary, supervisory and policy-level support that is required for successful implementation.
There is a growing consensus that despite the additional cost, robust training, QA/QC and supervision systems are essential to ensuring the reliability and reproducibility of POC tests [15,52,53]. We recommend that these implementation programme aspects are included in future models of the cost-effectiveness of RST testing programs. The transfer of responsibility for QA/QC to district-level staff may require greater emphasis in training, specific budget allocation and prolonged supervision and monitoring until they are well established. Involving district-level personnel in supervision could increase opportunities for external QA/QC and remedial training at facility-level. Integrating transport of QA/QC samples and RST kits within existing supply chain mechanisms could potentially improve quality and consistency of testing at little added cost. Given these findings, continued RST scale-up efforts in Zambia could benefit from quality spot-checks to identify remedial support needs for training and QA/QC activities.
Supporting Information
S1 Table. Quality Control and Quality Assurance Associated Costs.
https://doi.org/10.1371/journal.pone.0125675.s001
(XLSX)
Acknowledgments
The authors would like to recognize the support provided by the Zambia Ministry of Health and District Health Teams from Mongu, Lusaka, Kalomo, and Mansa Districts for their role throughout the pilot RST integration study and the national RST rollout efforts. The EGPAF Zambia staff are thanked for their invaluable support during both the RST pilot and national rollout. Catherine Wilfert (Duke Medical Center, Durham, NC), Laura Guay (EGPAF, DC) and Tabitha Sripipatana (Office of Population and Reproductive Health, USAID, DC) are acknowledged for their technical guidance throughout development and implementation of the pilot study. We would also like to recognize the technical support provided during the pilot study by LSHTM, CIRDZ, and UNICEF/UNDP/World Bank/WHO Special Programme for Research and Training in Tropical Diseases. The authors also thank the journal reviewers for their helpful comments and suggestions for revising the original version of the manuscript.
Author Contributions
Conceived and designed the experiments: FTP RWP CF EMA S. Strasser. Performed the experiments: KDS S. Sweeney ATN CF EMA GTM MMG. Analyzed the data: KDS S. Sweeney CF ATN EMA. Contributed reagents/materials/analysis tools: FTP. Wrote the paper: KDS EMA FTP CF S. Sweeney. Reivew, revisions, and final manuscript approval: KDS EMA S. Strasser S. Sweeney GTM MMG CF ATN RWP FTP.
References
- 1.
WHO. The global elimination of congenital syphilis: rationale and strategy for action. WHO. 2007.
- 2.
WHO. Report on global sexually transmitted infection surveillance 2013 [Internet]. 2014. Available: http://apps.who.int/iris/bitstream/10665/112922/1/9789241507400_eng.pdf?ua=1.
- 3. Newman L, Kamb M, Hawkes S, Gomez G, Say L, Seuc A, et al. Global estimates of syphilis in pregnancy and associated adverse outcomes: analysis of multinational antenatal surveillance data. PLoS Med. 2013 Jan;10(2):e1001396. pmid:23468598
- 4.
Reproductive Health and Research, WHO. Sexually transmitted and other reproductive tract infections: a guide to essential practice [Internet]. World Health Organization. 2005. Available: http://whqlibdoc.who.int/publications/2005/9241592656.pdf?ua=1.
- 5. Janier M, Hegyi V, Dupin N, Unemo M, Tiplica GS, Potocnik M, et al. 2014 European guideline on the management of syphilis. J Eur Acad Dermatology Venereol. 2014;28(12):1581–93. pmid:25348878
- 6.
CDC. Diseases Characterized by Genital, Anal, or Perianal Ulcers—2010 STD Treatment Guidelines [Internet]. Division of STD Prevention, National Center for HIV/AIDS, Viral Hepatitis, STD, and TB Prevention, Centers for Disease Control and Prevention. 2010 [cited 2015 Feb 13]. Available: http://www.cdc.gov/std/treatment/2010/genital-ulcers.htm.
- 7. Watson-Jones D, Gumodoka B, Weiss H, Changalucha J, Todd J, Mugeye K, et al. Syphilis in pregnancy in Tanzania. II. The effectiveness of antenatal syphilis screening and single-dose benzathine penicillin treatment for the prevention of adverse pregnancy outcomes. J Infect Dis. 2002 Oct;186(7):948–57. pmid:12232835
- 8.
WHO. Provision of Effective Antenatal Care: Integrated Management of Pregnancy and Childbirth (IMPAC). Geneva, Switzerland: Department of Making Pregnancy Safer (MPS) and World Health Organization (WHO); 2002.
- 9. Gloyd S, Chai S, Mercer MA. Antenatal syphilis in sub-Saharan Africa: missed opportunities for mortality reduction. Health Policy Plan. 2001 Mar;16(1):29–34. pmid:11238427
- 10. Potter D, Goldenberg RL, Read JS, Wang J, Hoffman IF, Saathoff E, et al. Correlates of syphilis seroreactivity among pregnant women: the HIVNET 024 Trial in Malawi, Tanzania, and Zambia. Sex Transm Dis. 2006 Oct;33(10):604–9. pmid:16601659
- 11. Herring A, Ballard R, Mabey D, Peeling RW, WHO/TDR Sexually Transmitted Diseases Diagnostics Initiative. Evaluation of rapid diagnostic tests: syphilis. Nat Rev Microbiol. 2006 Dec;4(12 Suppl):S33–40. pmid:17366685
- 12. Mabey D, Peeling RW, Ballard R, Benzaken AS, Galban E, Changalucha J, et al. Prospective, multi-centre clinic-based evaluation of four rapid diagnostic tests for syphilis. Sex Transm Infect. 2006 Dec;82(Suppl 5):v13–6. pmid:17215274
- 13.
Anderson T. Evluating Diagnostics: Introducing evidence-based measures to an unregulated world. TDRnews. 2009 Jun;No. 83:12–7.
- 14. Schito M, Peter TF, Cavanaugh S, Piatek AS, Young GJ, Alexander H, et al. Opportunities and challenges for cost-efficient implementation of new point-of-care diagnostics for HIV and tuberculosis. J Infect Dis. 2012 May;205 Suppl: S169–80. pmid:22457286
- 15. Palamountain KM, Baker J, Cowan EP, Essajee S, Mazzola LT, Metzler M, et al. Perspectives on Introduction and Implementation of New Point-of-Care Diagnostic Tests. J Infect Dis. 2012 May;205(suppl 2):S181–90. pmid:22402038
- 16. Peeling RW, Holmes KK, Mabey D, Ronald A. Rapid tests for sexually transmitted infections (STIs): the way forward. Sex Transm Infect. 2006 Dec;82 Suppl 5:v1–6. pmid:17151023
- 17. Parekh BS, Kalou MB, Alemnji G, Ou C- Y, Gershy-Damet G- M, Nkengasong JN. Scaling up HIV rapid testing in developing countries: comprehensive approach for implementing quality assurance. Am J Clin Pathol. 2010 Oct;134(4):573–84. pmid:20855638
- 18. Shott JP, Galiwango RM, Reynolds SJ. A Quality Management Approach to Implementing Point-of-Care Technologies for HIV Diagnosis and Monitoring in Sub-Saharan Africa. J Trop Med. 2012;2012.
- 19. Schmid G. Economic and programmatic aspects of congenital syphilis prevention. Bull World Health Organ. 2004 Jun;82(6):402–9. pmid:15356931
- 20. Kuznik A, Lamorde M, Nyabigambo A, Manabe YC. Antenatal Syphilis screening using point-of-care testing in Sub-Saharan African countries: a cost-effectiveness analysis. PLoS Med. 2013 Nov;10(11):e1001545. pmid:24223524
- 21. Kahn JG, Jiwani A, Gomez GB, Hawkes SJ, Chesson HW, Broutet N, et al. The cost and cost-effectiveness of scaling up screening and treatment of syphilis in pregnancy: a model. PLoS One. 2014 Jan;9(1):e87510. pmid:24489931
- 22. Levin C, Steele M, Atherly D, García S, Tinajeros F, Revollo R, et al. Analysis of the operational costs of using rapid syphilis tests for the detection of maternal syphilis in Bolivia and Mozambique. Sex Transm Dis. 2007;34(7 suppl):S47–54. pmid:17220812
- 23. Sweeney S, Mosha JF, Terris-Prestholt F, Sollis K a, Kelly H, Changalucha J, et al. The costs of accessible quality assured syphilis diagnostics: informing quality systems for rapid syphilis tests in a Tanzanian setting. Health Policy Plan. 2014 Aug; 29(5):633–41 pmid:23894075
- 24. Owusu-Edusei K, Tao G, Gift TL, Wang A, Wang L, Tun Y, et al. Cost-effectiveness of integrated routine offering of prenatal HIV and syphilis screening in China. Sex Transm Dis. 2014 Feb;41(2):103–10. pmid:24413489
- 25. Strasser S, Bitarakwate E, Gill M, Hoffman HJ, Musana O, Phiri A, et al. Introduction of rapid syphilis testing within prevention of mother-to-child transmission of HIV programs in Uganda and Zambia: a field acceptability and feasibility study. J Acquir Immune Defic Syndr. 2012 Nov 1;61(3):e40–6. pmid:22820810
- 26. García PJ, Cárcamo CP, Chiappe M, Valderrama M, La Rosa S, Holmes KK, et al. Rapid Syphilis Tests as Catalysts for Health Systems Strengthening: A Case Study from Peru. PLoS One. 2013;8(6):e66905. pmid:23840552
- 27.
Standard Diagnostics. SD Bioline 3.0 Syphilis anti-TP Test: Product Insert [Internet]. Yongin-Si, South Korea; 2009. p. 2. Available: http://www.ctr.com.mx/img_prod/sifilis30_i.pdf.
- 28. Jafari Y, Johri M, Joseph L, Vadnais C, Pant Pai N. Poor Reporting of Outcomes Beyond Accuracy in Point-of-Care Tests for Syphilis: A Call for a Framework. AIDS Res Treat. Hindawi Publishing Corporation; 2014 Jan;2014:465932. pmid:24795821
- 29.
EGPAF. Quality Management Brief: Rapid Syphilis Testing Study in Zambia and Uganda [Internet]. 2011. Available: http://b.3cdn.net/glaser/8612ec09ed7d57624a_5sm6ii32e.pdf.
- 30.
Strasser S, Vwalika C, Njobvu P, Ncube A, Tembo-Mumba G, Sripipatana T, et al. Translating HIV integration research into policy and practice change: lessons learned from Zambia. 19th International AIDS Conference: Abstract no THPE722 [Internet]. 2013. Available: http://iasociety.org/Abstracts/A200744667.aspx.
- 31.
Tembo-Mumba G. Riding the road to introduction and scale up of rapid syphilis tests in Zambia. Global Syphilis Meeting. London, UK; 2012.
- 32.
Ansbro EM, Gill M, Reynolds J, Shelley KD, Strasser S, Sripipatana T, et al. Introduction of syphilis point-of-care tests, from pilot study to national programme implementation in Zambia: a qualitative study of impact onf healthcare workers’ perspectives on testing experience, training and quality assurance. In review. 2015.
- 33. Conteh L. Cost and unit cost calculations using step-down accounting. Health Policy Plan. 2004 Feb;19(2):127–35. pmid:14982891
- 34.
Terris-Prestholt F, Santos A, Sweeny S, Kumaranayake L. Guidelines for Cost Effectiveness Analysis of Syphilis Screening Strategies. The Rapid Syphilis Test Toolkit: A Guide to Planning, Managment, and Implementation [Internet]. 2010. Available: https://www.lshtm.ac.uk/itd/crd/research/rapidsyphilistoolkit/rapid_syphilis_test_toolkit.pdf.
- 35.
OANDA. Historical Exchange Rates [Internet]. 2010. Available: http://www.oanda.com/currency/historical-rates/.
- 36. Kumaranayake L. How to do (or not to do) … The real and the nominal ? Making inflationary adjustments to cost and other economic data. Health Policy Plan. 2000;15(2):230–4. pmid:10837047
- 37.
Bureau of Labor Statistics. Consumer Price Index [Internet]. Washington, DC; 2012. Available: http://www.bls.gov/cpi/
- 38.
Zambia National Statistical Office. Zambia: Inflation, average consumer prices (Index) [Internet]. International Monetary Fund. 2011. Available: http://www.imf.org/external/pubs/ft/weo/2012/02/weodata/index.aspx.
- 39.
OANDA. Historical Exchange Rates [Internet]. 2012. Available: http://www.oanda.com/currency/historical-rates/.
- 40.
Creese A, Parker D. Cost analysis in primary health care: a training manual for programme managers [Internet]. Geneva, Switzerland: World Health Organization; 1994. 147 p. Available: http://apps.who.int/iris/bitstream/10665/40030/1/9241544708.pdf?ua=1.
- 41. Walker D, Kumaranayake L. How to do (or not to do) … Allowing for differential timing in cost analyses : discounting and annualization. Health Policy Plan. 2002;17(1):112–8. pmid:11861593
- 42. Larson B, Lembela-Bwalya D, Bonawitz R, Hammond EE, Thea DM, Herlihy J. Finding a Needle in the Haystack: The Costs and Cost-Effectiveness of Syphilis Diagnosis and Treatment during Pregnancy to Prevent Congenital Syphilis in Kalomo District of Zambia. PLoS One. 2014 Jan;9(12):e113868. pmid:25478877
- 43. Blandford JM, Gift TL, Vasaikar S, Mwesigwa-Kayongo D, Dlali P, Bronzan RN. Cost-effectiveness of on-site antenatal screening to prevent congenital syphilis in rural eastern Cape Province, Republic of South Africa. Sex Transm Dis. 2007 Jul;34(7 Suppl):S61–6. pmid:17308502
- 44. Schackman BR, Neukermans CP, Fontain SNN, Nolte C, Joseph P, Pape JW, et al. Cost-effectiveness of rapid syphilis screening in prenatal HIV testing programs in Haiti. PLoS Med. 2007 May;4(5):e183. pmid:17535105
- 45.
Bonawitz RE, Duncan J, Hammond E, Hamomba L, Namule J et al. Introduction of Rapid Syphilis Tests Improve Syphilis Testing Services but not Treatment in Antenatal Care Services in Zambia. In review. 2014.
- 46. García SG, Tinajeros F, Revollo R, Yam EA, Richmond K, Díaz-Olavarrieta C, et al. Demonstrating public health at work: a demonstration project of congenital syphilis prevention efforts in Bolivia. Sex Transm Dis. 2007 Jul;34(7 Suppl):S37–41. pmid:17179776
- 47.
Zambia MoH. National programme for the prevention and control of sexually transmitted infection: guidelines for use of rapid syphilis tests in Zambia. Lusaka, Zambia; 2011.
- 48. Owusu-Edusei K, Gift TL, Ballard RC. Cost-effectiveness of a dual non-treponemal/treponemal syphilis point-of-care test to prevent adverse pregnancy outcomes in sub-Saharan Africa. Sex Transm Dis. 2011 Nov;38(11):997–1003. pmid:21992974
- 49.
Bristow CC, Adu-sarkodie Y, Ondondo RO, Bukusi EA, Dagnra CA, Oo KY, et al. Multisite Laboratory Evaluation of a Dual Human Immunodeficiency Virus (HIV)/Syphilis Point-of-Care Rapid Test for Simultaneous Detection of HIV and Syphilis Infection. Open Forum Infect Dis. 2014;1–5.
- 50.
Valderrama M, Chiappe M, Garcia PJ, Lopez-Torres D. TP168: Acceptance of HIV/Syphilis duo Rapid Testing: Survey of Attitudes of Antenatal Care Providers. CDC STD Prevention Conference. 2014.
- 51.
Bristow CC, Larson E, Klausner JD. Cost-effectiveness of dual HIV and syphilis tests. AIDS: 20th International AIDS Conference [Internet]. 2014. Available: http://pag.aids2014.org/abstracts.aspx?aid=9202.
- 52. Yao K, Wafula W, Bile EC, Cheignsong R, Howard S, Demby A, et al. Ensuring the quality of HIV rapid testing in resource-poor countries using a systematic approach to training. Am J Clin Pathol. 2010 Oct;134(4):568–72. pmid:20855637
- 53. Mabey DC, Sollis K a, Kelly H a, Benzaken AS, Bitarakwate E, Changalucha J, et al. Point-of-care tests to strengthen health systems and save newborn lives: the case of syphilis. PLoS Med. 2012 Jan;9(6):e1001233. pmid:22719229