Overview

To understand the degree to which different strategies achieve the balance between antibiotic overuse and underuse, we seek to quantify the expected consequences and proportions of patients falling in both categories for each strategy. To capture the consequences of antibiotic underuse, we developed a Markov cohort model that describes febrile disease progression with and without treatment. The long-term consequences of antibiotic overuse, such as the development of bacterial resistance, were much more challenging to estimate. Consequently, we assigned a monetary penalty (weight) to every patient unnecessarily prescribed a course of antibiotics, given the disease etiology, in order to capture how different values for this weight might drive optimal decision-making.

Balancing antibiotic overuse and underuse

We first considered a simplified setting where patients presenting with fever had one of four possible disease etiologies: 1) a specific bacterial infection for which a diagnostic test exists and is treatable with a commonly prescribed antibiotic (i.e., doxycycline [12, 13]); 2) other bacterial infections for which no test is available, but also treatable with the same antibiotic; 3) a specific viral infection for which a test exists and it is not treatable with the antibiotic; 4) other infections for which there is no test and are not treatable with the antibiotic.

In this setting, we considered three strategies: 1) empirical antibiotic treatment (prescribing antibiotics to all patients, without any tests); 2) testing all patients using the bacterial infection test, and prescribing antibiotics to patients with positive results; 3) testing all patients using the viral infection test, and prescribing antibiotics to patients with negative results.

If the penalty (weight) for each patient with antibiotic-treatable (AT) bacterial disease who is not prescribed antibiotics is w_under, and the penalty (weight) for each patient with non-antibiotic-treatable (NAT) disease who is prescribed antibiotics is w_over, then we choose a strategy that minimizes the expected total penalty per patient:

The first and second term corresponds to antibiotic underuse and overuse penalty, respectively, associated with each strategy. We rewrite this objective as (1)

We derived analytical expressions of antibiotic overuse and underuse probabilities for each strategy, estimated the weights (S1 Text, Section 2), and evaluated all three generic strategies using the objective defined in Eq (1) in order to understand the role that overuse penalties, along with disease probabilities of occurrence and test accuracy parameters, play into determining the best strategy among the three options. Notably, we found simple conditions involving only the probabilities of disease occurrence and the ratio w_under / w_over that guarantee that administering a specific test leads to a lower total penalty than administering the other test or no test at all (and administering antibiotics to all patients).

Viral- versus bacterial-endemic settings: a case study for Thailand

After gaining intuition from analyzing the simplified version of the balancing problem, we turned to a more realistic setting in order to account for multiple disease scenarios, incorporate test, treatment and other direct and indirect costs, as well as the effect of delays in seeking care and in test turnaround times (TAT).

We categorized sources of infections into the following five types based on whether they are bacterial or viral, and whether they are treatable with antibiotic: 1) Leptospirosis (bacterial, treatable with the antibiotic); 2) Scrub typhus (bacterial, treatable with antibiotic); 3) “Other bacterial”, treatable with antibiotic, such as spotted fever group rickettsioses [13]; 4) Dengue (viral, not treatable with antibiotic); and 5) All other possible causes of infection as a general category of “others” which are assumed not to be treatable with antibiotic. We assumed that all cases were malaria negative, which is consistent with current practices of screening with highly accurate rapid tests for malaria first [22–24].

We selected two contrasting settings in Thailand with varying infection occurrence probabilities (Table 1): Northern Thailand, where leptospirosis, scrub typhus (bacterial infections) are more endemic–Scenario A [1], and Bangkok, where dengue (viral) is the most common cause of acute fever–Scenario B [21].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Probabilities of common causes of febrile diseases among hospital patients in Thailand in selected settings with contrasting occurrence probabilities (obtained from [1, 21]).

https://doi.org/10.1371/journal.pone.0227409.t001

The disease progression model

We developed detailed Markov cohort models to capture disease progression for each febrile disease and used the model outcomes to evaluate alternative strategies for diagnosis and antibiotic treatment. We modeled a hypothetical cohort of 40-year-old adult patients (age varied in sensitivity analysis) with acute, undifferentiated fever since symptom onset. We set the time horizon to 45 days since most patients would be either recovered or deceased by then. We tracked patient health states daily. On each day, patients may recover, progress to severe disease stage or die. We considered patients presenting to hospitals on their first, fourth or tenth day of illness, which captures the minimum, average, and maximum of time for patients presenting to hospitals seeking care. We simplified the health states for any infection type to four core states: Mild, Severe, Recovered and Death. The exact daily transition probabilities depended upon the specific infection etiology and treatment (i.e., with or without antibiotics). We assumed that all Severe patients were immediately hospitalized and assigned to a diagnostic and treatment strategy if it was the first hospital visit (no prior testing). We also assumed that deaths during the 45-day time horizon were caused only by severe complications of infections.

Diagnostic and treatment strategies

In the two Thai settings, we evaluated fifteen strategies with different test and treatment options towards febrile disease management (Table 2). Treatment for leptospirosis and scrub typhus was assumed to be a one-week course of antibiotics (e.g., doxycycline) [12, 13]. For dengue and other viral infections, supportive care is standard practice and no antibiotic treatment was assumed. Diagnostic testing included pathogen-specific rapid diagnostics tests (RDT) and Polymerase Chain Reaction (PCR) tests for leptospirosis, typhus, and dengue. A positive bacterial test result or a negative viral test result would lead to the prescription of antibiotics. Although still not widely available in health care settings, we also considered the use of nucleic acid amplification for all three pathogens in a single test (e.g., multiplex PCR) [25, 26]. We assumed TAT was one day and two days for RDT and for PCR, respectively (varied in sensitivity analysis). In both Thailand settings examined, the probability of presenting with leptospirosis was higher than scrub typhus infection. As a consequence (and to limit the total number of possible candidate strategies), we did not include typhus-led testing strategies (i.e., single typhus RDT/PCR). We evaluated additional candidate strategies in S1Text, Section 7.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Strategies evaluated for two Thai settings using Markov cohort models.

https://doi.org/10.1371/journal.pone.0227409.t002

We assumed all Mild state patients received outpatient care (with or without antibiotic prescription), where practices such as follow-up of patients, adequate bed rest and fluid intake are common. Patients who progressed to Severe state (i.e., patients demonstrating various clinical signs in severe febrile illness) were hospitalized. The current protocol recommended by the WHO in resource-limited settings is empirical antibiotic treatment to patients presenting with severe illness [12]. For comparison purposes, we also considered an expanded empirical treatment strategy that includes treating both Mild and Severe patients with antibiotics without any diagnostic testing. Lastly, we also considered a “No Antibiotics” strategy to monitor natural disease progression, where no test and no antibiotics were given, but patients in Severe states were hospitalized. A diagram of patient flow through the model can be found in S1 Fig.

Outcomes measured

Outcome measures included:

1. Costs and health outcomes: direct and indirect healthcare costs incurred and health burden (in DALYs, definition and calculation procedure in S1 Text, Section 3.1.3),
2. Antibiotics underuse and overuse. For a given strategy, we tracked the proportion of patients over- and under-treated with antibiotics (Prob(over) and Prob(under)).

To account for antibiotic overuse when evaluating the strategies, we first considered the three-dimensional outcome space consisting of DALYs, costs, and Prob(over) instead of the standard two-dimensional outcome space consisting of only DALYs and costs. Secondly, for each strategy, we assigned a penalty per unnecessarily prescribed course of antibiotics (w_over), and monetized DALYs by assigning a willingness-to-pay (WTP) per DALY averted. DALYs averted, incremental costs and the incremental Prob(over) for a given strategy is the difference between the DALYs, costs, and Prob(over) incurred by the strategy and those incurred by the “No tests/No Antibiotics” strategy (Strategy 1 in Table 2). In our main analysis, we set the WTP value to be Thailand’s GDP per capita in 2016: $5,907.91 [27]. Then for each strategy, we computed its resulting net monetary benefit (NMB), where

By considering a range of values for willingness-to-pay per DALY averted, and for penalty per unnecessary course of antibiotics, we quantified the ranges of the NMB of each strategy, and conducted a three-dimensional cost-effectiveness analysis comparing different strategies. We provide discussion on the form of augmented cost-effective analysis in S1 Text, Section 4, and the methodology for finding a three-dimensional effectiveness frontier in S1 Text, Section 5.

Markov model calibration

The transition probabilities were calibrated to match quantities reported in the medical literature, such as mean duration of fever, and mortality rates with or without antibiotics. Test costs, test sensitivities and specificities, health costs and DALY-related parameters for all possible causes of infection were obtained from the published literature. More detailed explanations, exact values, and references are provided in S1 Text, Section 3.1 and S1 Table.

Sensitivity analysis

We conducted univariate and probabilistic sensitivity analyses by varying the disease occurrence probabilities, antibiotic effectiveness to lower disease progression from Mild to Severe, delay in patients presenting for care, as well as other parameter values including test sensitivities and specificities, test and treatment costs, health utilities and disease progression parameters.

Balancing antibiotic over- and underuse

We denoted by p_bac the probability of presenting with bacterial infection for which a test exists, the probability of presenting with bacterial infection for which no test exists, p_viral the probability of presenting with a viral infection for which a test exists, and p_other the probability of presenting with infections not treatable with the primary antibiotic for which no test exists. We assumed that all test sensitivities and specificities fall between 50% and 100%, and that any test’s sensitivity is less than or equal to any specificity, and the sum of the four disease occurrence probabilities (p_bac, , p_viral and p_other) is 1.

Our metric for evaluation was the total weighted penalty of antibiotic under- and overuse (Eq (1)) in the generic setting with four infection types and three treatment strategies (antibiotic to all patients; bacterial test and if positive administer antibiotic; viral test and if negative administer antibiotic). When comparing two of the strategies at a time, we found the following sufficient conditions that guarantee strategy dominance. Notably, these conditions only involve a minimal number of parameters, and do not depend on test accuracy and illness stage:

• If , then both the bacterial and the viral test strategies have a smaller overall penalty than the empiric antibiotic strategy. Here, is the overall probability of presenting with bacterial disease, treatable with antibiotics; w_under or w_over is the penalty per patient under- or over- treated with antibiotics, respectively. This condition states that, if the probability of presenting with bacterial disease is low (and therefore the probability of presenting with viral disease or other, non-bacterial disease is high), or the penalty for overuse of antibiotics is large, then using a strategy that tests for either viral or bacterial disease before prescribing antibiotics is preferred to the strategy that distributes antibiotic treatment to all patients.
• If , then the strategy administering the bacterial test incurs a smaller overall penalty than the strategy administering the viral test. This condition states that, when choosing between the viral and bacterial test, if the probability of presenting with an antibiotic-treatable infection is low, or the penalty for over-treating is high, then the bacterial test strategy outperforms the viral test strategy.
• If , then the strategy administering the viral test leads to a smaller overall penalty than the strategy administering the bacterial test. This condition states that, when choosing between the viral and bacterial test, if the probability of presenting with infections non-treatable with antibiotic is low, or the penalty for under-treatment is high, then the viral test strategy outperforms the bacterial test strategy.

Numerical results: Bacterial- versus viral-endemic settings in Thailand

We now turn to evaluate health outcomes, costs and antibiotic overuse outcomes in the more realistic bacterial-endemic setting (Scenario A) and viral-endemic setting (Scenario B) in Thailand. The per-patient costs incurred (in USD), health burden (in DALY), and antibiotics overuse (Prob(over)), underuse (Prob(under)) for each strategy are shown in Table 3 for patients seeking care on the fourth day of illness (average day at hospital presentation). Results for patients seeking care on the first (min), and tenth (max) day of illness can be found in S3 Table and S4 Table. Figs 1–3 show the optimal policy (highest NMB) at varying values of WTP and antibiotic overuse penalty (w_over) for patients presenting on day one of illness (Fig 1), day four of illness (Fig 2) and day ten of illness (Fig 3).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Highest net monetary benefit policies for patients presenting on the first day of illness.

We vary willingness-to-pay (WTP) on the y-axis and penalty (w_over) on the x-axis. (A): Bacterial-endemic Scenario A (B): Viral-endemic Scenario B.

https://doi.org/10.1371/journal.pone.0227409.g001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Highest net monetary benefit policies for patients presenting on the fourth day of illness.

We vary willingness-to-pay (WTP) on the y-axis and penalty (w_over) on the x-axis. (A): Bacterial-endemic Scenario A (B): Viral-endemic Scenario B.

https://doi.org/10.1371/journal.pone.0227409.g002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Highest net monetary benefit policies for patients presenting on the tenth day of illness.

We vary willingness-to-pay (WTP) on the y-axis and penalty (w_over) on the x-axis. (A): Bacterial-endemic Scenario A (B): Viral-endemic Scenario B.

https://doi.org/10.1371/journal.pone.0227409.g003

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Per-patient costs (USD), DALYs incurred, antibiotic overuse (Prob(over)) and underuse (Prob(under)) for febrile patients seeking care on the fourth day (average day) of illness and undergoing various test and treat strategies.

https://doi.org/10.1371/journal.pone.0227409.t003

As shown in Table 3, Figs 1–3, S3 Table, and S4 Table, under a standard analysis that considers only the dimensions of health benefit (DALYs averted) and incremental costs, without penalizing antibiotic overuse, empirical antibiotic treatment to all patients (Strategy 2) always dominated all other strategies, regardless of when patients presented for care or which endemic scenario we considered. When patients sought care on day four of illness (Table 3), empirical antibiotic treatment incurred the lowest cost: $100.6/patient in bacterial-endemic Scenario A and $108.7/patient in viral-endemic Scenario B. This strategy also incurred the lowest DALYs of 0.9 DALYs/patient in Scenario A and 0.7 DALYs/ patient in Scenario B. If patients delayed in presenting for care to day ten of illness (S4 Table), both costs and DALYs increased (compared to day four), but still remained the lowest for Strategy 2 at $149.5/patient (Scenario A), $122.1/ patient (Scenario B) and 1.7 DALYs/patient in Scenario A, and 0.9 DALYs/patient in Scenario B. For patients presenting to the hospital on the first day of illness (S3 Table), both costs and DALYs decreased compared to day four: for Strategy 2, costs were $47.5/patient (Scenario A) and $94.1/ patient (Scenario B) and 0.2 DALYs/patient in Scenario A, and 0.5 DALYs/patient in Scenario B.

However, Strategy 2 resulted in the highest likelihood of antibiotic overuse per patient (Scenario A: 38.1%, 19.3%, 7.5% for patients presenting on day one, day four, and day ten respectively; Scenario B: 82.9%, 42.1%, 16.3% for day one, day four, and day ten respectively) (Table 3, S3 Table, S4 Table). In Scenario A, Strategy 2 became suboptimal with respect to NMB (at WTP = Thailand’s GDP per capita) when the antibiotic overuse penalty was larger than $12,800/course, $18,400/course, $23,900/course for patients presenting on day one, day four, and day ten, respectively (Figs 1–3). In the viral-endemic Scenario B, Strategy 2 became suboptimal when the antibiotic overuse penalty was larger than $1,100/course, $1,500/course, $1,600/course for patients presenting on day one, day four, and day ten, respectively (Figs 1–3). When setting the penalty (w_over) to be equal to the WTP (Thailand’s GDP per capita), we found that empirical antibiotic treatment (Strategy 2) was only optimal in Scenario A (not in Scenario B), which is consistent with the analytical results (detailed estimation procedure in S1 Text, Section 3.2).

Strategy 3 (empirical antibiotic treatment to patient in Severe states only) was optimal in two settings and had very small optimal regions at low WTP and low penalty. In Scenario A on day four (Fig 2A), Strategy 3 was optimal for a 1 to 3 ratio of WTP to penalty, from $0 to $300 WTP. In Scenario B on day one (Fig 1B), Strategy 3 was optimal for WTP from $0 to $20 and penalty from $120/course to $130/course.

Compared to Scenario A, we observed that single test strategies (Dengue PCR or Dengue RDT, and/or Lepto RDT) had the highest NMB for a much wider range of WTP and penalty values in Scenario B than in Scenario A. For example, when we fixed the WTP to Thailand’s GDP per capita, and presentation day as the first day of illness (Fig 1), Dengue PCR was optimal in terms of NMB when the overuse penalty was about $13,000/course in Scenario A. On the other hand, Dengue PCR was optimal for penalty between $1,100/course to $13,000/course in Scenario B. These findings were consistent with our analytical results when balancing antibiotics overuse and underuse: testing strategies are more likely to be optimal when the probability of presenting with AT disease is smaller. Lastly, we observe from Fig 1 that Multiplex PCR test (Strategy 15), with sensitivities ≥ 90% for each disease component, dominated other strategies for penalty ≥ $13,000/course in both settings when the WTP per DALY averted is set to Thailand’s GDP per capita.

Policies with highest NMB for patients presenting on the fourth day of illness (Fig 2) are similar to those for patients presenting on the first day of illness (Fig 1). Lastly, RDT-led strategies had higher NMB when patients presented on the tenth day of illness (Fig 3). This is because test sensitivities of RDT tests are higher than those of PCR tests when patients present to hospitals late (S1 Table).

The flexibility of the model allows us to consider multiple additional strategies. For example, in settings where viral disease is more prevalent, healthcare decision-makers and administrators might consider sequential test strategies that include viral disease tests. We described in supplementary material (S1 Text Section 7, S8 Table and S9 Table) four additional sequential strategies starting with a dengue test, and if negative, a test for leptospirosis. The dengue-led strategies were optimal with respect to NMB for WTP values between $0 and $250 in Scenario A, and between $0 and $4,000 in Scenario B (S13– S15 Figs).

Sensitivity analysis

The general structure of the optimal strategy sequence when varying antibiotic overuse penalty while setting the WTP to Thailand’s GDP per capita did not change when considering: 1) two additional disease probability distribution scenarios in Thailand (S1 Text, Section 6.1); and 2) reduced antibiotic effectiveness in preventing Mild patients progressing to Severe states (S1 Text, Section 6.2). In all cases (including the base-case) considered, RDT-led strategies became optimal in terms of NMB when patients presented on the tenth day of illness. We also observed that patients who presented early incurred better health outcomes.

Under additional disease probability distributions, test strategies were optimal at lower probability of bacterial disease and higher penalty (w_over). In the average day-of-presentation (day four) scenario and the high bacterial disease prevalence setting (Set 1 in S5 Table), Strategy 15 (Multiplex PCR test) had the highest NMB for penalties greater than $20,000/course (S3 Fig). In the low bacterial disease prevalence setting (Set 2 in S5 Table), Strategy 15 (Multiplex PCR test) had the highest NMB for penalties greater than $8,500/course (S4 Fig).

Reduced antibiotic effectiveness in preventing progression to Severe states makes Strategy 2 (empirical antibiotic treatment to all patients) less preferable: For example, in Scenario A, when fixing the WTP to Thailand’s GDP per capita and assuming patients present on day four (S8 Fig), Strategy 2 had the highest NMB for penalties less than $18,000/course, $16,000/course, and $12,000/course when antibiotic effectiveness was assumed to be 100%, 75%, and 50% respectively.

We performed one-way sensitivity analysis for all variables while fixing the disease occurrence probabilities to the ones in Scenario A, and assumed that patients presented on the average day of presentation (day four) to hospitals (S1 Text, Section 6.3). We found that empirical antibiotic treatment to all patients had the highest NMB at younger patient age, lower leptospirosis-test sensitivity, higher disease-specific mortality rate, and longer TAT (S2 Table, S5 Fig). Varying remaining parameters, such as test, health costs and disability weights, did not change our conclusions.

Lastly, we conducted a probabilistic sensitivity analysis (PSA) by varying all model parameters simultaneously using Monte Carlo simulation (S1 Text, Section 6.4, and S6 Fig). Our analyses indicated that when the penalty on antibiotics overuse was between $0 to $10,000/course, empirical treatment to all patients was most likely optimal, while strategies with single dengue tests (Strategy 5 Dengue PCR and Strategy 4 Dengue RDT) could be alternatives to reduce antibiotics overuse. Another critical observation was that PCR-led strategies (Strategy 15, Strategy 11) had a much higher chance of being optimal (40% probability for Strategy 15, 25% probability for Strategy 11) than RDT-led strategies even when delay in seeking care (0–10 days) and TAT (0–3 days) were randomly sampled, suggesting that PCR-led strategies were robust under the sampled parameter distributions.