Compartmental models

We considered four epidemiological models: a Birth-Death (BD) model (Fig 1a), a Susceptible-Infected-Removed (SIR) model without demography (i.e. with a constant host population size, Fig 1b), a Susceptible-Infected with Differential-Risk (SI-DR) model and a Birth-Death model with an Exposed class (BDEI, Fig 1c).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. The epidemiological models.

(A) The Birth-Death (BD) model. (B) The Susceptible Infected Removed (SIR) model. (C) The Susceptible Infected with Differential Risk (SI-DR) model. (D) The Birth-Death model including an Exposed class (BDEI). The four compartments correspond to susceptible (S), exposed (E), infectious (I) and removed (R) individuals. In BD and BDEI models, new infections arise at a constant (“birth”) rate β per infectious individual. In SIR and SI-DR models, the number of new infections depends on the number of susceptible individuals, the transmission rate β and the number of infectious individuals. In the SI-DR model, it also depends on the contact rates c_i associated with each risk group i = {1; 2}, and an assortativity term p_ij (j = {1; 2}). In both SIR and SI-DR models, the total host population size is assumed to be constant (N). In all models, infections end (i.e. “die”) at a rate γ. All models, except the SI-DR model, account for sampling that occurs at a rate ε. The SI-DR model accounts for demography (new susceptible individuals arise at a rate Λ_i and all individuals die at a rate μ).

https://doi.org/10.1371/journal.pcbi.1005416.g001

These compartmental models are defined by ordinary differential equation (ODE) systems (see [40] for the SI-DR model and S1 Text for the three other models).

In these models, individuals susceptible to the pathogen become infected after contact with infectious individuals and successful transmission, which occurs at an overall transmission rate β [2], except for the SI-DR model [40] where the transmission rate is equal to β c_i h_ij depending on the risk groups of the “infectee” (i = {1; 2}) and the “infector” (j = {1; 2}). In the latter model, c_i is the contact rate of the individuals belonging to risk group i and the h_ij are the elements of an assortativity matrix (which [40] refers to as an “homophily” matrix) that describes the propensity of individuals from risk group i to have contact with individuals from risk group j (see [40] for more details on the computation of this matrix).

Following infection, individuals either become infectious immediately (BD, SIR and SI-DR models) or at a rate σ after a latency period in the Exposed class (BDEI model). They are then “removed” (i.e. recover with a lifelong immunity or die) at a rate γ. Lastly, they can be sampled, at a rate ε. By sampling, we mean that the pathogen is sequenced from the patient. Because sampling generally leads to treatment or at least to behavioral changes, we assumed that infected individuals are also “removed” after sampling. This assumption is commonly made in phylodynamics [21, 41, 42] and we kept it here to facilitate comparisons. However, it could easily be relaxed. The sampling proportion p is defined as the ratio of the sampling rate (ε) over the total removal rate (γ + ε).

The critical difference between BD models and the SIR model, lies in the transmission rate per infected individual λ(t): this rate is constant in BD models (λ(t) = β), but it depends on the susceptible population size in the SIR model (, where S(t) is the number of susceptible individuals at time t and N is the effective population size). In other words, the SIR model assumes that the effective host population has a fixed size N and is initially fully susceptible (S(t = 0) = N). The susceptible population is depleted as the epidemic spreads (S(t > 0) < N) and this depletion decreases the speed of the spread of the epidemic (λ(t > 0) < λ(t = 0)).

In the SI-DR model used in [40], the number of new infections also depends on the susceptible population size, but there is no sampling because the model assumes that the sampling dates are known. The SI-DR model also accounts for demography since all individuals die at a rate μ and susceptible newborns of risk group i appear at a rate Λ_i.

Our overall goal was to infer a vector of epidemiological parameters θ, from time-scaled phylogenies. For reasons related to method comparison, the composition of θ depends on the model:

• ,
• ,
• θ_SI-DR = {c₁; β; γ; N = S + I}; c₂, μ, ρ and f being fixed,
• .

Contrary to the likelihood-based phylodynamics methods [8, 41, 42], we did not attempt to infer the sampling proportion using ABC, since only two out of the three parameters (β, γ and ε) are identifiable in the epidemiological models that account for sampling (see S1 Text) [43].

Simulation of sampled transmission trees

The compartmental models described above are deterministic continuous-time models. However, whatever method is used (likelihood-based or not), epidemiological parameter inference requires taking the stochasticity of events at the individual level into account.

A time-scaled phylogeny of an epidemic can be viewed as a sampled transmission tree in which each branching represents a transmission and each leaf represents a sampled infected individual. There are several ways to simulate sampled transmission trees from epidemiological models. They all involve two processes: the simulation of the trajectory of the epidemic (i.e. the chronology of epidemiological events) and the construction of the sampled transmission tree based on this trajectory. In this study, we used two tree simulation approaches that can be applied to a wide variety of epidemiological models.

The first approach is implemented in the software MASTER [44] and is based on Gillespie’s direct method [2, 45] also known as the Stochastic Simulation Algorithm (SSA). This algorithm enables epidemiological models to be converted into event-driven models. A great advantage of the SSA is that there is an exact correspondence between the stochastic simulations and the deterministic ODE-based model. With this approach, trees are generally simulated alongside the trajectory, that is, through a forward-in-time birth-death process, where each birth in the tree corresponds to a transmission and each death corresponds to an end of infection with or without sampling. Unless the epidemiological model includes sampling as an event, MASTER produces full transmission trees. The computational complexity of this method is linear with respect to the total event count () with an additional time penalty associated with the tree update [44]. For the BD and the SIR models, is the sum of the numbers of birth and death events. To obtain a sampled transmission tree of n leaves simulated assuming a sampling proportion p with either model, we need to simulate a full transmission tree composed of leaves (and internal nodes). Thus we need events (births and deaths) to be performed. Gillespie’s SSA complexity is then in , where is at most proportional to with large n, for both models.

The second approach has been implemented in the rcolgem R package [23, 24]. In this approach, epidemiological models are translated into continuous-time stochastic models to simulate trajectories. Trees are simulated afterwards based on trajectories, through a backward-in-time coalescent-like process. The coalescent approach assumes that sampling dates are known, which means the epidemiological models do not require assumptions about the sampling process. With careful implementation and reasonable approximation, the trajectory can be generated in a time that is proportional to the simulated epidemic duration (t_end − t₀) over the chosen time step (δt), and the tree can be built in a time that is proportional to its size (n). This approach becomes valuable when , n representing the number of leaves. This can be the case, for instance, when simulating large trees with very sparse sampling or for epidemiological models more complex than the SIR model, where the number of events does not depend only on the tree size and sampling proportion.

We used the MASTER-like approach for the BD, SIR and BDEI models, which all include sampling, and the rcolgem R package for the SI-DR model. Note that we implemented our own SSA instead of using MASTER to facilitate the addition of constraints on the simulations (see below).

Summary statistics

Sampled transmission trees are complex objects. Therefore, we used summary statistics to compare them and capture the epidemiological information they may contain. We decided to compute many summary statistics to capture as much information as possible. This was motivated by the fact that there is no consensus in the field regarding which summary statistics to use. Importantly, our decision was made possible by the existence of efficient regression models that perform variable selection and can be combined to ABC (see below). Overall, we used 83 summary statistics, which we grouped into three “families” to better identify where the epidemiological information is in the phylogeny: branch lengths (Table 1), tree topology (Table 2) and Lineage-Through-Time (LTT) plot (Table 3) [46].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Summary statistics based on branch lengths (bl set).

^• Statistics computed on three time-based parts of the tree. Internal branches belong respectively to the first (k = 1), second (k = 2) or third (k = 3) part of the tree if they end before the first, second or third delimitation, respectively.

^÷ Ratios between each piecewise statistic related to internal BL and the same statistic computed on all external BL.

https://doi.org/10.1371/journal.pcbi.1005416.t001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Summary statistics based on the tree topology (topo set).

https://doi.org/10.1371/journal.pcbi.1005416.t002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Summary statistics based on the LTT plot (ltt set).

^• Computed on three part of the tree. Consecutive steps up respectively to the first (k = 1), second (k = 2) or third (k = 3) part of the tree if the second steps happens before the first, second or third delimitation, respectively.

https://doi.org/10.1371/journal.pcbi.1005416.t003

Since branching occurs throughout the phylogeny at a rate that varies over time (the number of infected and susceptible hosts vary in the SIR model), we designed all the summary statistics related to branching and internal branches (linking two internal nodes) in a piecewise manner (Table 1). We temporally cut the tree into three equal parts: internal branches belong respectively to the first, second or third part of the tree, if they end before the first (), second () or third (max_H) delimitation, respectively, where max_H represents the height of the farthest leaf. We only computed global summary statistics (on the whole tree) to describe sampling events and external branches (linking internal nodes to the leaves).

It is known that the topology of a phylogeny can be driven by processes such as immune escape [10]. Moreover, it has been shown recently that different transmission patterns can result in quantitatively different phylogenetic tree topologies. In particular, heterogeneity in host contact can influence the tree balance [49]. That is why we also used phylogenetic topological indexes as summary statistics (Table 2).

The Lineage-Through-Time (LTT) plot provides a graphical summary of a phylogeny [46]. It represents the number of lineages along the phylogeny as a piecewise constant function of time (Fig 2). Each step up in the LTT plot corresponds to a branching in the phylogeny, and each step down to a leaf. If all the infected individuals of an epidemics are sampled, the phylogeny corresponds to the full transmission tree and the LTT plot is identical to the prevalence curve. Therefore, as noted in earlier studies [22, 51–53], it is reasonable to presume that this plot could contain relevant information about epidemiological parameters. We summarized the LTT plot with two sets of summary statistics: one that captures particular metrics of the plot (Table 3) and another that simply uses the coordinates of its points as “summary” statistics. For this latter set of summary statistics, because the LTT plot contains as many points as there are nodes in the phylogeny (a phylogeny of n leaves has 2n − 1 nodes), and because here we consider phylogenies with more than 100 leaves, we averaged the points into 20 equally-sized bins, thus generating 40 summary statistics (20 x-axis coordinates and 20 y-axis coordinates).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Simulated phylogenies of 100 leaves assuming a BD model and their corresponding LTT plot.

The red phylogeny was simulated assuming θ = (R₀ = 10, d_I = 5, p = 0.5) and the blue phylogeny was simulated assuming θ = (R₀ = 2, d_I = 5, p = 0.5). Different R₀s lead to different LTT plots and different tree shapes.

https://doi.org/10.1371/journal.pcbi.1005416.g002

To summarize, we used two main sets of summary statistics, the:

• sumstats set, with 43 summary statistics related to the tree and its LTT plot, which itself comprises three sets:
  • topo set: 8 topology summary statistics,
  • bl set: 26 branch-length summary statistics,
  • ltt set: 9 summary statistics related to the LTT plot,
• coords set, with 40 mean coordinates of the LTT plot.

Each summary statistic and all coordinates are computed recursively in , where n is the number of leaves in the tree. This was a key criterion for the choice of the 83 statistics and is an important reason for the efficiency of our regression-ABC.

Simulation study

We wanted to assess the potential of regression-ABC methods to infer epidemiological parameters from phylogenies. To this end, we first compared these methods to likelihood-based methods. We simulated “target” trees under several scenarios. In particular, we used the BD and the SIR epidemiological models to perform exhaustive comparisons. We expected our method to perform less well than likelihood-based methods since ABC, by definition, only approximates the likelihood function. However, practically speaking, the implementation of likelihood-based approach often requires simplifying assumptions to allow for efficient computation, which sometimes affects the results, as we show here.

We then compared a regression-ABC method to the kernel-ABC method presented by Poon [40], assuming the SI-DR model.

Simulated “training” trees for regression-ABC.

To train the regression-ABC, we simulated a set of 10,000 trees for each of the scenarios, using the same simulation system used to produce the target trees.

For comparison with likelihood-based methods on the BD and SIR models, we assumed the values of all the epidemiological parameters to be distributed in uniform priors (see Table 4). Again, for computational reasons, we imposed that the number of infected individuals through time remained lower than 3 ⋅ 10⁵ during simulation, when assuming a BD model.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Prior table.

https://doi.org/10.1371/journal.pcbi.1005416.t004

For the comparison with the kernel-ABC method, we used the same prior distributions as in [40] (S3 Text).

Data analysis: The early stages of the 2014–2015 Ebola epidemic in Sierra Leone

Stadler et al. inferred epidemiological parameters using Ebola full-genome sequences from the 2014–2015 epidemic using BEAST2 and assuming the BDEI model (BEAST2-BDEI) [8]. Even though many more sequences have been released since then, this dataset remains interesting and relevant for comparing our regression-ABC to another likelihood-based approach. From an epidemiological standpoint, it remains one of the most densely sampled outbreaks in their early phase.

For this data analysis, Stadler et al. used 72 sequences obtained from patients in Sierra-Leone by Gire et al. [60]. We therefore used the RaxML phylogeny inferred by Gire et al. [60], which was computed on 81 sequences: 3 from Guinea patients and 78 from Sierra-Leone patients. We pruned all non-Sierra-Leone leaves. To compare our estimates with theirs, we followed their protocol by also pruning 6 leaves of the phylogeny corresponding to a sub-epidemics in Sierra-Leone. The remaining 72 sequences were sampled from late May to mid-June 2014. Using the known sampling dates, we scaled the phylogeny over time using the Least-Squares Dating (LSD) software, which uses fast algorithms and achieves accuracy comparable to more sophisticated methods [61].

We assumed a BDEI model and therefore estimated R₀, d_I and the mean duration of latency d_E, as in Stadler et al. [8]. As for previous models, the sampling proportion could not be estimated together with the other parameters due to identifiability problems [43].

The Ebola epidemic in Sierra Leone is thought to have started 6 months before it was officially identified and the first sample collected [8, 60]. Since our simulations start assuming the insertion of an infectious individual in a fully susceptible population of hosts, we therefore need to consider an additional simulation parameter, origin, which, in our simulations, corresponds to the time (in days) between the beginning of the epidemic in Sierra Leone and the beginning of sampling. Over this time period, the sampling rate was assumed to be ε = 0.

We simulated a set of 10,000 “training” trees assuming a BDEI model. For comparison purposes, we first used priors identical to those used in Stadler et al. for their BEAST2-BDEI inferences (see column p ≈ 0.7 in Table 5). We then used a different interval for the prior on the sampling proportion (p ≈ 0.4), because another study suggested that the sampling proportion lies between 0.2 and 0.7 [9]. Moreover, to simulate only biologically realistic epidemiological scenarios [62], we discarded all simulations where the total number of cases rose above 50,000 individuals.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Prior table for Ebola data.

https://doi.org/10.1371/journal.pcbi.1005416.t005

As in the simulation study, we computed Spearman’s correlation coefficients between each parameter of the set of simulated trees and the summary statistics.

Rejection is a determinant step in regression-ABC with adjustment because it selects the simulated data that will be used for learning. Even if the chosen regression model is robust, it can collapse if the rejection step fails to retain a relevant training set. The goodness-of-fit test implemented in the gfit function of the abc R package [54, 63] is an important preliminary test to be made in data analysis because it indicates whether the summary statistics are informative regarding target parameters. This test uses rejection based on the Euclidean distance on normalized entries, as defined by Beaumont et al. [33].

As dating of the Ebola phylogeny seemed poorly estimated (S1 Fig), we performed an upstream test of summary statistics goodness-of-fit of the “training” set against the phylogeny.

We inferred the posterior distributions of d_E, d_I and R₀ for the Ebola phylogeny using our ABC-LASSO regression model with P_δ = 0.5. We then compared our own estimates for the epidemiological parameters of the early spread of the Ebola epidemic in Sierra Leone with those obtained using the likelihood-based methods of Stadler et al [8]. Lastly, we analyzed the variables selected by the LASSO.

Locating the epidemiological information in the phylogeny

Fig 3 shows that the summary statistics computed on the Lineage-Through-Time plot (ltt set) are those that most correlate to the epidemiological parameters of the SIR model. The summary statistics describing the branch lengths (bl set) are less correlated and the topological summary statistics (topo set) are, in general, poorly correlated to the parameters. However, the topo set becomes more informative when the tree size increases, most likely because topological patterns become more distinguishable. There is little difference in the summary statistics histograms for trees of 100 leaves and trees of 1,000 leaves, the latter being more heavy tailed. bl set summary statistics are correlated positively to the duration of infection (d_I) and correlated negatively to R₀ (S1 and S2 Tables). None of the topological summary statistics are correlated to d_I, even though they are correlated with R₀. The coordinates of the LTT plot that are the most correlated to the epidemiological parameters are those of the x-axis, which are correlated positively to d_I and negatively to R₀ (S3 and S4 Tables). Y-axis coordinates of the LTT plot strongly correlate positively with the R₀ and weakly with the effective population size N.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Heat map and histogram of Spearman’s correlations between the SIR model parameters and all sets of summary statistics for trees of 100 (A and C) or 1,000 (B and D) leaves.

In panels A and B, the colors correspond to the bl (light green), topo (dark green) and ltt (magenta) sets. Panels C and D show the coords set related to the LTT plot with x-axis (dark gray) and y-axis (light gray) coordinates. Bar heights in the histograms represent the sum of the absolute correlations of each summary statistic to the whole set of parameters. Summary statistics and coordinates are ranked from the most to the least correlated. Correlation values between each summary statistic (or coordinate) and each epidemiological parameter are displayed in the heat map, where squares are colored with a gradient going from red (highly correlated positively) to white (no correlation) and blue (highly correlated negatively). The summary statistics names and correlations values for panels A, B, C and D, are given in S1, S2, S3 and S4 Tables respectively.

https://doi.org/10.1371/journal.pcbi.1005416.g003

Overall, R₀ is the epidemiological parameter that is the most correlated to all summary statistics, which suggests that ABC approaches should be able to infer this parameter reliably. On the opposite, Fig 3 raises doubts about the ability of ABC approaches to infer the effective population size from phylogenies, because this parameter is poorly correlated to all of the summary statistics.

The correlations found for the BD model are very similar to those of the SIR model (S2 Fig and S5, S6, S7 and S8 Tables).

For the SI-DR model, which introduces host heterogeneity, the LTT plot summary statistics (ltt set) are correlated less strongly to the epidemiological parameters, whereas the y-axis coordinates of the LTT plot are correlated more strongly (S3 Fig, and S9, S10, S11 and S12 Tables). These y-axis coordinates are mostly correlated positively to c₁ (contact rate associated with risk-group 1), β (transmission rate) and N, and negatively to γ (virulence). The summary statistics of the topo set are more correlated to the SI-DR parameters when trees are non-ultrametric than when they are ultrametric. However, even for this model, correlation remains low.

Estimating the appropriate tolerance value

In this sub-section, we study the influence of the tolerance parameter used in the rejection step, on the inference error of our four ABC methods: standard rejection (ABC), rejection using the function distance between two LTT plots (ABC-D), rejection and adjustment using regularized neural networks (ABC-FFNN), and rejection and adjustment using LASSO (ABC-LASSO).

We expected the errors of inference of ABC and ABC-D to increase with tolerance. Indeed, higher tolerance values should cause the rejection step to retain trees that are increasingly dissimilar to the target tree, that is, which have been generated by parameter values that are increasingly distant from the target values. Globally, this is what we observe in Fig 4. With large tolerance values, the error seems to converge towards that of the prior (the horizontal gray line), suggesting that there is not sufficient signal in the summary statistics to infer d_I by ABC and ABC-D.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Influence of the tolerance parameter on the error for four ABC approaches used on all summary statistics.

The x-axis shows the tolerance value. Squares represent the mean relative errors for each tolerance value with their standard errors. We show errors generated by ABC-D in turquoise, by ABC in blue, by ABC-FFNN in orange and by ABC-LASSO in red. The gray horizontal lines correspond to the mean relative error of the prior (i.e. expected error in rejection with a tolerance of 1). Results are displayed for both BD and SIR models and trees of both 100 leaves and 1,000 leaves.

https://doi.org/10.1371/journal.pcbi.1005416.g004

Regarding the ABC-FFNN method, when the tolerance value increases, we expected the error to decrease at first (because the adjustment method used here requires a certain amount of training data) and finally to reach a plateau (when we have enough data and regularization can control for overfitting effects). This is the case for the inference of epidemiological parameters on small trees. For large trees, the error increases at the end for high tolerance values, which could be due to a poorly controlled regularization or to the limited size of the neural-network in the abc R function.

Concerning the ABC-LASSO method, we expected an increase in the tolerance value to decrease the inference error at first for the same reason as for the FFNN. However, in Fig 4, we only observe this effect for the SIR model with large trees. We then expected the error to reach a plateau and finally to increase because increasing the size of the training data increases the probability of non-linearity, which is problematic for the LASSO (linear) regression model. ABC-LASSO does not seem to reach the non-linearity zone in the tolerance range we considered here. The relative errors of the ABC-LASSO method remain below the threshold represented by the error induced by the prior (S5 Fig). Overall, the error with this approach is quite stable, likely due to well-controlled regularization.

We also analyzed the influence of the tolerance parameter on the 95% Highest Posterior Density (HPD) width (width_95%). As expected, the posterior distributions obtained using regression-ABC methods are more adjusted than those obtained using the ABC-D or standard ABC method (S6 Fig). The width_95% of the posteriors obtained using ABC, ABC-D or ABC-FFNN increases with the tolerance, whereas that of the ABC-LASSO posteriors seems to be insensitive to tolerance parameter.

Overall, 0.01 is the best tolerance value for rejections without adjustment, and 0.5 is the best value with adjustment. Since this result was observed for both the BD and the SIR models, we adopted these values as default values for the remainder of the study.

Comparison with likelihood-based approaches

Globally, BEAST2 achieved good convergence toward the epidemiological parameters posterior distributions, except for the large target trees simulated assuming the SIR model with p = 0.05 and R₀ = 2. For those target trees, less than 20% of the N parameter posterior distributions had an ESS above 100.

Fig 5 shows that, for the SIR model and for large trees (1,000 leaves), regression-ABC methods can approach the accuracy of the likelihood-based approach (BEAST2-BDSIR, in black) and even outperform it for the inference of the effective population size. This can be explained by the fact that the BEAST2-BDSIR assumes an approximation of the true SIR model to speed up MCMC computations. Moreover, in the BDSIR model, the approximation of the number of susceptible individuals through time, S(t), potentially makes the effective population size N hard to estimate [42].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Inference errors on epidemiological parameters of SIR model using four ABC approaches with different sets of summary statistics.

The x-axis shows the sets of summary statistics used. Squares represent mean errors with their standard errors. Empty squares correspond to results obtained on trees of 100 leaves and filled squares correspond to results on trees of 1,000 leaves. We show errors generated by ABC-D in turquoise, by ABC in blue, by ABC-FFNN in orange, by ABC-LASSO in red and by BEAST2-BDSIR in black. We show the average errors (bottom row) and the error for each parameter of interest.

https://doi.org/10.1371/journal.pcbi.1005416.g005

The standard ABC method (in blue) already provides good estimations of R₀, consistently with Spearman’s correlations (Fig 3). We also find that the Euclidean distance between LTT plot coordinates (coords set, in blue) yields more accurate estimates than the functional distance between two LTT plots (ABC-D, in turquoise). This can be explained by the fact that in the functional distance we only consider the differences on the y-axis of the LTT plots, while in the standard ABC using the coords set we also consider the differences on the x-axis, which represents the time variable and are the most correlated to epidemiological parameters (Fig 3).

The performance of both regression-ABC methods is comparable when we consider small trees, and the accuracy of epidemiological parameter inference is always better for large trees. Note that, ABC-FFNN provides highly variable results for large trees, suggesting that regularization is poorly controlled in the algorithm we used.

ABC-LASSO always gives better estimations than the standard ABC on large trees. It also gives reliable results regardless of the set of summary statistics used. This suggests that our LASSO implementation is robust concerning the high number of explanatory variables. We analyzed which variables were selected in the LASSO regression models but we did not identify any strong selection pattern. This might be explained by the fact that many variables are highly correlated. It is also a known fact that variable selection using LASSO can be unstable [64].

Results concerning the BD model are presented in S7 Fig and are globally similar to observations for the SIR model, except that none of the ABC methods outperforms BEAST2-BD. This is consistent with the fact that BEAST2-BD is based on the exact likelihood function of the BD model. Nevertheless, the accuracy of ABC-LASSO on large trees is close to that of BEAST2-BD.

Fig 6 gives the example of a particular SIR scenario (dense sampling, high R₀, and high d_I), where for large time-scaled phylogenies (Fig 6B), the majority of the replicates of ABC-LASSO converge towards a posterior distribution, which is adjusted and centered approximately on the target value. This is also true for the BD model (S8 Fig). We find similar posterior distributions for the likelihood-based approach except for the N parameter, where the posterior clearly reveals a lack of convergence.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Prior and posterior distributions for parameter estimations by ABC-FFNN, ABC-LASSO and BEAST2-BDSIR.

Prior distributions are in gray, posterior distributions obtained by ABC-FFNN are in orange, those by ABC-LASSO are in red and those by BEAST2-BDSIR are in black. All summary statistics were used for both regression-ABC approaches. We displayed the results for one particular epidemiological scenario (R₀ = 10, d_I = 30 and p = 0.5) and for large trees. There are 100 replicates in this scenario. The dots represent the median of the posterior distribution merging for all replicates. The vertical black line represents the true value for each epidemiological parameter.

https://doi.org/10.1371/journal.pcbi.1005416.g006

Results for the SI-DR model

We only ran ABC-LASSO using the sumstats and coords sets of summary statistics together, and set P_δ to 0.5. As shown in Table 6, for non-ultrametric target trees simulated with c₁ = 2, ABC-LASSO infers c₁ very accurately (MRE = 0.065). Inferring β with this method is slightly more difficult (MRE = 0.24), but the target value of β always falls into the 95% Highest Posterior Density (accuracy_95% = 100). Unfortunately, we fail to infer γ and N. However both parameters are easier to infer when c₁ = 2 than when c₁ = 0.5. As shown in Table 6, with ABC-LASSO, all four parameters of the SI-DR model, especially N and γ, are better inferred from large trees ( whereas ). We also observe an effect of the ultrametric nature or not of the target trees. Unlike other parameters, the inference error on β is lower with non-ultrametric trees than with ultrametric trees. Despite these contrasted results, ABC-LASSO outperforms the kernel-ABC method from [40] for all parameters. This is not affected by increasing the length of the MCMC chain to 50,000 steps for kernel-ABC.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. Performance of the ABC-LASSO and the kernel-ABC methods on non-ultrametric trees (c₁ = 2).

Mean Relative Error (MRE) and 95% HPD accuracy (accuracy_95%) of inference of the SI-DR epidemiological parameters by both ABC-LASSO and kernel-ABC approaches. For the ABC-LASSO method, we show the results obtained on the 100 large target trees (1,000 leaves) enclosed in brackets. For the kernel-ABC method, we show the results obtained after extending the MCMC chain length to 50,000 steps for 10 target trees enclosed in square brackets.

https://doi.org/10.1371/journal.pcbi.1005416.t006

We ran additional analyses to compare the kernel distance with the our summary statistics using a simple rejection (S4 Text). Results indicated that the kernel distance is less correlated to the inference task than the Euclidean distance computed from all of our summary statistics together (S9 Fig).

Ebola phylodynamics

We analyzed the correlation between the epidemiological parameters of the BDEI model and the summary statistics or coordinates of the LTT plot for trees of 72 leaves (S4 Fig). As previously observed for the SIR model, we see that the summary statistics computed on the Lineage-Through-Time plot (ltt set) and those computed on the branch lengths (bl set) are the most correlated to the epidemiological parameters. Conversely, the topological indexes (topo set) contain very little information about the parameters. The bl summary statistics are correlated positively to both the duration of infectiousness d_I and the duration of latency d_E, except the ie_BL_median_[1] statistics, which is correlated negatively to d_E and correlated positively to d_I (S13 Table). The coordinates of the LTT plot (coords set) are correlated poorly to d_E (S14 Table).

As for any data analysis, it is important to assess the fitness of the summary statistics to infer the epidemiological parameters from the “target” phylogeny. We did this for the sumstats and coords sets together and separately. The goodness-of-fit test revealed that the coords set of summary statistics was not fit to infer the epidemiological parameters of the Ebola phylogeny (p-value < 0.05). Therefore we only used the sumstats set of summary statistics.

Fig 7 shows that the median of the posterior distribution of R₀, inferred by Stadler et al. using BEAST2-BDEI, is close to the median of their prior distribution (in gray). The duration of latency seems very difficult to infer using the BEAST2 approach, as d_E HPD 95% is almost as large as that of the prior.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Prior and posterior distributions of parameter estimations from the Ebola phylogeny.

We show the results for two different inference methods: ABC-LASSO (in red) and BEAST2-BDEI (in black). Gray distributions correspond to the prior and red distributions correspond to ABC-LASSO posterior distributions. The dots represent the median and the vertical lines represent the 95% highest posterior density of each distribution. Statistics on the BEAST2-BDEI posterior distributions were obtained from [8].

https://doi.org/10.1371/journal.pcbi.1005416.g007

Our parameter estimates differ slightly from those of Stadler et al. We find a longer incubation period (11.7 [HPD_95%: 6.77–17.74]) and a longer duration of infectiousness (4.5 [HPD_95%: 1.41–10.79]) than Stadler et al (4.92 [HPD_95%: 2.11–23.20] and 2.58 [HPD_95%: 1.24–6.98] respectively). Both of these are more in line with the estimations from the WHO Ebola Response Team [65], which found that the fitted incubation period was 9.9 ± 5.6 days and the mean duration of infectiousness in the community was about 4.6 ± 5.1 days. We also infer a greater value for R₀ than Stadler et al (5.92 [HPD_95%: 2.97–11.12] instead of 2.18 [HPD_95%: 1.24–3.55]), which is probably driven by the longer duration of latency. Indeed, even if the duration of latency does not appear in the deterministic formulation of R₀ for the BDEI model, it may have an effect in the stochastic setting. Put differently, we have more infected individuals in our simulations, but a high proportion of these individuals are still latent and do not propagate the disease. Our R₀ estimation is more in line with [9], which used the same dataset but fixed the duration of latency, and found that R₀ = 2.40 [HPD_95%: 1.54–3.87] if d_E = 5.3 days and R₀ = 3.81 [HPD_95%: 2.47–6.3] if d_E = 12.7 days.

As the phylogeny from [60] that we used in this study is poorly supported (average bootstrap support = 0.23), we performed a supplementary analysis to assess the robustness of our method in the presence of phylogenetic uncertainty (S5 Text). We used 10 additional trees with nearly optimal likelihood scores, and showed that, despite the presence of substantial topological differences (average normalized RF distance among trees equal to 0.23 [66]), the posterior distributions inferred by ABC-LASSO are very similar (S10 Fig).

Summary statistics

For the BD and the SIR models, we found that the shape of the phylogeny contained less information about the epidemiological parameters than the LTT plot and the branch lengths. We also did not find any strong correlation between topological statistics and epidemiological parameters for the SI-DR model, which captures host structure and therefore could be expected to make these statistics more relevant [39, 40, 67, 68]. However, we found the lineage component (y-axis) of the LTT plot, which is related to the topology, to be more correlated to the epidemiological parameters in the SI-DR model than in all the other models we studied. Our current set of summary statistics seems to be sufficient to infer the epidemiological parameters of the BD and the SIR models, but not those of the SI-DR model. In fact, our results on this model show that our summary statistics are quite poorly correlated to the two epidemiological parameters that we have difficulties to infer (infection duration and population size). This suggests that there is no universal set of summary statistics and that there is room for additional ones, to be used to analyze the SI-DR model and likely other complex models.

Regression-ABC

Summary statistics are sometimes viewed as the Achilles’ heel of ABC, because “summarizing” suggests a loss of information. Furthermore, complex objects such as phylogenies can contain information unrelated to epidemiological parameters, which may dilute the desired signal. Selecting the “relevant” summary statistics could improve the method’s accuracy, but this is notoriously difficult [39, 69–71]. Here, we show that current machine learning techniques are efficient at performing variable selection on a large number of summary statistics.

One potential limitation of the rejection approach we used is that it relies on the simple Euclidean distance between unweighted summary statistics. One option could be to use adaptive methods of distance weighting, but these are time consuming and tend to be replaced by machine learning techniques.

The comparison between the LASSO and FFNN regression methods revealed that ABC-LASSO was more robust to the choice of summary statistics than ABC-FFNN. This was likely due to the R packages we used, and we expect that re-implementating an FFNN model with regularization tuning would remove this difference. The non-linearity of FFNN could then become an advantage. In theory, an advantage of LASSO compared to FFNN is that it provides us with an output on the selected summary statistics. However, we were unable to identify sets of summary statistics that were always selected or always discarded. This is likely due to the high degree of correlation between our summary statistics. A random forest approach could possibly circumvent these difficulties [72].

Method comparison

We compared regression-ABC methods to the kernel-ABC method [40] and to likelihood-based approaches based on birth-death-sampling (BDS) processes [21, 22, 42]. Our choice was motivated by the fact that the former relies on ABC and that the latter is widely used thanks to BEAST2. Another powerful method, which is also likelihood-based, involves coalescent processes [19, 23]. We did not use this method for parameter inference because, to the best of our knowledge, it is currently only implemented in R and we anticipated issues with computing time. However, we did use the tree simulator (rcolgem) associated with this method for comparison with kernel-ABC.

In short, when comparing our ABC-LASSO method to the BDS methods, we obtained comparable (but slightly lower) accuracy when estimating R₀ and infection duration. We also found that the accuracy of our ABC method always increases with phylogeny size. When assuming an SIR model, we obtained more accurate estimates of host population size than the BEAST2-BDSIR approach. The SI-DR epidemiological model is the model where the accuracy of the estimates using ABC-LASSO was globally the most disappointing (even though it was still better than with kernel-ABC). This could be due to the fact that we made several assumptions in order to compare our results to [40]. For instance, the tree size was relatively small (300 leaves) and our results showed that accuracy is better on larger trees (1000 leaves). It could also be that the target values chosen for the contact rates of the two host classes were too close (c₁ = 0.5 or 2, and c₂ = 1) to be well differentiated. The SI-DR model is a complex epidemiological model with many parameters and the four chosen by [40] are perhaps not all identifiable, at least when using our current set of summary statistics. It may be that developing additional summary statistics or using larger training sets to learn the regression model could improve the approach’s accuracy.

When comparing methods, we saw that posterior distributions can be much more valuable than statistics such as the relative error. Indeed, if the prior distribution is centered approximately on the targeted value, without any selection on parameter values the posterior will not deviate from it. This is illustrated, for instance, by the population size parameter in the SIR model, where some models have reasonable relative error even though the posterior is often identical to the prior (Fig 6).

Our results are consistent with those reported recently by the PANGEA-HIV consortium [28]. One aspect that deserves more investigation is related to computing time. Indeed, the most successful method in PANGEA-HIV required “considerable resources” in terms of CPU. The most time-consuming part in our ABC-LASSO is the simulation and the computation of the training set summary statistics. Rejection in itself is very fast, and LASSO is a fast machine-learning technique even if it is combined with cross-validation to avoid over-fitting. The computational complexity of simulation is generally linear with respect to the number of samples and the number of time-steps (or events) considered during the simulation. Moreover, the approach’s complexity also depends on the number and type of summary statistics. We chose to use a large number of summary statistics, but each of these is computed quickly in time at most linear in the tree size. Furthermore, the simulations and computation of summary statistics can both be run easily in parallel. In the likelihood-based methods we used, computing time depends on calculation of the likelihood function (which can be easy for the simple BD model and most coalescent models, but can be complicated due to the necessity to integrate over time for some others [22]) and on the convergence towards a posterior distribution (which is generally led by an MCMC search). Lastly, for the kernel-ABC approach, the computational complexity depends on that of the simulation procedure, the functional distance (which is much longer to compute than our simple Euclidean distance) and the MCMC search (which depends on the length of the MCMC chain and on the number of epidemiological parameters). This list suggests that regression-ABC may become advantageous when the number of training trees to learn the regression model becomes smaller than the length of the MCMC chain required to obtain convergence. Further investigation is warranted on this topic since both of these mehtods depend on the number of parameters to estimate, the size of the phylogenies, and also the relevance and information content of summary statistics.

Perspectives

Our goal was to compare existing methods to determine whether regression-ABC can be an alternative to MCMC-based methods. We showed that this approach can reach an accuracy comparable to state-of-the-art techniques, which allows us to envisage several paths for future studies.

A direct extension of our approach could be to investigate more complex models, since the major requirement of our approach is to be able to rapidly simulate data assuming such models. Additional efforts will likely be needed to design new relevant summary statistics.

Another possibility would be to modify the method in order to take into account surveillance data [73] or to directly analyze sequence data. This latter modification would be valuable when the inference of a time-scaled phylogeny is difficult or impossible [12]. We could also include natural selection in the model to allow pathogen strains to spread at different speeds.

On the technical side, a promising extension would be to explore random forest algorithms, which are powerful tools for clustering and non-linear regression with high explanatory power [72]. These algorithms have already led to promising results in the ABC framework [74]. Lastly, we focused here on phylogenies of epidemics but this method could be extended to infer parameters from phylogenies generated using ecological or evolutionary models [75, 76].