Related literature

Previous works have modelled the impact of NPIS using extensions of the classical Susceptible-Infected-Removed (SIR) model [9] with the inclusion of compartments corresponding to asymptomatic population which are either exposed to infections but not yet infectious, or infected and infectious [10]. Given the current knowledge of the COVID-19 disease, this constitutes a complete description of the possible states.

Among the works based on the aforementioned state space representation, [5], one of the main inspirations behind our work, uses an age-structured model to quantify the effect of control measures imposed in Wuhan, China and concludes that there exists a large potential on the use of NPIS for mitigating the COVID-19 pandemic. Furthermore, the authors recommend a gradual relaxation strategy in comparison to an early lifting of the imposed lockdown measures to avoid possible second and third waves of the pandemic. Using a stochastic modification of a compartmental model without age structure, [2] reaches similar conclusions and quantifies the effectiveness of lockdown measures by estimating the reproduction number, which decreased from a median value of 2.35 before travel restrictions were imposed, to 1.05 one week after the implementation of travel restrictions.

In the context of resorting to optimal control methods to determine a lockdown policy, [11] compares a switching on-off strategy with a two-stage release from quarantine (with part of the population released first, and the others later). The authors consider a threshold-based sanitary cost functional aiming at releasing the largest possible population without exceeding the availability of hospital beds, and their conclusion is to favour the second strategy. [12] consider multiple control levers, such as the number of tests (both virologic and anti-body tests) and the increase of ICU beds in addition to the reduction of social contacts, and uses a cost functional involving both economic and sanitary costs. The authors suggest an optimal lockdown policy which involves a quick and strong isolation, followed by a large increase in the number of tests.

Our contribution

While the qualitative modelling and control concepts of the aforementioned works are aligned with the epidemiological literature, we remark that most of these papers use parameter values collected from previous works or estimated using either different models or from clinical knowledge. Hence, they cannot be directly applied to populations with different spatio-temporal patterns. In contrast, the main goal of this paper is to propose a framework which calibrates the model using epidemiological and real-time mobility data from a specific population, measured by Google Mobility through Android devices, and computes an optimal lockdown strategy for that population at any point of the pandemic. To achieve this, we combine parameter estimation for an epidemiological model with a subsequent optimal control step. Secondly, our framework includes the computation of the optimal lockdown policy in a nonlinear model predictive control framework, leading to a robust feedback protocol which allows not only real-time adaptation of the release strategy but also a partial lockdown, as opposed to a switching on/off strategy which can be too restrictive.

Methodological summary

Our epidemiological model simulates the transmission dynamics of COVID-19 spread in the English and French populations using anonymised data on the reduction of the population mobility collected through smartphones and released by Google. Further, exploiting Approximate Bayesian Computation (ABC) [13], we calibrate this model using data on daily deaths and the number of people in hospitals with COVID-19 released from public health authorities (Public Health England (PHE) and the National Health Service (NHS) in England, and Santé publique France (SpF) in France) [14–17]. In ABC, we assume a prior distribution for each parameter value and, given a dataset with an inherent observation noise, we obtain a non-parametric estimate of the joint probability distribution of the parameter values. This allows us to estimate parameters of our model for the specific cases of England and France, as opposed to inheriting parameters from epidemiological models from other countries. Some key attributes of our model are the inclusion of age-dependent transition probabilities between the different compartments which are also estimated from data, as well as age-dependent social distancing, and the use of Google mobility data to quantify the effect of social distancing measures in reality. Having calibrated our model, we design a lockdown strategy which is differentiated according to social contact categories including schools, work, and others. This allows us to assign a different economic penalty for each one of them. Borrowing a leaf from optimal control theory, we synthesize an optimised lockdown and exit strategy which minimizes the number of COVID-19-related casualties in the population, but also takes into account economic constraints. In order to perform this task, we quantify the relation between the decrease in social contacts with the reduction in the population mobility and optimise with respect to the latter, which is an effectively measurable quantity compared to an abstract decrease in social contacts. A methodological summary is depicted in Fig 1, illustrating the interaction among the different building blocks of our approach. It is important to note that the proposed methodology transcends the design of lockdown strategies for the COVID-19 pandemic, and can be applied for more general epidemiological models, different datasets, and a variety of control objectives. What is fundamental in our approach is the existence of a dynamical model, the assimilation of data for the optimal estimation of model parameters and uncertainties, and the optimization of an external input action to control the system towards a desired state.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Flow diagram of the data-driven approach for the synthesis of optimal lockdown policies.

The initial step consists of a policy maker defining a performance measure based on sanitary and economic objectives, and a modeller selecting a consistent generic epidemiological model. Then, public healthcare/mobility data is used in conjunction with Approximate Bayesian Computation to calibrate the dynamical model and determine the degree of uncertainty in the model parameters. This assists the formulation of an optimal control problem where the original sanitary + economic performance measure is optimized constrained to the calibrated epidemiological model. An optimal lockdown policy is then computed via global optimization techniques, and the final output is an optimal lockdown policy. The optimised lockdown is then applied and its real-time effects can be sensed through public data, which fed back into the learning and optimization framework for re-computation and update.

https://doi.org/10.1371/journal.pcbi.1009236.g001

Model dynamics

Assuming a well-mixed population (namely, each person has the same probability of interacting with any other person in the population), we consider a compartmental model [20], splitting the population in different compartments representing different states of the infection. Although this is extremely simplifying, compartmental models under this assumption are widely used to describe the dynamics of epidemics over large populations. Our model considers the following compartments:

• Susceptible (S), meaning people who did not have any contact with the infection,
• Exposed (E) to the infection, but not yet infectious,
• Infected SubClinical (I^SC, split in I^SC1 and I^SC2), not needing medical attention,
• Infected Clinical (I^C, split in I^C1 and I^C2), needing medical attention,
• Recovered (R), which we assume are resistant to a new infection,
• Deceased (D).

As strong evidence towards the age-dependent severity of COVID-19 has been observed in previous research works [1, 7], we consider age-stratification of all of the states along 5 age groups: 0–19, 20–39, 40–59, 60–79, 80+, hence we will use the notation E_i to denote the Exposed population in the i-th age group, and similarly for the other states. The model will assume that all the age groups are susceptible to the infection in the same way, but that the severity is strongly dependent on the age of the patient through age-dependent probabilities of necessity of hospitalization (ρ_i) and death if hospitalized () for the i-the age group.

Another key assumption of our model is that when a patient is hospitalized and diagnosed, they are isolated and therefore not able to spread the infection. To reflect this scenario, we assume that from the exposed state and after some incubation period, all patients will become sub-clinical I^SC, in which state they are infectious. Afterwards, some of them will recover (R) and others will need clinical help (I^C); we model this by splitting I^SC into two categories: the ones recovering straightaway (I^SC2) and the ones in need of clinical care (I^SC1). The split happens with an age-dependent probability ρ_i. After some time, people in I^SC1 will go to hospital, therefore moving to the I^C state; similarly as before, the latter state is split in two categories according to the final outcome: the ones in I^C1 will decease (D) after some time, while the ones in I^SC2 will recover (R). This split is again described by an age-dependent probability, which we denote as . A visualization of the dynamics is given in Fig 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Graphical representation of the model, for each age group.

The green color represents a compartment that is observed independently for each age group, while blue represents a compartment whose sum across age groups is observed.

https://doi.org/10.1371/journal.pcbi.1009236.g002

Mathematically, this can be described using the following system of ordinary differential equations (ODEs): (1) (2) (3) (4) (5) (6) (7) (8) where and C is the contact matrix representing the frequency of contacts between different age groups [18], where each element C_ij represents the average daily number of contacts a person in age group i has with people in age group j. To simulate from the model, the ODEs (Eqs (1)–(8)) are integrated using a 4th-order Runge Kutta integrator, with a timestep dt = 0.1 days; the dynamics is started on the 1st of March.

Influence of mobility on the dynamics.

The contact matrices are a crucial component defining our model dynamics, and were estimated in the POLYMOD study [19] for a large set of countries (among which England and France). Specifically for the contact matrices for England, the findings of the more recent BBC Pandemic project [18] were integrated using the procedure described in [18]. Note that [18] provided contact matrices for the whole UK, We assume that the contact matrix for England to be the same as that for the whole of the UK. Finally, the age groups considered in these studies (namely, 5 year bands) are finer than the ones we consider in the present work; we therefore aggregate the data to make contact matrices suit our needs. The (i, j)–th entry of this contact matrices at different locations (eg. home, workplace, school and other locations) represent the amount of daily contacts an individual in age group i has with individuals from age group j in different settings (see Fig 3). Before the lockdown, the total contact matrix is simply the sum of the contributions of these different locations:

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Contact matrices at different locations in the UK for the age groups used in the present study (0–19, 20–39, 40–59, 60–79, 80+); these are obtained by aggregating and combining the contact matrices for 5-year bands provided by [18] and [19].

https://doi.org/10.1371/journal.pcbi.1009236.g003

However, the introduction of lockdown measures lead to considerable change to people’s social activity and mobility; we model this by introducing a set of multipliers (for each age group and for each of the locations) which will represent the change in the number of social contacts: (9) where , represents the change of social contacts for age group i in the locations school, other, andwork. These multipliers are a function of time and not easily accessible. Instead, it is rather easy to measure the reduction of people’s mobility towards the different locations; we choose then to express the α’s as a function of the mobility values provided by Google, as explained in the next paragraph.

Mobility data is collected by Google to reflect the reduction of the population mobility during lockdown for each country, by following the movements of Android phones; anonymised data is publicly available [21]. Similar datasets can be retrieved directly from other sources, such as mobile phone companies [22]. This dataset captures mobility towards the following locations: “residential”, “workplaces”, “parks”, “retail and recreations”, “transit stations” and “grocery and pharmacy”, which we denote respectively as m^residential, m^work, m^parks, m^retail, m^transit, m^grocery; the changes in mobility are reported with respect to the baseline values prior to introduction of lockdown measures. As can be seen in the left panel of Fig 4, a strong weekly periodicity is present in this data; we therefore use a Savitzky–Golay filter [23] to remove it.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Raw and elaborated mobility data in the UK.

In the raw mobility data, which is scaled with respect to a baseline value representing average mobility in the months prior to the pandemics, a strong weekly seasonality is present, which we mostly removed using a Savitzky–Golay filter. Moreover, we note that the mobility towards “residential” (which is not used in our analysis) locations is larger during the lockdown months than before, as people spend more time in their homes. Finally, we note the very large increase in people’s mobility towards parks around the end of the winter season. That contribution is however only one of the components in our aggregated m^other mobility value, so the latter does not increase that abruptly.

https://doi.org/10.1371/journal.pcbi.1009236.g004

We moreover combined the mobility values m^parks, m^retail, m^transit, m^grocery in order to obtain an aggregated value for the reduction of mobility towards “other locations”: m^other = 0.1 ⋅ m^parks + 0.3 ⋅ m^retail + 0.3 ⋅ m^transit + 0.3 ⋅ m^grocery. Even though the numerical values of the weights are arbitrary, this choice is motivated by our observation that the value of the different contributions of the mobility data is quite similar; we also attribute a smaller value to “parks” as people get less in strict contact with each other there with respect to “retail”, “transit” or “grocery” locations.

As no data with regards to schools were provided, we fixed the value of m^school to be 0.1 from the day schools and universities were closed except for children of essential workers (23rd of March [24] for England, 16th March for France [25]). The mobility data obtained after the aggregation and smoothing operations described above is presented in the right panel of Fig 4.

In order to connect the reduction in mobility to the reduction in the number of contacts, we proceed in the following way: first we assume that the number of residential contacts stays constant; in fact, we expect the behavior of people at home not to differ too much with respect to what it was prior to the introduction of lockdown measures; for our model, this amounts to fixing . With regards to the remaining contributions to the total number of contacts, we expect data on mobility reduction to be representative of the subset of the population which mostly uses smartphones, which is likely to be younger than 60 years old. Moreover, a large part of the population older than 60 years old does not take part in work or school activities. Motivated by these arguments, we define the values of and separately for population below and above 60. Specifically, we assume the following for age groups below 60 years old (age group index 1,2,3): (10) where we made the dependence on time explicit in order to highlight that α₁₂₃ ∈ [0, 1] is a time-independent scalar, which is an additional parameter in our model; this amounts to assuming that the reduction in the number of contacts is due to a combination of reduced mobility and increased awareness of people, for instance by maintaining social distancing. For people above 60 years old (age group index 4 and 5) we assume instead that the reduction in contacts stays constant since the introduction of lockdown measures and that such reduction is equally distributed across the different categories (as the contacts for work and school will be relatively few): (11) where α₄, α₅ ∈ [0, 1] are time-independent scalars, which are parameters in our model as well. This latter assumption is motivated by the fact that such part of the population is more susceptible to the disease, so that the official advice will be for them to be as isolated as possible throughout the epidemics.

We initialise our implementation of the model dynamics on the 1st of March and we fix the contact matrix to be the standard one relative to the country until the introduction of lockdown measures (which we assume to be on the 18th March, i.e. two days after the UK government advised people to self-isolate [26] and one day after the French government banned all except essential journeys [27]); from that day onward, we use the contact matrix obtained from Eq (9), by fixing α^home = 1 and obtaining the values for the other α’s by Eqs (10) and (11).

Uncertainty quantification

We use approximate Bayesian computation (ABC) [30] to calibrate the parameters of our model, by using the datasets reporting on the number of hospitalized and deceased patients released by the public health authorities:

• The daily number of deaths in hospitals attributed to COVID-19 (per age group)
• The daily number of hospitalized people with COVID-19 related diseases

We calibrate our model on data from the 1st of March up to different ending times t_obs (eg. 31st of August). ABC is suitable for the considered task as it relies only on simulations from the model, and works by looking for values of parameters such that the integrated dynamics is close to the observed one. In this way, it provides the user with samples from an approximate posterior distribution of the parameters given observed data (π_ABC(θ|x_obs)). The better the match between the simulation and the observation, the better is the approximation of the true posterior π(θ|x_obs). However, this comes at higher computational cost, so that the level of approximation needs to be balanced with computational considerations. We discuss additional details regarding the use of ABC to calibrate our model in Section 2 of S1 Appendix. Moreover, ABC allows us to fix a prior uncertainty on the values of the parameters (defined by a prior distribution π(θ)), and to quantify prediction uncertainty through the approximate posterior π_ABC(θ|x_obs), which we will use in designing the optimal control task next.

We remark that the ABC algorithm which we employ (discussed in Section 2 of S1 Appendix) provides us with a set of samples from the posterior distribution associated with importance weights. We can obtain independent and identically distributed (i.i.d.) samples from it by using bootstrap; namely, we choose (with replacement) from the set of ABC samples with probability proportional to the importance weight itself. The new set of samples generated in this way can be considered as i.i.d. from the posterior distribution of the parameters given data.

Posterior loss based cost functional

In this step we identify the cost functional we will optimize to determine an optimal lockdown strategy. We consider a cost functional which combines the economic cost of lockdown with the sanitary cost of lifting restrictions. We focus on the representation of these terms and on the inclusion of the posterior distribution of the model parameters in its modelling. Determining the relevance of sanitary versus economic costs is a task left to the policy maker. However, when a quantitative choice has been made, our methodology allows to test the effect of such a choice and to evaluate the stability of the optimal strategy.

We denote by t₀ the start of the optimisation interval, corresponding to the day we wish to start the new lockdown policy (in our first example, the 24th of May), and by T_h the length of the interval in days, for which we want to apply the lockdown strategy, also known as optimisation horizon. To study the cost of lifting restrictions, we penalise the predicted number of deaths during the optimisation horizon [t₀, t₀ + T_h]. This is given by where ΔD_i(t) is the daily increase in the number of deceased in the age group i. Furthermore, we need to guarantee that the number of infected individuals who need hospitalisation, , remains below the overall hospitals capacity H_max. This could be included as a hard state constraint, here instead we penalise the event in which I^C surpasses the total capacity by including a term of the form We point out that this term will not be activated if the levels of infected people who need hospitalisation remain well below H_max, which here we take to be H_max = 10000. For example, the number of people in hospital in England with COVID-19 related symptoms at the end of our first training window (23rd of May) was 7106, while at the height of the peak of the first wave of the epidemic in England (on the 12th of April) this number was 17933. For France, numbers reach higher values; however, we keep the same value of H_max = 10000, in order to penalize a large number of infected people in the same way across the two countries, and to have comparable values in the optimization strategy. As a final measure of the sanitary cost, we introduce a final time cost, where we penalise the basic reproduction number () at the end of the optimisation horizon (details about number for our model can be found in Section 1 of S1 Appendix). This terminal penalty ensures that the control strategy does not simply output an optimal solution which switches off the reduction on mobility towards the end of the optimisation horizon. While such a solution is consistent with the optimal control design and is an interesting instance of the turnpike phenomenon [31], it is not suitable in our context. Finally, to account for the economic cost of lockdown, we penalise the mobility reduction by introducing a quadratic cost of the form ‖1 − m^loc(t)‖², for loc ∈ {school, other, work}.

As mentioned before, the forward model is run for different sets of parameters drawn from the posterior distribution obtained via ABC inference. For this reason, the ΔD_i(t) and variables appearing in the cost functional depend on the chosen value of the parameters of the model. We hide the explicit dependence for notational convenience. The cost is computed by taking an expectation over the posterior distribution for the parameters, since each parameter value will lead to a different realisation of the dynamics. Due to the nonlinearity of the system, this is clearly not the same as minimizing the objective using the posterior mean of the parameters only. In practice, we use 50 i.i.d. samples from the posterior distribution and approximate the expectation with an average over the trajectories obtained with those parameter values. Minimizing an expected cost in this way is much more computationally expensive than a cost computed on a point estimate of the parameters (for instance the posterior mean). However, it provides a way to take into account the uncertainty in the parameters while producing more robust results.

Collecting the different terms in our cost, we optimise (13) through the control signal where ϵ_loc, for loc ∈ {school, other, work}, represents the relative cost of limiting the mobility to schools, workplaces and other locations with respect to the sanitary cost, and where the expectation is taken over the posterior distribution of the parameters of the model. The choice of the values for ϵ_loc affects the optimal policy by attributing a larger economic or social cost of closing one of the categories with respect to the others. Determining adequate weights for these costs is the ultimate task of the policy maker.

Nonlinear model predictive control

To close our optimal control formulation we add specifications to the controls we expect to obtain, restricting the space of admissible signals. As the values of α are between 0 and 1, a reasonable assumption is to expect the m^loc to be in the interval [0, 1] as well, for loc ∈ {school, work, others}. However, due to the fact that we cannot impose a 100% closure of all settings, we set a lower bound for m^loc to be the lowest value of each m^loc observed during the lockdown period. This results in the constraints m^work ∈ [0.31, 1], m^school ∈ [0.1, 1], and m^others ∈ [0.41, 1].

As can be seen from Eq (13), the controls m^loc, loc ∈ {school, work, others} are time-dependent, and they are computed by minimising the cost functional (13) subject to the state constraints (1)–(8). Ideally, the numerical realization of the optimal control strategy would be driven by the calculation of first-order optimality conditions and a reduced gradient approach to minimise . However, the nonlinearities in the dynamics and in the terminal penalty, where the reproduction rate is expressed as an eigenvalue of a parameter-dependent matrix, make our problem highly non-convex. Moreover, the penalty Φ(I^C) is non-differentiable. For the purposes of this paper we will compute the optimal control by using generalized simulated annealing (or dual annealing) [32]. The use of meta-heuristics for the solution of large-scale nonlinear optimal control problems has been assessed in [33, 34]. We embed the solution of the optimal control problem (13) in a nonlinear model predictive control (NMPC) framework [35]. To this end, we select a prediction horizon T_opt, and optimise the control variables m^loc(t), t ∈ [t₀, t₀ + T_opt] using the current state at t₀ as our initial condition. From the optimal control sequence we recover the optimal action for a single day, that is m^loc(t), t ∈ [t₀, t₀ + 1] and evolve the dynamics for the same amount of time, and repeat the optimisation procedure in the updated time frame [t₀ + 1, t₀ + 1 + T_opt]. This process is then repeated until the complete optimisation interval [t₀, t₀ + T_h] is covered. The NMPC methodology recovers a robust optimal control law in feedback form that can be adjusted to account for disturbances in the control loop. Therefore, instead of using the current state predicted by the model as initial condition, we can update this to be the current state of the population in the considered country estimated from data, every time new data becomes available. This ensures that the control methodology accounts for noisy observations, or for unexpected variations in the data.

Proof of concept of the methodology

We now proceed to present the results of our methodology as described above on data for England observed until 23rd May. Each of the figures below is either of two types: 1) Lockdown strategy and its effect on the number of hospitalized individuals, and 2) Influence of the lockdown strategies on the reproduction number . In the first case, we plot the number of hospitalised people (full red line) for each control strategy—these results are associated with the red axis, on the left of the figure—and the value of the optimized mobility values m^loc for loc ∈ {school, work, others} in the three blue lines, associated with the blue values on the right axis. The full, dashed, and dash-dotted lines correspond to work, school and other settings, respectively.

Dependence on T_opt.

A crucial issue in the NMPC framework is the selection of the prediction horizon T_opt. For small prediction horizons, the optimal action tends to be instantaneous and loses its capability to foresee long-term consequences of the policy. On the other hand, a sufficiently long prediction horizon will enforce a stabilizing control law, but its numerical realization becomes increasingly complex. Therefore, at the core of the selection of a suitable prediction horizon there is a trade-off between short-sightedness, stabilization capabilities of the policy, and computability. This is exemplified in Fig 7, where we illustrate the role that is played by the prediction horizon in the performance of the control loop for a fixed value of ϵ_loc, for loc ∈ {school, work, others}. It can be observed that a short-sighted policy, with a prediction horizon of 20 days (left panel), is less stringent in the mobility reduction, causing a larger number of hospitalized people in the long term and a large uncertainty in the end result.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Dependence on the prediction horizon T_opt for determining and optimal control strategy for England.

Here, (ϵ_s, ϵ_w, ϵ_o) = (100, 100, 100) and the lockdown strategies were applied for T_h = 90 days starting on the 24th of May.

https://doi.org/10.1371/journal.pcbi.1009236.g007

However, it is clear that the lockdown strategies are similar for both prediction horizons of T_opt = 30 (middle panel) and T_opt = 40 (right panel): here, we observe a smaller overall number of hospitalized people over time, while the uncertainty on the results remains bounded, with slightly smaller credibility intervals in the second case. For this reason, we will from now on use the prediction horizon of T_opt = 30, which is long enough to avoid a short-sighted strategy, while being short enough to be computationally cheaper. We point out that this is correctly aligned with the COVID-19 time scale for transmission, is a reasonable time frame for a policy maker, and agrees with PHE definition of death due to COVID, where a death is considered COVID-related if it is within 28 days of a positive COVID test [36].

Influence of the relative weights of the control penalties (ϵ_school, ϵ_work, ϵ_other).

The choice of suitable control penalties ϵ_loc, loc ∈ {school, work, other} is a sensitive issue in any optimal control problem, and we now proceed to study its effect on the resulting lockdown strategies. In this context, there are two important properties to explore: the relative weight between the sanitary and economic costs of our lockdown strategy, which is represented by how large the values of each of ϵ_school, ϵ_work, ϵ_other are, and the relative importance between each of the ϵ_school, ϵ_work, ϵ_other. We observe that the predicted number of daily deceased at the end of the training interval is between 200 and 400 (see Fig 6). Having in mind that the control variables are constrained to [0, 1], we conclude that each ϵ_loc should be in the order of 100, so that the control strategies are sensitive to both the sanitary and economic costs.

In our next example, we assume that the economic cost of opening each type of location is the same (i.e. ϵ_school = ϵ_work = ϵ_other) and analyse the effect of varying their relative weight to that of the sanitary cost. This is shown in Fig 8, where we present the lockdown strategies and corresponding values of hospitalized people for ϵ_loc = 100 (left) and ϵ_loc = 200 (right), for loc ∈ {school, work, other}. We observe that higher values of ϵ_loc result in the strategies which open workplaces earlier, as keeping them closed is more expensive, but also result in higher uncertainty. Interestingly, however, we observe that both control strategies keep m^other constant at a value of approximately 0.41, which is the minimum value allowed for this parameter. This behaviour is consistently reproduced for any of the values ϵ_loc that we explored as we will see below; the only situation in which we did not observe this behaviour was by attributing an unreasonably high weight to the economic cost, in which case all the lockdown measures are lifted, leading to a large increase of the number of infected people and to a second wave of the epidemics. To better understand this phenomenon, we inspect the contact matrix relative to the “other locations” category (Fig 3). There, it can be observed that the setting in which there are more contacts between the part of the population in which our optimisation strategy acts, (under 60 year-olds), and the older population is the “other locations”. As the latter are the most vulnerable to the disease, the optimisation is limiting the number of contacts they have by reducing m^other.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. Different relative weights between the sanitary cost and economic cost of the lockdown measures with the lockdown strategies starting on the 24th of May and applied for T_h = 120 days, for England.

https://doi.org/10.1371/journal.pcbi.1009236.g008

The last property we investigate is the effect of varying the values of ϵ for each category, which can produce a more economically viable solution. In the cases we will explore, we observe that opening workplaces is clearly an important part of the economy, and this might not be achievable without opening schools as well. For this reason, we will attribute the highest economic costs to closing workplaces, followed by schools and then others. We point out that this is ultimately a choice of the policy maker, and in principle any combination of values for each ϵ_loc could be considered. We tested a variety of combinations of these cost weights, each providing us with a lockdown strategy. In this case, we compare the efficiency of each strategy in Figs 9 and 10. As can be seen in Fig 9, all of the strategies propose to open workplaces and schools earlier or later (or not at all, in the case of schools) depending on the relative weights between sanitary and economic costs, and, as before, keep other locations at its minimum value. We show an additional comparison in Fig 10, where we plot the reproduction number resulting from each strategy, compared with what it would be if the mobility values remained unchanged from their estimated values on the 29th of May. We note that the optimal strategies always keep the value of smaller than 1 for most of the optimization horizon, but the confidence intervals allow for values larger than 1 at the end of the optimization window.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 9. Different economic cost weights produce different opening strategies and predicted hospitalized people, with lockdown strategies starting on the 24th of May and applied for T_h = 120 days, for England.

https://doi.org/10.1371/journal.pcbi.1009236.g009

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 10. Evolution of corresponding to different opening strategies starting on the 24th of May, learned using different economic cost weights for different ϵ values, with 99% credibility intervals; these plots are referred to England.

https://doi.org/10.1371/journal.pcbi.1009236.g010

Dynamic update of the model and optimal control strategy

As new data becomes available, we can re-perform the model fit, and obtain a new posterior distribution on the parameters to use in order to find the optimal mobility values. An instance of this dynamic update can be seen in Fig 11, where we show the optimal mobility values and the corresponding basic reproduction number after the model parameters and the optimal solution are updated at four instances: the 11th of April, 26th of April, 11th of May and 23rd of May. Here we used (ϵ_school, ϵ_work, ϵ_other) = (150, 300, 3) as from the results in Figs 9 and 10 it can be seen that such a choice leads to a lockdown strategy that increases the mobility towards both workplaces and schools while keeping the value of .

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 11. Dynamically updated control strategy: We fit the model on data for England up to t_obs = 11th of April and determine the optimal mobility strategy up to the next observation point t_obs = 26th April.

Data until the latter is used to repeat the procedure, in order to update the optimal control strategy exploiting newly available information. Here (ϵ_school, ϵ_work, ϵ_other) = (150, 300, 3). In panel (a), we show the resulting optimal mobility values, with the corresponding values of and their credibility intervals shown in panel (b).

https://doi.org/10.1371/journal.pcbi.1009236.g011

We observe that the lockdown strategy determined with parameter values fitted on data up to the 11th of April is extremely restrictive, as the predicted dynamics on that date badly overestimates the number of deceased and hospitalized people (see S2 Appendix). However, we see that recalibrating the parameters and updating the lockdown strategy using newly available data is beneficial: using data until the 26th of April results in a better prediction of hospitalized and deceased numbers, and we see an increase of the mobility towards schools already at the end of April. We repeat the procedure on the 11th and 23rd of May, resulting on a proposed strategy allowing schools to be open even further at the first instance, and then remaining almost unchanged, with an increased mobility towards workplaces around mid-June. We also note that, with the exception of the time interval between the 11th and the 26th of April—where the high value of is due to the fact that for the current values of the parameters the epidemic is predicted to increase rapidly, independently of the lockdown measures—the predicted value always stays below 1. This dynamic update of the model allows us to rely less on long-term predictions from the model, which are more computationally expensive and become more biased the farther in the future the prediction is. Together with the knowledge on model-specific biases coming from fitting the model to different horizons in the past, this approach enables a policy maker to assess the validity of the proposed optimal mobility strategy and update it on a periodic basis as new data becomes available.

Optimal strategy for England and France on 1st of September

After exploring all the different properties of our optimal control strategy, we proceed to test it in two populations, England and France, in a more recent setting, which is closer to a second wave of the epidemics. We fit the model again with data from the PHE and the NHS for England, and from SpF for France, and using the Google mobility data in both cases. The inference was performed on both cases for data up to the 31st of August and the results are presented in Fig 12 below.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 12. Comparison of predictions of our model with the real number of hospitalized people with COVID-19 and total daily deaths (green) on 31st August in England and France.

The solid red line denotes the median prediction, filled spaces denote the 99% credible interval and the vertical dashed line denotes the observation horizon. The different rows represent different observation horizons, while the columns represent number of people in hospital (I^C compartment, left column), daily deceased (middle column) and value of (right column).

https://doi.org/10.1371/journal.pcbi.1009236.g012

Optimal lockdown strategy for England starting in September 2020.

We apply our optimal control strategy again for the population of England, for a more recent time interval. As mentioned above, the parameters were fitted up to the 31st of August, and we apply a lockdown strategy for T_h = 120 days, starting on the 1st of September. The prediction horizon was kept at T_opt = 30 days and we used two possible sets of values for ϵ_loc. The corresponding results are shown in Fig 13, where the first two panels show the number of hospitalized people and corresponding credibility intervals (full red line and dashed area) and the mobility values (blue lines), while the corresponding values of (and credibility intervals) are presented in the right panel.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 13. Optimal strategy for England with parameters fitted with data up to the 31st August.

The lockdown strategy is applied for 120 days starting on the 1st of September and uses a prediction horizon of T_opt = 30 days.

https://doi.org/10.1371/journal.pcbi.1009236.g013

Optimal lockdown strategy for France starting in September 2020.

Our methodology is valid in general, and in particular it can be applied to different datasets. Our last example applies the strategy to the French population, where the epidemics has exhibited a different evolution than in the UK. As in the previous paragraph, we fitted the parameters with data up to the 31st of August, and apply the lockdown strategy for T_h = 120 days, starting on the 1st of September. The prediction horizon was kept at T_opt = 30 days and we used two possible sets of values for ϵ_loc. The corresponding results are shown in Fig 14, and are organised in a similar manner to Fig 13. We observe that in this case the relative values of ϵ_loc have a much stronger influence, with the lockdown strategy shown in the left panel having extremely better results than that on the middle panel. This is visible both on the number of hospitalized people and the resulting reproduction number, and illustrates the importance of the choice of these parameters and associated discussion with policy makers.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 14. Optimal strategy for France with parameters fitted with data up to the 31st August.

The lockdown strategy is applied for 120 days starting on the 1st of September and uses a prediction horizon of T_opt = 30 days.

https://doi.org/10.1371/journal.pcbi.1009236.g014

We can see a significant difference in the data reported in these two countries (green lines in Fig 12) which leads to very different parameter estimates of the epidemic model (see S2 Appendix). This is reflected in the optimal control we inferred for these two countries, highlighting the adaptability of our methodology for different countries and different scenarios. We reiterate that our methodology for both England and France recommends to keep mobility of “others” as low as possible if not at the lowest possible label of full lockdown. Remembering that the others category refers to “retail”, “transit” or “grocery” locations, our methodology is recommending that the we should keep a stringent lockdown until December, while opening up the work and school places in a socially-distanced manner. Deviations from that will likely result in a second-wave of infection spreading.