Abstract
We consider here an extended SIR model, including several features of the recent COVID–19 outbreak: in particular the infected and recovered individuals can either be detected (+) or undetected (–) and we also integrate an intensive care unit (ICU) capacity. Our model enables a tractable quantitative analysis of the optimal policy for the control of the epidemic dynamics using both lockdown and detection intervention levers. With parametric specification based on literature on COVID–19, we investigate the sensitivities of various quantities on the optimal strategies, taking into account the subtle trade-off between the sanitary and the socio-economic cost of the pandemic, together with the limited capacity level of ICU. We identify the optimal lockdown policy as an intervention structured in 4 successive phases: First a quick and strong lockdown intervention to stop the exponential growth of the contagion; second a short transition phase to reduce the prevalence of the virus; third a long period with full ICU capacity and stable virus prevalence; finally a return to normal social interactions with disappearance of the virus. The optimal scenario hereby avoids the second wave of infection, provided the lockdown is released sufficiently slowly. We also provide optimal intervention measures with increasing ICU capacity, as well as optimization over the effort on detection of infectious and immune individuals. Whenever massive resources are introduced to detect infected individuals, the pressure on social distancing can be released, whereas the impact of detection of immune individuals reveals to be more moderate.
1 Introduction
Within a few weeks, the COVID-19 took most of the world by surprise, starting with Asia, then Europe and finally America (see World Health Organization (2020); Kucharski et al. (2020); Magal and Webb (2020)). More specifically, the COVID-19 turned out to be much more contagious than expected, and official health authorities understood that letting the virus spread freely would probably saturate hospital capacities. In most countries, aggressive quarantine and lockdown measures have been taken, hopefully to stop the spread of the virus and infection, or at least to slow down its diffusion and delay the cases over time to avoid saturation of healthcare systems. The effectiveness of these non-pharmaceutical interventions have been questioned in Hellewell et al. (2020); Kucharski et al. (2020). It was already suggested in Qualls et al. (2017), but then emphasized by Ferguson et al. (2020), that these measures should rather aim at ‘flattening the curve’: mitigation seems more realistic than suppression, since eliminating the threat necessarily takes time, and a massive arrival of people in need in hospitals would be dramatic.
The first control used by health authorities was based on quarantines, historically a natural technique to lower social interactions and reduce the spread of the disease, as recalled in Tognotti (2013). Classically, infected people are isolated, to avoid further transmission, but COVID-19 has a potentially high rate of asymptomatic infected individuals, as mentioned in Al-Tawfiq (2020); Day (2020); He et al. (2020); Nishiura et al. (2020) or Zhou et al. (2020). This explains why most countries used indiscriminate lockdown. In order to distinguish between the susceptible, but still unaffected, individuals and the either asymptomatic infected or undetected recovered ones, testing is necessary. Piloting these interventions is a key problem, in particular for monitoring exit strategies.
In the recent literature on the COVID-19, many papers have focused on the effect of lockdown on the spread of the disease (e.g. Berger et al. (2020); Roques et al. (2020)) or on the evaluation via simulations of exit-strategies (e.g. Domenico et al. (2020); Evgeniou et al. (2020)). In this paper, we suggest a different approach for evaluating lockdown and exit scenarios based on Optimal Control theory (e.g. Pontryagin et al. (1964)). Although the analysis of public health policies using controls is of clear practical value, there has not been a so dense literature on the subject (e.g. Abakuks (1973, 1974); Behncke (2000); Greenhalg (1988); Hansen and Day (2011); Agusto and Adekunle (2014); Sharomi and Malik (2017)). We consider a modified SIR model, incorporating all the features related to the recent COVID-19 outbreak mentioned above, and use optimization algorithms to compute the ‘best’ level of lockdown and detection effort in testing individuals. The criterion to rank the different possible strategies takes into account: 1) the death toll of the epidemic, 2) the economic and social costs of the lockdown, 3) the required effort for detecting infected or immune individuals. Besides, we only consider strategies never exceeding the capacity level Umax of the ICU sanitary system, chosen to be dedicated to COVID-19 patients. One of the originality of the model considered here is the particular focus on the outcomes of the intensive care units (ICU) saturation, which in many countries have been put under great pressure and whose capacities are determining factors for the death toll of the disease. We assume a capacity constraints on how many individuals can be treated in ICU at a given time. Once the threshold is reached, the fatality rate for the additional patients sent to ICU rises sharply. For the COVID-19 outbreak, we are aware of some works by Alvarez et al. (2020); Djidjou-Demasse et al. (2020); Piguillem and Shi (2020), but none of these work models the ICU capacities. In Acemoglu et al. (2020), no ICU capacity is explicitly introduced, but these authors choose a death rate in hospitals that depends linearly on the number of patients. The non-linearity induced by the threshold that we introduce provides very interesting phenomena and insights for the control of the disease. Furthermore, we investigate cases where the ICU capacity is increasing and evolving dynamically over time.
A second feature in our work is that we consider two levers for control: the quarantine ratio and the detection effort for infectious and immune individuals. One of our motivation is to study qualitatively the balance existing between efficient testing and strong confinement strategies. Actually, two testing procedures can be performed here. A short range one, that can detect, at time t if someone is infected, or not (Polymerase chain reaction or PCR tests, that can detect whether an individual is infected and send her/him in quarantine). And a long range one, that can detect if someone is immune or not (a blood test as in Huang et al. (2020)). For this purpose, we introduced a so-called SIDUHR+/− propagation model for the epidemic, which explicitly allows to take into account the detection ratios of immune and infected individuals.
Our main findings consist in identifying four main phases in the optimally controlled epidemic dynamics. First, a strong and quick lockdown intervention allows to stop the exponential growth of the epidemic spreading and bring the effective reproduction number ℜt under level 1. Then begins a short transition period during which ℜt remains below 1 as the prevalence of the virus within the population diminishes progressively until reaching a threshold I∗. Since the beginning of the epidemic, the number of individuals requiring ICU is rising and just reaches Umax when the virus prevalence attains I∗. Then begins a very long period during which the reproduction number ℜt stabilizes at level 1 while the virus prevalence and the ICU occupation remain stable at levels I∗ and Umax. Finally, when a sufficient proportion of the population has already been infected and the herd immunity level is close enough, a light final release of lockdown measure allows to retrieve a regular level of social interactions and end the epidemic phase. Such a control strategy allows to achieve an epidemic size very close to the herd immunity level, which corresponds to the minimal size of an epidemic, in comparison to a benchmark case without intervention for example. Besides, given the current level of information and confidence on the COVID-19 dynamics modelling, the design of robust optimal control strategy should be seen as a minimal requirement. For our model, the enhanced structure of the optimally controlled epidemic dynamics reveals to be very robust to variations of the model specifications within a reasonable range.
The effect of the detection effort within the population reveals to be more subtle. As explained above, two types of detections are considered. Detecting quickly infected individuals and breaking the contamination chains reveals to be key in reducing the economic and sanitary burden of the epidemic. First, it allows to slow down the exponential growth of the epidemic and decrease hereby the required epidemic size for reaching the herd immunity and the induced death toll. It also reduces the lockdown effort on the population, which helps minimize the socio-economic impact. Optimizing over the detection rate of infected individuals reveals the huge impact on both sanitary and economic burdens of early effort in detection, as well as a stable threshold level in detection. Using the detection of infectious individuals as a control lever allows to keep the reproduction number below 1 while limiting the lockdown effort and the induced socio-economic cost. Concerning the detection of immune individuals, it allows to reduce significantly the socio-economic burden, later in time during the epidemic, whereas its impact on the sanitary outcome seems rather limited.
The rest of the paper is organized as follows. In the ‘Methods’ section (2), we derive our SIDUHR+/− model from a standard SIR model (see Section 2.1), and explain all the components of the dynamics. The infectious individuals may either: i) be asymptomatic or have mild symptoms that do not require hospitalization, ii) need hospitalization with or without intensive care. In section 2.2 we present the optimal control problem and objective function of public health authorities. Possible control levers are discussed in section 2.2.1, with lockdown, detection efforts, and ICU capacities limitation. In section 2.2.2 we motivate the choice of the objective function, and discuss optimal control in sections 2.2.3 and 2.2.4.
In the ‘Discussion’ section (3), we compute the optimal strategy based on a choice of parameters from the COVID-19 epidemic in France, but of course the methods and findings can be transposed to any other country or set of parameters. In section 3.1, the benchmark scenario is described. Then, in section 3.2 we consider optimal lockdown, and discuss the 4 stage dynamics. In section 3.3, we provide a sensitivity analysis, and discuss the robustness of our findings in response to changes in the model specifications and parameters. In sections 3.4 and 3.5, we consider the impact of two other controls: the ICU sustainability level and detection resources. Given the simplicity of the compartmental model considered here, our purpose is not to give detailed prediction of the epidemic but rather to highlight very interesting aspects of the interplay between different levers (lockdown, detection and ICU capacity), provide qualitative description of the outcomes in each scenarios and give rough estimates of the duration and size of the epidemics.
Let us finally emphasize that the level on uncertainty upon the characteristics of the epidemic dynamics is still currently very high. The enhanced conclusions of this paper remain valid within the scope of the chosen model specifications chosen here, which are in line with the current understanding of the epidemic dynamics.
2 Methods
2.1 The SIDUHR+/− model and its dynamics
Compartmental models, where the population is divided into classes defined by the status of the individuals with respect to the disease or how they are taken in charge by the healthcare system are very popular in mathematical epidemiology (see e.g. Anderson and May (1991); Diekmann et al. (2012); Ball et al. (2019) for an introduction). One of the simplest and most fundamental of all epidemiological models is the popular SIR model, developed in Kermack and McKendrick (1927). The population, of fixed size, is divided into three categories: susceptible (S), infected (I) and recovered (R) individuals. We denote by St, It and Rt the respective proportions in the populations at time t (measured in days). In order to adapt the SIR model to the COVID-19 epidemic and incorporate lockdown and testing strategies with their economic and social impacts, we modify some rates and add compartments to the SIR model.
We want to take into account two important features of the COVID-19 pandemic. The first one is the important proportion of asymptomatic patients, and the small use of testing in several countries. We will split the I group in two: I− the non-detected, mostly asymptomatic individuals, but contagious, and I+ the detected part of infected individuals. The second feature is the problem highlighted in the ‘flatten the curve’ concept (discussed in Ranney et al. (2020), Kissler et al. (2020) or Gudi et al. (2020)) which means that a goal in the choice of a public health intervention should be to avoid a surge of demand on the health care system and in particular in ICUs. From a modelling perspective, we introduce the proportions Ht and Ut of individuals using the health care system with or without the need of ICUs. There is an exogenous sustainable limit Umax for the ICU system, above which the death rate is significantly higher.
Compartments
More specifically, all possible states can be visualized on Figure 1. The parameters for the dynamics are described in the next section.
S: susceptible, never tested positive, never infected and not dead (where people are by default). The can get infected by infectious and not quarantined individuals, i.e. individuals in I−. When susceptible individuals get infected, they move to I−.
I−: infected non-detected. Those individuals can be asymptomatic, not sick enough to go the the hospital, and are not detected. They can either get tested (with type-1 tests, and then move to I+), get sicker and then go to the hospital (H) or simply recover and get immune, but still not detected (R−)
I+: infected detected (and non-hospitalized). They can either get sicker and then go to the hospital (H) or recover, in which case they end up in the compartment R+ since they were tested positive. Here, we assume that the individuals in I+ cannot infect anyone since they are strongly isolated.
R−: recovered non-detected. Those can be detected using blood-type tests (also named type-2) and move to R+.
R+: recovered detected. We assume here that all recovered individuals (both R− and R+) are immune long enough with respect to the epidemic duration.
H: hospitalized (and detected). All people entering the hospital get tested. They can either get sicker and then go to intensive care (U), or get immunity after recovering, and then move to R+. Note that H is an intermediary state, before getting really sick, or die. Individuals are not allowed here to go from I (detected or not) to D. Furthermore, we did not distinguish explicitly between mild and severe infected (as in many other models) in order to emphasize the role of detection resources, but, somehow, H could correspond to severe cases.
U: hospitalized in ICU, for people at a more advanced stage (and detected), who need ventilators and other specialized medical care. They arrive from H, and they can either get immunity after recovering, and then move to R+, or die (D).
D: dead, from the disease.
The SIDUHR+/− model
Here, recovered are not contagious anymore and we assume that they get immunity (and therefore, can not get infected again). This is a rather strong assumption, especially on a long term horizon. Choe et al. (2017) used a longitudinal study to show that those infected by MERS in South Korea (with a significant degree of immunity post-infection) had immunity that lasted for up to a year, as well as, more recently Aldridge et al. (2020), on historical patterns of three common corona-viruses.
Dynamics
The evolution of the sizes of each compartments is assumed to be modelled by the following system of ordinary differential equations:
Let us explain the different parameters. The transmission rate 1 δt)β is the rate at which an undetected infected individual transmits the disease to a susceptible one. Multiplying this parameter by 
, which is proportional to the number of pairs that we can form with an S and an I− individual, provides the total infection rate at the level of the population and at time t: 
. Heuristically, this quantity can also be understood as the fraction of the population infected during a unit time interval at time t. The infection rate β is reduced by a time-dependent factor (1-δ) where δ ∈ [0, 1] comes from social distancing, quarantine, isolation interventions that reduce infectious contact rates. The strength of the lockdown intervention interprets as follows: δ = 0 means no intervention while δ = 1 means no social interaction. We consider two ways of detecting individuals: short range tests allow to discover new infectious individuals in I− and we denote by λ1 the rate at which each of these infectious individuals gets detected, long range test allow to find formerly infected and hereby immune individuals that are now in R− and we denote by λ2 the corresponding rate. Here we are interested in the case where the parameters δ and λ1 are control variables and in the sequel, these parameters are hereby time dependent.
The recovery rate for an infected individual is γI R. The rate at which an infected individual enters the health system is γI H. We assume here that both rates γI R and γI H are the same for detected or non-detected infected individuals. The individual transition rates from H to U or R+, from U to R+ or to D are denoted by γHU, γH R, γUR(U) and γUD(U) respectively. Notice that the two later rates γUR(U) and γUD (U) are not constant but depend on Ut : when the ICU limitation is reached, the extra patients who are not taken in charge by the system can not recover and are exposed to extra death rates, as explained in the Appendix A.
These parameters are chosen following the tracks of Domenico et al. (2020); Salje et al. (2020). In these papers, some parameters are fixed according to clinical studies and other are estimated from French data. The detail on how these parameters are computed in the present paper is given in Section A. However, we emphasize that although we choose French benchmarks, our input is more on methodology and can be of interest for other countries or epidemics as well.
The system described here can be derived from individual based stochastic processes (e.g. Ball et al. (2019)) and thus, there is an underlying individual based model where all the rates have a probabilistic and statistical interpretation (see Section A, with connection to continuous time Markov chains. As we will discuss later on, those parameters can therefore be interpreted as inverse of transition time lengths.
The initial time t = 0 corresponds to the time at which policies that reduce the infection rate are put in place (δ > 0), or the time at which testing is started (λ1 > 0). For the French
COVID-19 epidemic, it would correspond to March 17th 2020, but again, we emphasize that our methodology remains of interest for other countries or diseases (see e.g. Pedersen and Meneghini (2020) for country specific informations).
Basic and effective reproduction numbers ℜ0 and ℜt
The basic reproduction number ℜ0 corresponds to the expected number of individuals directly contaminated by a typical infected individual during the early stage of the outbreak (when the proportion of susceptible individuals is close to 1), see Diekmann et al. (1990). When ℜ0 < 1, the disease does not spread quickly enough, resulting in a decay in the number of infected individuals. But when ℜ0 > 1, the infected population grows over time.
As recalled in Trapman et al. (2016), the value of the ℜ0 is model-dependent. For the model considered here, only the I− individuals can transmit the disease to infect susceptible individuals. If we see I− as the infected class, and if we merge the classes I+, R−, R+, H, U and D into a large ‘recovered’ class, in this SIR type of model, the dynamics of 
has infection rate (1 − δ) β and removal rate λ 1 + γI R + γI H, assuming that δ = δ 0 and 
are constant over time. In our model, the basic reproduction number is
Without intervention (i.e. for δ, λ1 = 0, 0 over time) and with our chosen parameters provided in Table 1, this corresponds to ℜ0 = β γI R γI H = 3.3 which is in line with Salje et al. (2020). Sensitivity of our results with respect to variations of ℜ0 within the range 3.3, 3.6 is discussed in Section B.2.
Parameter values for the base-scenario, referring to (D) = Domenico et al. (2020), (S) = Salje et al. (2020) or (W) = World Health Organization (2020). Individual rates are given except for the transition rates from U to R or D where the global rates at the population level are given, to better explain the nonlinearity. Details are presented in Appendix A.
In the case where λ and δ evolve with time, we end up with a dynamic reproduction number (ℜt)t that is called effective reproduction number, see Anderson and May (1991). More specifically, at time t ≥ 0, let us define
Its interpretation is very close to the static version ℜ0. When ℜt < 1, the average number of secondary cases started from one primary case with symptom onset on day t, dies out quickly, resulting in a decay in the number of infected individuals in the population. But when ℜt > 1, the infected population grows over time. So we should expect to have controls on the quarantine level δt and on the testing rate λt that will constrain ℜt to be reduced beyond 1, at least after some starting time.
Another information brought by the ℜt < 1 is that, when S* is smaller than the so-called ‘herd immunity’ threshold, the epidemic enters into a sub-critical phase where it goes to extinction.
In our case, the herd immunity threshold S∗ without intervention is around 30% and durable lockdown and detection efforts can allow to reduce it significantly until the arrival of a vaccination solution.
2.2 The optimal control problem
In the previous section, we described our epidemiological model in order to characterize the dynamics of the virus spreading processes. We now consider a government acting as a global planer, who wants to control the epidemic dynamics in order to balance the induced sanitary and economic outcomes. In order to mitigate the effects of an epidemic, several parameters in the model are now interpreted as ‘control levers’, and we now focus on the derivation of their optimal dynamic choice. Optimizing intervention strategies is an important policy issue for the management of infectious diseases, such as COVID-19. Optimal control in the context of pandemic models has been used since Abakuks (1973, 1974), Bobisud (1977) or Sethi (1978), in the 70’s.
2.2.1 Possible control levers Quarantines and Lockdown
Lockdown measures proved to be effective in controlling the COVID-19 outbreak in China, as recalled in Kucharski et al. (2020) or Lai et al. (2020). Quarantine is a rather old technique used to prevent the spread of diseases. It refers to the restriction of movement of anyone (not necessarily sick people), and it usually takes place at home, and may be applied at the individual level. It should be distinguished from isolation, which refers to the restriction of movement of infectious individuals who have a contagious disease. It can be done in hospitals, in dedicated facilities, or at home. Those two are seen as active controls, and are different from contact surveillance, which is passive. Modern quarantine’s goal is to reduce transmission by increasing social distance between people, by reducing the number of people with whom everyone can be in contact with. This includes canceling public gatherings, closing public transportation, etc. A lockdown is intended to stop people from moving between places, and it could involve cancelling flights, closing borders, and shutting down restaurants. The idea is to reduce the flow of people to curb transmission, and we might consider here that lockdown and quarantine measures are almost equivalent.
Sethi (1978) suggested an optimal quarantine problem, in an SIR model, where a proportion of infected people could be quarantined. He proved that the optimal strategy was either to quarantine all infected individuals (if so, at early stages) or none. In the context of the 2003 SARS pandemic in Singapore, Ooi et al. (2005) recall that a Home Quarantine Order was signed, but it was more an isolation of sick people than a lockdown as the ones used to lower possible consequences of COVID-19. But options considered are different, from one country to another. Wong et al. (2020) mentions that several countries closesd their borders, in most European countries, schools and restaurants were closed for weeks.
Lagorio et al. (2011) tried to quantify the effectiveness of a quarantine strategy, where healthy people are advised to avoid contacts with individuals that might carry the disease, using network based models. Here, the parameter has a direct interpretation, since it is related to the average rate of contact between individuals. In Yan and Zou (2008) or Ku et al. (2020), quarantine and isolation are used as control. In our model, those controls correspond to the δt component. As in Chinazzi et al. (2020), such control can also be related to travel bans, for a weaker form than strict quarantine. Note finally that Alvarez et al. (2020) suggested that a complete lockdown was impossible to reach, and that an upper bound of 70% (of the population) should be considered. In this paper, we do not incorporate any constraint (except that we cannot lockdown more than the entire population), but interestingly, in the numerical simulation, the upper bound we obtained as optimal is very close to this value.
Testing, Tracing and Isolating
The second most important lever that can be used is detection. As claimed by several governments (see Gostic et al. (2020)) in the context of the COVID-19 pandemic, testing is key to exit lockdown, and mitigate the health and economic harms of the virus. Here, we assume that health authorities can perform two kinds of active detection actions.
Type-1 test, that could be an antigen test (Reverse Transcription Quantitative Polymerase Chain Reaction (RT-qPCR) – also called molecular or PCR – Polymerase Chain Reaction – test). This test allows to detect whether an individual is currently infected or not. It should be used to determine who self-isolates and for contact tracing. For that test, a sample is collected – usually with a deep nasal swab (and analysed in a laboratory). This is a short-term test : when performed at time t on an individual, we know whether an individual is infected or not. Together with proper detection means, the use of this test corresponds to the 
control in our dynamics. This test allows to detect and isolate infectious individuals. In our model, they transfer from compartment I− to compartment I+. It is performed as mandatory when someone arrives at the hospital. Hereby, any individual being at some point in compartment I+, H or ICU ends up in the identified immune compartment R+ if he survived the epidemic. We assume for simplicity that those tests do not have false positive or false negative and leave for further research the impact of the test sensitivity on the epidemic dynamics and optimal control. Thus, the tests are all positive on individuals in I− and negative on individuals in S and R−. Other individuals will not be tested. Type-2 test is an antibody test (using serological immunoassays that detects viral-specific antibodies – Immunoglobin M (IgM) and G (IgG) – also called serology or immunity test). It allows to test whether an individual can now be considered immune to the virus. It could potentially be used to issue immunity certificates in order to help restarting the economy quicker. It also reduces the overall uncertainty and potential fear of individuals to be infected, and has a positive impact on the economic consumption. It is also useful for contact tracing purpose as it helps identifying individuals that can not become infectious again. For such a test, a blood or saliva sample is applied to a strip that identifies presence of antibodies. We assume here that immunity lasts long enough with respect to the epidemic duration, and R+ regroups the collection of immune detected individuals. It is a long-term one : when performed at time t on an individual, we know if that person had the disease before. The detection using those tests identifies in our model to the control variable λ2. It allows to transfer individuals from compartment R− to compartment R+, but again, people in S will be tested also, as well as people in I−. As for Type-1 tests, we will suppose here that these tests do not make any wrong identification.
For type-2 tests, a small sample of a patient’s blood – for instance via a pin prick – and the test looks for two specific types of antibody: IgM and IgG. IgM are the first antibodies to be produced by the immune system. They have a half-life of around five days, and they usually appear within five to seven days of infection and peak at around 21 days. Detection of these antibodies suggests the person has existent or a recent infection. IgG antibodies are more numerous and can be detected around 10 to 14 days after infection. The presence of these antibodies indicates a person has recovered from the virus and is now immune.
‘Raising the line’ and ICU Capacities
So far, we have assumed that the sustainable capacity of ICUs was not a control, and should be considered as an exogeneous fixed boundary (denoted Umax). As discussed in Gelardi (2020), preventing a health care system from being overwhelmed requires a society to do two things: ’flatten the curve’ – that is, slow the rate of infection so there aren’t too many cases that need hospitalization at one time – and ’raise the line’ – that is, boost the hospital system’s capacity to treat large numbers of patients” (see also Barclay (2020)). Therefore, a natural control variable is the level of the line, or Umax as we named it.
Nevertheless, we will not consider Umax as a control variable that can be optimized (such as the strength of the lockdown, or the effort in detection), since it is difficult to assess the cost of raising it. Nevertheless, in Section 3.4, we consider the case where health authorities choose to increase that capacity, by making ventilators and other medical material necessary in the context of the pandemic. But it cannot be increased indefinitely since ICU also require trained personal and even if adjustments can be considered, we assume that it is not realistic to assume that Umax can be increased by more than 50%.
2.2.2 Objective function
We now turn to the design of the objective function, trying to take into consideration the sanitary and socio-economic outcomes of the lockdown and detection policy. In the context of planning vaccination campaigns, da Cruz et al. (2011) and Verriest et al. (2005) suggest a convex quadratic cost function by minimizing both the number of infected individuals in a time horizon and the cost to implement the control policy. In Kim et al. (2017), a model for 2009 A/H1N1 influenza in Korea is considered: the goal is to minimize the number of infected individuals and the cost of implementing the control measures, and the cost is taken to be a nonlinear quadratic function. Quarantine and vaccination are considered as control variables in Iacoviello and Liuzzi (2008), and the optimal control is obtained by minimizing a quadratic cost function. Inspired by those approaches, we will also consider quadratic cost functions. Before going further, let first introduce some relevant quantities characterizing the epidemic phase that will reveal useful in the upcoming analysis.
Quarantined individuals
Let denote by Q the quantity of people concerned by the lockdown policy and defined by:
All the individuals in Qt are having the level of contact rate 6t with the population at time t. This proportion of individuals identifies to the people for which we can not say if they already contracted the virus or not. Therefore, they are all concerned with non targeted lockdown strategies as well as possibly randomized testing trials. Recall that, on the opposite, we suppose that infected detected individuals in I+ are isolated and have no social interactions with the population, whereas detected immune individuals suffer no mobility constraints and can reach a level of local interactions similar to the one they had before the beginning of the pandemic.
Global level of social interactions
On the other hand, the global level of social interactions among the population is denoted Wt and given by:
The quantity Wt represents the proportion of social interactions in the population and can be interpreted as a macroscopic labour force level for the economy at time t.
Testing resources
Finally, in order to measure the detection efforts of infected and immune individuals, we introduce the following metrics; which identifies to the number of virologic and anti-body tests done in a (somehow quite unrealistic) fully randomized detection trial:
Observe that the different formulation between both metrics is due to the fact that individual are automatically tested using virologic detection methods, when admitted at the hospital.
Vaccination arrival date
All health authorities hope for the development of a vaccine for Covid19 in a close future. In order to encompass uncertainty concerns upon the arrival date of the vaccine, we consider the arrival date of the vaccination solution to be a random time with exponential distribution of parameter α, denoted τ. According to recent studies, Jiang et al. (2020); Cohen (2020), the creation of such vaccine for a sufficient quantity of individuals can presumably be assumed to require around 500 days. We shall consider a parameter α equal either to 0, 1 250 or 1 500 for our numerical experiments. For convenience, we assume here that both a vaccine and a cure simultaneously appear at time τ. Alvarez et al. (2020) also use such a discount approach, on top of some economic discount rate (assuming that the vaccine will arrive at an expected date of 18 months).
Sanitary cost
For the sanitary cost, it seems rather natural to simply consider a death count metric given by DT. Nevertheless, Oliver (2013) legitimates the use of an appropriate discounting of health outcomes. About the kind of discount we should use, van der Pol and Cairns (2002) discusses the use of hyperbolic discount (opposed to classical exponential discount), in the case of social choice regarding health outcome. The hyperbolic discount allows for asymmetry in discounting, and non-stationarity, in the sense that postponing in two years now can be substantially different from postponing in three years (from now) next year. On the other hand, van der Pol and Cairns (2002) claims that is not necessarily a realistic assumption, so that using a standard time-consistant exponential discount makes sense in terms of health outcomes. Besides, the longer we can postpone deaths, the more likely a vaccine will be found before. Adding up a exponential discounting factor reveals to be also perfectly consistent with the consideration of random development time of a vaccine solution at large scale, with exponential distribution. Hence, we will consider a sanitary cost of the form:
recalling that the random arrival time of the vaccine follows an exponential distribution with parameter α.
Economic and social cost
The quantification of the socio-economic induced by lockdown restrictions is not easy to quantify properly. More involved modelling may wish to take into account a feedback effect between labor and consumption levels together with the epidemic propagation characterized by the global rate of social interactions within the population. Such approach goes beyond the scope of this paper and we solely intend to quantify the impact of the epidemic prevalence in terms of reduction of social interactions. Here, we would like to capture the loss of productivity and well-being among the population due to lockdown situation. In a very general setting, the production function is a function of capital input (K, that usually corresponds to machinery, equipment, buildings, etc.) and labor (W, that represents the total number of people working): the output produced in a given period of time is classically given by a Cobb-Douglas production function function (as introduced in Cobb and Douglas (1928)) K1−aWa, with a ∈ (0, 1). Power coefficients identify to the output elasticities. Here, the function display constant returns to scale, in the sense that doubling the use of capital K and labor W will also double produced output. Assuming that, on the short term (say less than a year) capital remains constant, it means that if at time t, only a proportion Wt of individuals can be economically active, the productivity loss is proportional to 
. Thus, the inter-temporal economic cost at time t is proportional to 
, or (up to an additive constant) 
. Nevertheless, several other metrics have been used on the literature in order to account for the economic cost induced by the lockdown situation: Guerrieri et al. (2020) or Eichenbaum et al. (2020) considered a linear function, while Piguillem and Shi (2020), Bernstein et al. (2020) or Djidjou-Demasse et al. (2020) considered a convex function. As for the other cost structures, we choose for simplicity to keep a quadratic structure quantifying the distance between the current global rate of social interactions W given in (5) (and identified as a metric for welfare, well-being or working-force) and its optimal value 1. Hereby, the social and economic cost will be given by:
where we recall that the discounting rate α encompass the distribution of arrival time of a vaccination solution solution.
Detection cost
The quantification of efforts put into the detection of infectious individuals is a difficult task, as outlined in Liu et al. (2020) in the case of children, but several detection means are possible. As mentioned in Salathé et al. (2020) and Hellewell et al. (2020), testing is only part of a strategy. The World Health Organization recommends a combination of measures: rapid diagnosis and immediate isolation of cases, rigorous tracking and precautionary self-isolation of close contacts. Cohen and Kupferschmidt (2020) compares various techniques used to detect infected people. Recall that we assume that a virologic testing is performed for all individuals entering into the hospital. Besides, proper tracing of contamination chains together with potential random screening of the population should be conducted on the non detected individuals, i.e. on the sub-population Qt. This group contains the susceptible St, the undetected infected 
as well as the undetected immune 
. As antibodies detection should also be performed on the same group Qt, it seems important to take into account the impact of the size of this group within the population. Besides, as testing resources are sparse, the metric quantifying the effort of detection must take into account the increasing difficulty for scaling massively testing resources. For all these reasons, we pick a cost for detecting infected or immune individuals of quadratic form as follows:
where we recall that 
and 
are given in (6). Again, we consider here a detection cost, not a testing cost.
2.2.3 Optimal control under ICU capacity constraint
We are now in position to turn to the design of the global objective function. We are looking towards the societal optimum chosen by an omniscient government acting as a global planer and is facing a multi-objective control problem. As in Kumar and Srivastava (2017), we shall consider a weighted sum of different costs:
the sanitary cost described in (9) directly related to the mortality rate of the virus;
An economic and social cost given in (8) due to the reduction of social interactions within the population;
A detection cost for identifying infectious and immune individuals; which will be mainly discussed in Section 3.5 below.
Combining these effects, the global objective function is given by:
where the wi terms are positive weights associated to each marginal cost. The choice of proper weights wi is a hard ethical task, that we choose to avoid as out main purpose is to highlight the main patterns of the optimally controlled epidemic dynamics. We pick some weight levels for our numerical experiments and study in Section B.3 the sensitivity of our findings to strong variations of these weights.
In our approach, the global planer tries to minimize the global cost function given in (11) using the lockdown control lever δ together with the detection lever λ = λ1; λ2. Nevertheless, we did not yet take into account the risks and outcomes induced by the saturation of ICU hospital care facilities. The dynamics of the SIDUHR+/− model already takes into account the strong impact of ICU saturation on the mortality rate on individuals with severe symptoms. Whenever the proportion of individual in U exceeds the upper bound level Umax, patients can not be treated correctly so that patients are more likely to transfer to D instead of R+, in comparison to calmer times where U < Umax. With such feature, the optimal lockdown and detection strategies should undoubtedly try to limit the duration of periods where the maximal ICU capacity is exceeded. Nevertheless, as we believe that exceeding capacities of ICUs would have a strong negative impact on the hospital staff and even the entire population, we rather choose to impose the non-saturation of ICU as a required state constraint on the system. Hence, we try are looking for a robust solution to the following control problem:
where 𝒜 denotes the set of admissible strategies driving the dynamics of the SIDUHR+/− model and is given by
2.2.4 Numerical approximation of the solution
From a mathematical point of view, the optimal control problem of interest falls into the class of control for deterministic dynamical system over infinite horizon in the presence of additional state constraint. The detailed presentation of the numerical resolution of this problem is presented in Section C.2 below. We briefly recall here the main underlying approximations used in the numerical approach.
In order to optimize the impact on the dynamical structure of the underlying SIDUHR+/− of the chosen control, we make use of Pontryagin maximum principle. For numerical purposes, we restrict our analysis to the consideration of a control problem with finite horizon. We thus pick a maturity T large enough in order to have a very small remaining level of infected individuals at time T. Besides, whenever the level of infected is small enough, SIR type dynamics are not fully reliable anymore, limiting the impact of such approximation on the derived optimal strategy. In practice, the numerical experiments are indeed not sensitive to the choice of maturity T, as soon as it is picked large enough. For the graphs shown here, we used T = 700 days (almost two years), corresponding to a scenario where a vaccination solution is available within 2 years. We did run computations up to T = 900 looking for a potential impact on the interpretation of the optimal strategy. But since we did not observe relevant ones, we restricted our attention to T = 700 in order to simplify the overall presentation of the results. Besides, in order to encompass the state constraint into the maximum principle based algorithm, we simply represent it under the form of a penalization cost, whenever the ICU constraint is not satisfied. The penalization is strong enough to discourage crossing this threshold for most scenarios. See Appendix C for more details on the implementation.
The algorithm then relies on an iterative procedure. Time is discretized so that the search for an optimal control reduces to the search for one vector per control variable (δ, λ1, λ2), with one value per time step. Starting from an initial guess, at each iteration the approximation for the optimal control is updated based on the expression of the gradient in terms of the solution to a forward-backward system of ordinary differential equations. In order to ensure that the controls remain between 0 and 1, a projection of the new controls on the interval 0, 1 is performed at each iteration.
3 Discussion
3.1 Benchmark scenario without intervention
Let us first investigate the outcomes of the epidemic dynamics when no lockdown, detection nor quarantine policies are available. As commonly done in the current literature, we do not take into account the reaction of individuals in response to the virus prevalence dynamics, as discussed e.g. in Elie et al. (2020). Anticipating realistic individuals behaviors in such context is still difficult, as no available data allows to tackle this phenomenon properly. We suppose that the virus dynamics has no impact on the level of social interactions of individuals, which remains constant equal to β 0. Such analysis provides a very useful no-intervention benchmark scenario for the rest of our study. The dynamics of each compartment is provided in Figure 2 while Table 2 provides important epidemic characteristics computed at terminal date T = 700 days1.
Evolution of states without control, over 200 days.
As no intervention interferes with the epidemic dynamics, the overall duration of the epidemic is very short as the number of susceptible vanishes exponentially fast. The initial reproduction number ℜ0 = 3.3 induces an epidemic lasting around 80 days. The virus prevalence is quite important over that period, leading to the contamination of almost 96% of the population, way above the herd immunity threshold, S∗ ∼ 71% in our example. The epidemic peak happens after a one month period, and as much as one third of the population is infected at that date. Around 7% of the population is hospitalized due to the virus, leading to a highly severe overwhelming of the healthcare capacity system for around 60 days. The epidemic induces in total a mortality of almost 1% of the population, while half of deaths occur while the ICU capacity is saturated.
It is worth noticing that our model, with the parameter values of Table 1 that were calibrated on the French epidemic of COVID-19 (Domenico et al. (2020); Salje et al. (2020)), gives predictions that are in accordance with the ones made by Roux et al. (2020) in absence of interventions and neglecting the ICU capacity constraint (see Appendix A for details).
3.2 Optimal lockdown policy without detection effort
In order to emphasize the marginal role of each component in the optimal control problem of interest, we first focus on the optimal lockdown strategy and corresponding epidemic trajectory induced without any detection effort. This does not mean that we do not test anyone, we simply assume that only the people entering in the hospital are tested with type-1 tests. No testing is done on infected individuals without severe symptoms (λ1 = 0) and there no testing is done to detect recovered people after being asymptomatically sick (type-2 test, λ2 = 0).
Hereby, we are starting with a weaker form of the optimal control problem, where we control only the lockdown intervention policy:
with the following choice of parameters and wsanitary = 100000, weco = 1 and α = 0. The sensitivity of the enhanced results to this ad-hoc parametric choice is discussed in Sections B.3 and B.4. Besides, once we will have a better understanding about the derived lockdown strategy and its robustness, we will focus on the impact of detection strategies in Section 3.5. The optimal lockdown strategy together with the main characteristics of the pandemic dynamics are presented in Figure 3 and Table 2. The optimal control problem has been solved using the numerical algorithm described in Appendix C.2 and numerical approximation errors undoubtedly remain in the results presented here.
Evolution of state proportions – St (susceptible), It (infected, detected or not, 
), Rt (recovered and immune, detected or not, 
), Ut (hospitalized in ICU), Dt (dead), ℜt(dynamic version of the reproduction number), Wt (labor force available), Qt (susceptible to be quanrantines, 
) and δt (lockdown intensity) – with optimal control δ (plain line) and without control (dashed line). States vary here from t = 0 (beginning of the pandemic, or at least of possible measures, with δ ≥ 0) to t = 700, and computations are based on a dt = 1/5 days.
When we follow the lockdown intervention (δt) proposed by the optimal response policy, four distinct consecutive phases can be observed:
Quick activation of a strong lockdown: The optimal strategy consists in activating as soon as possible a strong lockdown measure on the entire population as no targeting or detection is considered or possible at that stage. This strategy has been widely used in March and April 2020 as half of the world population was under lockdown. With our parameters, the contact rate over the population is reduced by 80% over a couple of days, in order to stop immediately the exponential growth of infected and infectious individuals. Hereby, the size of the epidemic peak is highly reduced and, after less than one month, only 2% of the population is contaminated at the peak of the epidemic (instead of 34% in the no-intervention benchmark case). Of course, at the exact same date, the effective reproduction number ℜt goes below 1. During this initial short period, the effort in economic activity reduction has been very important: the welfare level W went down by 80%, while at the same time the occupation rate in ICU hospital care system has been constantly rising.
Light lockdown release and prevalence decrease: During the short second phase, the intensity of the lockdown remains intense (above 60%) in order to keep ℜt below 1. The prevalence of the virus decreases slowly, while the level of ICU admissions is still rising. After the previous phase where urgent decisions had to be taken by the global planner, a short transition period of a couple of days now starts. During this period, the proportion of infectious individuals decreases strongly in order reach an optimal prevalence level I∗ in the population, required for the very long next phase.
Long period with stable prevalence and ICU sustainable capacity: The third phase lasts around one year during which the effective reproduction number ℜt remains stable at level 1. Hereby, the prevalence of the virus remains stable while the ICU capacity remains at the sustainable level Umax. During this one year period, the level of contact rate within the population is slowly growing at a regular constant rate, moving gradually from 40% to 80%. The working force is rising back to more normal levels and the economy is slowly restarting: one year after the beginning of the pandemic, around 70% of regular social interactions level is already attained within the population. The mortality rate over this period remains stable, and the death toll grows linearly, almost reaching its terminal value at the end of this period.
Terminal slow progressive release of the lockdown: The last phase is a couple of months long and consists in bringing progressively back to normal the level of social interactions, while the occupation level in ICU is decreasing and the prevalence of the virus is heading towards 0. The effective reproduction number ℜt is smaller than 1. At some point, the deterministic equations (1) are no more valid and the system becomes stochastic, but the extinction of the disease occurs with probability 1.
The optimal structure in four steps for controlling the pandemic dynamics is quite robust and very natural. It is worth observing that although the dynamic reproduction number is not part of the criterion of interest, reaching its optimal trajectory with four successive patterns is key for controlling the dynamic level of admissions in ICU. Using the optimal control policy, the infections by the virus in the population have been spread out over 500 days, providing a sustainable level of admissions in ICU for the sanitary system. In comparison to the no-intervention benchmark case, the death toll has been divided by almost 100, thanks to a strong reduction in the level of social interactions. The size of the pandemic only represents 70% of the population in comparison to 96% in the no-intervention scenario.
A nice visualisation of the epidemic dynamics is provided by looking at the phase diagram presented in Figure 4. In the no-intervention scenario, the epidemic dynamics starts from the point (S, I = (1, 0) and moves up and left until reaching the herd immunity threshold S∗ corresponding to the time of epidemic peak. Then, I starts decreasing until the end of the epidemic, point where S is valued around 4% in our example: hence, the final epidemic size is almost 96% of the population. In the optimally controlled scenario, we can clearly identify the 4 phases described above: I is still rising during the first phase until reaching a small epidemic peak level, it starts decreasing during the second transition phase until reaching the optimal level I∗ corresponding to the level Umax of ICU capacity. It remains stable there during the long third phase and finally decreases in the last phase until reaching a susceptible proportion close to the herd immunity level S∗.
Phase diagram St (susceptible) versus It (infected, detected or not, 
), together with the herd immunity threshold S∗ and the level I∗ of infected corresponding to U = Umax.
Evolution of states with optimal control δ when ‘raising the line’ (increase of Umax by 50%). The plain line is the benchmark scenario discussed in Section 3.2; the dashed line corresponds to the scenario with a capacity of Umax which increases linearly from time t = 0 until t = 60 days up to 1. 5 times the initial capacity and then stays at this level.
Of course, both dynamics depend on the value of the parameters, such as the initial virus prevalence 
, the starting basic reproductive number ℜ0, expectations about possible arrival of vaccine and cure (through parameter α or weights considered in the objective function). But as we can see in Appendix B, and as we will discuss more now, the enhanced results are robust to the parameter specifications.
3.3 Sensitivity of the optimally controlled epidemic dynamics to the model specifications
The calibration of SIR type dynamics to current available data is currently a challenging task, as the quality of available mortality and hospitalization figures is still questionable and incomplete. It is worth noting that one of the main advantages of the SIDUHR+/− dynamics is the possibility of calibrating its dynamics to mortality, infected detected, hospitalized and ICU data points. Still, such inverse calibration problem remains ill-posed and the precision range over the calibrated parameters is not satisfying.
In order to demonstrate the robustness of our findings in such context, we provide a sensitivity analysis of our results to modifications in the model specifications. More specifically, we analyse the impact of reasonable variations with respect to the initial prevalence 
in Appendix B.1, the basic reproduction number ℜ0 in Appendix B.2, the balance between sanitary and socio-economic cost in Appendix B.3, and the arrival date of a vaccination solution in Appendix B.4.
Let now present the main outputs of those numerical experiments, while keeping in mind that numerical errors due to the optimization approximation scheme can still alter some numerical results presented in this study.
On Figure 6, we can visualize the impact of a change in 
, with a smaller (five times) and larger (also five times) value than the one used in benchmark computations (we used 
= 5‰). The lower value was suggested from Sachdeva and Sheth (2020), with 1‰, five times lower than our estimate, and an upper value five times upper, of 2.5%, close to the situation two weeks after according to Sachdeva and Sheth (2020). A smaller value means that measures were taken earlier, but overall, it looks like the overall impact would have been rather small. Note that we observe here that a smaller 
might cause more deaths (DT), because of some balance with economic gains (economics losses are smaller with a lower 
). Observe also that variations in 
have almost no impact on the ICU saturation dynamics: it might start earlier with a large 
, but the duration remains more or less the same.
Evolution of states with optimal controls (δ, 0, 0) – without any testing – for three different values of 
. The plain line is the benchmark scenario discussed in Section 3.2; the dashed line corresponds to a smaller weight for 
while the dotted line corresponds to a larger 
.
On Figure 7, we can visualize the impact of a change in ℜ0, with a smaller (−0.3 i.e. −10%) and larger (+0.3 i.e. +10%) value than the one used in benchmark computations (ℜ0 = 3.3). Such variation in ℜ0 is compatible with the literature, see Salje et al. (2020); Domenico et al. (2020). The overall 4-phases global patterns of the optimally controlled epidemics remain identical. The lockdown effort increases slightly but significantly with ℜ0, together with the ICU saturation phase duration, the global death toll and global economic cost. Note that a decrease of ℜ0 from 3.6 to 3.0 (attained for example through extra hygiene habits) would decrease ICU saturation time length by almost 25%.
Evolution of states with optimal controls (δ, 0, 0) – without any testing – for three different values of ℜ0. The plain line is the benchmark scenario discussed in Section 3.2; the dashed line corresponds to a smaller value of ℜ0(−0.3) while the dotted line corresponds to a larger value of ℜ0 (+0.3).
On Figure 8, we can visualize the impact of a change of the ratio between the sanitary weight wsanitary and the socio-economic one weco, with a smaller (half) and larger (twice) value than the one used in benchmark computations. The global patterns in the epidemic dynamics remain globally similar. A larger sanitary weight induces a smaller death toll, a smaller epidemic peak together with a shorter ICU saturation period. The economic gain or loss seems to be mostly sensible during the second half of the epidemic duration.
Evolution of states with optimal controls (δ, 0, 0) – without any testing – for three different values of wsanitary. The plain line is the benchmark scenario discussed in Section 3.2; the dashed line corresponds to a smaller weight for wsanitary (half) while the dotted line corresponds to a larger weight (twice).
Figure 9 provides the shapes of numerically approximated optimal epidemic curves for different beliefs about the expected arrival time of a vaccine (and a cure). In comparison to the benchmark case where no vaccine potential creation is taken into account, the arrival date τ of the vaccine is supposed to follow an exponential distribution with expectation 100, 250 or 500 days, while the optimization is occurring on the time interval [0, T ∧ τ. The epidemic curves are very similar when the anticipated time for vaccine availability is long enough, confirming the robustness of the approach and main findings. Whenever a vaccination procedure is supposed to arise rather early, the optimal strategy consists in postponing the mortality while keeping a stronger and longer lockdown effort, hoping for a quick vaccination solution. The sanitary burden is hereby smaller during the early stage of the epidemics but rises more strongly at the terminal phase.
Evolution of states with optimal controls (δ, 0, 0) – without any testing – for three different values of α, the discount rate that reflects the expected arrival time of a vaccine (and a cure). The plain line is the benchmark scenario discussed in Section 3.2; the dotted line corresponds to α = 1/500, dashed line corresponds to α = 1/250 and the mixed line corresponds to α = 1/100.
Our main overall observation is that the dynamics of the optimally controlled epidemic dynamics always present the same patterns and enlights the robustness of our findings. It divides into four successive phases in a similar way to the one described in the previous main scenario analysis. Besides, the effective reproduction number ℜt together with the optimal lockdown intervention policy also have the exact similar patterns. Of course small variations in the specifications still have an influence on the exact figures characterizing the balance between sanitary and economic outcomes, the total duration of the epidemic or the pressure on the ICU hospital system, but the overall characteristics remain identical. These observations confirm the robustness of our findings when dealing the the optimal control problem at hand, under specification uncertainty. The more specific dependence to the ICU care system capacity is studied in more details and discussed in the next section.
3.4 On the impact of additional ICU capacities
As discussed earlier, most countries have been urged to take public health measures to ‘flatten the curve’. An important motivation was to avoid that hospital systems get overwhelmed, and more specifically, social distancing was introduced to ensure that the occupation of ICUs with ventilators remain within the capacities of hospitals, dedicated to COVID-19 patients. Previously, we integrated an upper limit to ICU – denoted by Umax – and in this section, we discuss the impact of a potential increase of that capacity. Of course, increasing capacity does not simply mean getting more masks, beds, ventilators, or medicines for the serious cases. It also means diverting resources, training more staff to work in the ICU, and probably introduce telemedicine solutions for non-COVID-19 patients (who do not require hospitalization) to free up beds.
On Figure 5, we consider a change in Umax, where we assume that health authorities are able to increase ICU capacities by 50% over a 2 months period and remain at that level. We can imagine an increase in the number of ventilators or back-ups with staff coming from other regions that are spared by the epidemics, as was done in China or France for instance. Another possible interpretation is that the medical staff on the front line might be infected in early periods leading to an initial shortage of personal at the beginning followed by a return to normal within a few weeks. Here, we consider a regular (linear) increase of ICU capacities over two months, until reaching a new limit, 50% higher than the original one.
This scenario has a major impact on all quantities. First, we observe that the derived optimal lockdown strategy is able to let the ICU admission level bind the sustainable constraint over time, although it now grows dynamically with time. This corresponds to a longer second phase duration until the stable in prevalence and ICU capacity starts. The final number of dead people is overall smaller than in the original scenario, as the expansion of ICU capacity allows to treat more patients simultaneously. Therefore, the social distancing can be much weaker during the long ICU saturation phase, which also happens to be way shorter. Furthermore, ICU saturation duration is much shorter with a major positive impact on the economy from day 100 until day 400, as the lockdown intervention remains much weaker (after 200 days, the global level of social interactions is around 65% in comparison to less than 50% in our benchmark scenario). This reduces the socio-economic impact of the epidemic and hereby allows to reach a smaller mortality count at time T. The overall gain at terminal date is massive (even if here, we do not include a cost for increasing capacities). Nevertheless, the augmentation of ICU capacity implies also a higher daily dynamic death toll, so that an early non-anticipated vaccination solution at an intermediate date would induce a higher sanitary cost in comparison to the scenario without ICU capacity rising.
3.5 On the impact of detection resources
We now focus our analysis on the impact and optimization over the detection efforts λ1 and λ2 of infectious or immune individuals. As stated in Peto (2020), mass testing could end the epidemic rapidly, and we now try to emphasize the impact of light or mass testing upon the lockdown optimal intervention policy. The numerical results are presented in Appendix B for ease of readability and we present here the main outputs of our experiments.
Effort in virologic detection
We first quantify on Figure 10 the impact of introducing additional virologic detection resources with fixed success rate λ1. Namely, we consider a constant daily detection level λ1 of 1‰, 1%, 5% or 10%. Of course, the more effort we put into detecting infected people, the better the output. For instance we observe a smaller death toll, a weaker economic impact as well as less ICU saturation. Up to remaining numerical errors, moving the detection capacity λ1 from 1% to 10% has a strong impact on the overall outcomes of the epidemic: The epidemic size drops from 70% to 30%, the remaining number of susceptible individuals ST ≈ 70% corresponds to the herd immunity obtained with λ1 = 0. 1 and δ ≈ 0.25, the duration of ICU capacity saturation is divided by a factor larger than 2, and the overall sanitary and economic costs are divided by 2. The huge detection of infected individuals has a strong impact on the socio-economic cost for two reasons: First, detection of infected individuals allows to slow down the epidemic propagation. Second, detected infected individuals are later on identified as immune people, i.e. are in compartment R+, so that they are not impacted by social relations limitation, which reduces the overall economic cost. Finally, observe that massive testing has a major impact on most output (and must be massive to really impact, as suggested in Peto (2020)). Observe that ST remains rather large, but hopefully any epidemic restart is avoided by the detection effort λ1 which must not be released (the patterns of some figures close to terminal date T are due to the numerical approximation of the infinite horizon problem by a finite one).
Evolution of states with optimal control δ for three different values of λ1, or (δ∗, λ1, 0.) The plain line is the benchmark scenario discussed in Section 3.2, with λ1 = 0; and then three scenarios, with λ1 = 1% (dashed line), λ1 = 5% (dotted line) and λ1 = 10% (mixed line).
Another experiment presented in Figure 11 considers the case where the detection resources are rising linearly from 0% to 20% aver 700 days. In comparison to the corresponding case where detection resources of level λ1 = 10% are constantly available, the sanitary death toll induced by a rising λ1 is way higher. This confirms the intuition that intense detection resources are required from the beginning of the epidemic. More specifically, we now focus on the optimal dispatch over time of allocation resources: Figure 12 provides the optimal policy for the detection of infected whenever no lockdown measures are in place, while Figure 13 provides the optimal combination of detection policy and lockdown intervention. For both cases, the weight w prevalence has been set up equal to 1. As we can observe on both Figure 12 and Figure 13, and as stated clearly in Peto (2020), massive testing and detection strategies could have been used to end the epidemic rapidly. The optimal strategy consists in having extremely massive and early type-1 detection resources, in order to isolate infected individuals and break the contamination chains. Herd immunity is ensured while we keep the detection active so that ℜt remains below 1, allowing to epidemic size to remain very small, while everyone simply waits for for a vaccination cure.
Evolution of states with optimal control δ for two different patterns for λ1. The dotted line corresponds to the case λ1 = 1% and the mixed line is the increasing scenarios for 
(from 0% and 20% over two years).
Evolution of states with optimal control (0, λ1, 0), where the plain line corresponds to the benchmark scenario discussed in Section 3.1 (with no intervention δ = λ1 = λ2 = 0); and evolution with optimal control λ1 (when δ = λ2 = 0).
Evolution of states with optimal controls (δ, λ1, 0), where the plain line corresponds to the benchmark scenario (δ, 0, 0) discussed in Section 3.2 (plain line); and evolution with joint optimal controls (δ, λ1, 0) (dashed line).
With that strategy, the peak on ICU is 25% of the upper limit Umax, DT is almost null, with no real impact on the economy (there is still an impact since, in order to be safe, we still quarantine a significant proportion of people, above 10%). Note here that the optimal λ1, once we passed the peak, exceeds 25%, which is higher than the scenarios considered earlier. Observe also that massive detection tools induce a constant optimal lockdown level.
Effort in immunity detection
We now turn to the impact of detecting immune individuals, as found by the optimally controlled epidemic dynamics. First, we consider on Figure 14 the introduction of fixed type-2 detection strategy (or testing) and measure the impact on the optimal lockdown policy. Such detection resources allow to transfer individuals from compartment R− to R+. These additional resources have no significant impact on the optimal lockdown strategy and death toll dynamics of the epidemic. The only impact concerns the socio-economic cost of the pandemic, since identified recovered (and immune) individuals will be allowed to have a regular level of social interactions and won’t be quarantined.
Evolution of states with optimal control δ for three different values of λ2 given a fixed level of λ1 (here 1%), with λ2 = 0 (dotted line), λ2 = 1‰ (dashed line) and λ2 = 1% (mixed line), i.e. (δ∗, λ1, λ2).
On Figure 15, we look towards the optimal detection strategy (with wimmunity = 1) whenever no lockdown policy is in place, while Figure 16 focuses on the optimization on both lockdown intervention δ and immunity detection effort λ2. As expected, additional optimization over λ2 has few impact on the sanitary burden, but can be used to lower the economic cost. The optimal detection strategy consists in identifying recovered individuals as soon as possible, and requires a large amount of effort in the first part of the epidemic phase. As a consequence, in the presence of immunity detection policy, the level of social interactions during the last phase of the epidemic is significantly higher, thus reducing the induced economic cost.
Evolution of states with optimal controls (0, 0, λ2, where the plain line corresponds to the benchmark scenario discussed in Section 3.1 (with no intervention δ = λ1 = λ2 = 0); and evolution with optimal control λ2 (when δ = λ1 = 0).
Evolution of states with optimal controls (δ, 0, λ2), where the plain line corresponds to the benchmark scenario (δ, 0, 0) discussed in Section 3.2 (plain line); and evolution with joint optimal controls (δ, 0, λ2) (dashed line).
Finally, on Figure 17, we optimize over all possible control levers: lockdown measure, virologic detection effort λ1 as well as the immunity detection one λ2. We observe an optimal policy and induced epidemic dynamics rather similar to the one obtained in Figure 13, whenever no immunity detection mean was available. The optimal λ2 helps in reducing the overall economic costs by detecting immune people.
Evolution of states with optimal controls (δ, λ1, λ2), where the plain line corresponds to the benchmark scenario (δ, 0, 0) discussed in Section 3.2 (plain line); and evolution with joint optimal controls (δ, λ1, λ2) (dashed line).
4 Conclusion
The present work investigates, using an approach from Optimal Control theory, what could be optimal interventions for limiting the spread of a disease like the COVID-19 epidemic. We consider first the case where only the lockdown lever is operated, as was done in France when massive testing was still not an option. Then, in the other scenarios, optimization is done on both lockdown and detection capacity of infected and immune individuals.
The notion of ‘best scenario’ is of course very much impacted by the choice of objective function that is made. The optimization takes into account the trade-off between lowering the number of deaths and minimizing the economic and social costs. To our knowledge, taking into consideration these diverse aspects into the objective function has not been studied much in the recent literature (see e.g. Acemoglu et al. (2020)), although the literature on the topic is growing rapidly.
Another originality of our approach is the consideration of a strict constraint on the ICU capacity level dedicated to COVID-19 patients. We observe that in the solutions, the lockdown and detection controls are set so that the occupation of ICU remains a high as possible under Umax (due to the economic and social costs). It is really this constraint that shapes the decisions leading to a curve that is ‘flattened’ sufficiently to ensure the sustainability of the health system. This is particularly visible in the scenario where Umax is increased of 50% within 2 months at the beginning of the epidemic: we see a relaxation of the lockdown that is permitted by the increased capacity of ICUs.
The best scenarios that we obtain are structured into 4 phases: 1) quick and strong measures to recover the control of the epidemic, 2) relaxation of the epidemic once under control to reach the fluxes imposed by the sustainability of the health care system, 3) the fluxes obtained after the period 2 are kept as long as possible to flatten the curve in order to avoid overwhelming the ICUs, 4) once a herd immunity is reached, the lockdown and testing controls can be lowered.
Without being an explicit target in the control problem, the evolution of the effective reproduction number ℜt is extremely interesting, and consistent with existing epidemiological literature. Again, without monitoring that indicator, we obtain in all the optimal strategies, ℜt becomes controlled below 1 after a few days (after the phase 1, and at the price of an important initial lockdown), and remains just below that critical value2. It is allowed to reach and remain stable at level 1 in order to stabilize the ICU occupancy at their sustainable level Umax.
Of course, the model considered here is schematic because we wanted : 1) to put the methodology in evidence, 2) to explore easily the different scenarios. Generalizations can be carried in a quite straightforward manner. First, the major drawback of our approach is the necessity to fully observe some underlying latent variables and parameters of the epidemic dynamics, such as the prevalence of the virus. But we observed that the shape of the optimal control is quite robust to a variation of the initial prevalence 
, and more generally to variations upon lots of parameters in the model. An important issue about that COVID-19 pandemic was the poor quality of data in the beginning, especially when trying to compare among countries to get robust estimators of various quantities regarding the disease. In our model, we tuned parameters using values from existing literature, and most of the results we obtained were actually extremely robust to moderate changes of those values.
Second, most individuals will not experience severe symptoms whenever contaminated by the virus. Such analysis calls for the use of differential quarantine strategies based on an estimation of the own risk of each individual. In our approach we use immunity detection procedure for a similar purpose, but less accurate estimates of the risk of each individuals are also available depending on their current and past health characteristics. As discussed in Evgeniou et al. (2020), the key ingredient for considering and using a differential lockdown strategy is a proper handling of the risk of bad specification of the group associated to some individuals. A dynamic optimisation over both the dynamic lockdown intervention together with the risk level associated to each misspecification is left for further research. Similarly the risk of wrong identification of infected or immune individuals using parameters λ1 and λ2 should be also taken into account in the design of a proper and efficient detection strategy. Also, in view of more practical applications for health policy guidance, a vector based version of SIDUHR+/− should probably be derived, as in Acemoglu et al. (2020) for instance. Categories can be splited depending on the age of individuals. Hence, St will then be 
, with children, adults and seniors, for instance. Most of the equations of the dynamics become multivariate, and parameters will be vectors, except β that becomes a WAIFW (or Who Acquires Infection From Whom) matrix, as in Wallinga et al. (2006) or Mossong et al. (2008). The main interest would be to have target and age-specific controls, with testing and lockdown that can be per age group. The difficult component in this multivariate extension is on the objective function, where the economic component should probably be more related to well-being cost of being quarantined, especially for people who do not belong to labor groups.
Finally let us notice that in this paper, as always in epidemic modelling, an important feature is the −/+ problem, with undetected individuals. The huge impact of infected detection resources λ1 in our numerical experiments emphasize the critical role of tracing and testing resources upon the dynamic of the epidemic. As suggested in Eames and Keeling (2003); Clémençon et al. (2008); House and Keeling (2010); Kiss et al. (2013) (among many others), tracing is clearly a natural tool that should be optimized, to lower the socio-economic cost of the disease without endangering the entire population (and increase the sanitary cost of the pandemic). This paves the way for further research topics.
Data Availability
We have only used data from other research reports or publications or found on the website of the WHO.
A Parameters of the epidemic dynamics
First, let us notice, as mentioned in the main text, that there is an underlying individual based model for the system (1), in which the rates have a probabilistic interpretation, and can be interpreted as inverse of mean times (e.g. Ball et al. (2019)), possibly with thinning when the corresponding events only occur for a fraction of the population. Based on these remarks, we can compute the rates in our model with the parameters provided in the French COVID-19 literature (see Domenico et al. (2020); Salje et al. (2020)).
Obtaining the transition rates from the sojourn times and transition probabilities
First, we introduce some variables on which the mechanistic construction of our rates will depend:
pa : conditioned on being infected, the probability of having light symptoms or no symptoms;
pH : conditioned on being mild/severely ill, the probability of needing hospitalization;
pU : conditioned on needing hospitalization, the probability of needing ICU;
pd : conditioned on being admitted to ICU, the probability of dying, and for the durations:
: if asymptomatic, number of days until recovery;- : if symptomatic, number of days until recovery without hospital;
NH : (if severe symptomatic) number of days until hospitalization;
NHU : if in H, number of days until ICU ;
NH R : if hospitalized but not in ICU, the number of days until recovery;
NUD = 10: if in ICU, number of days until death;
NUR = 20: if in ICU, number of days until recovery.
These variables can be measured quite easily on the data. Then, once these parameters are chosen, we can propose the following transition rates for our model:
γI R, the daily individual transition rate from I to R, is assumed to be of the form:
where on the right hand side, the first term is associated to mild symptomatic while the second term is associated to severe ones.γI H, the daily rate from I to H, is assumed to be
γHU, the daily rate from H to U, is given by
γH R, the daily rate from H to R, is given by
For the transition rates from U to R or D, the nonlinearity induced by the constraint on the ICU capacity Umax implies that it is easier to give these rates at the population level rather than at the individual level. For the transition from U to R, the individual rate is denoted by γUR(U) and the population rate is then γUR(U) · U. We assume here that:
This means that when the ICU capacity Umax is reached, new patients can not be taken in charge any more and since they correspond to individuals with severe forms of the disease, they can not recover until the ICU occupation is lowered.
As for the transition rate from U to R, the transition rate from U to D is nonlinear. We denote by γUD(U) the daily death rate of an individual needing ICU, and by γUD(U) × U the transition at the population level, that we assume is of the form:
The hypothesis is that when ICU gets saturated, the patients who are not taken in charge experience an extra death rate.
Parameter values
For the infection parameter β, we fix the value of ℛ0 and then compute the corresponding value of β using (2). Salje et al. (2020) use ℛ0 = 3.3 (in absence of lockdown intervention and of testing policy), which we also recover from the early French dynamics (World Health Organization (2020)) using the methods described in Trapman et al. (2016) (not shown).
The other transition rates appearing on Fig. 1 can be obtained once the probabilities and durations detailed above are fixed. For this, we choose our parameters on the basis of the works by Domenico et al. (2020); Salje et al. (2020) for the French COVID-19 epidemics. We emphasize that despite of this choice, the methodology developed here is general and can apply to any country of disease.
Parameters. (a): parameters obtained from Di Domenico et al. Domenico et al. (2020). (b): parameters obtained from Salje et al. Salje et al. (2020). (c): parameters used in the benchmark scenario in this paper
The line (a) in the Table 3 can be obtained as follows: the model of Domenico et al. (2020) for Ile-de-France is structured into 3 age-classes: children (25%), adults (60%) and seniors (15%). Using the age-specific transition probabilities given in this reference and averaging over the age classes, we obtain the values announced:
However, it appears in our simulations that the probability pa = 0. 52 to be asymptomatic or only with light symptoms does not provide realistic orders for the proportion of dead individuals. In Salje et al. (2020), they estimate the proportion of being hospitalized when infected at 2.6%. This proportion corresponds to 1 − pa) pH in our model, yielding a probability pa = 0. 85 when we use the estimate pH = 0. 17 of Domenico et al. (2020).
In France, at the beginning of the lockdown on March 15th 2020, there was roughly 12,000 beds in ICUs, which provides an estimate for the value of Umax. The additional mortality rate for individuals in ICUs when Ut > Umax is chosen 20 times more than the usual death rate in (15). This factor is calibrated from the observation by Salje et al. (2020) that the mean time to death in hospitals is bi-modal, with a group dying quickly after 0.67 days on average and a second group who die after longer time periods of mean 13.2 days. Extrapolating we assumed that ‘urgent’ cases die 20 times faster than ‘usual cases’.
For the value of 
, (Flaxman et al., 2020, Table 1) report a total number of infected of 3%, with a 95% credible interval [1.1%;7.8%], as of 28th March 2020 (which is two week after our t = 0). Because of the exponential increase, a crude estimate would be below 1% two weeks before. The number of reported cases in France on March 15 was around 5,000. According to Sachdeva and Sheth (2020), when Italy had 5,000 reported cases, an estimation of 60,000 cases was suggested, which would imply a 
close to 1‰. In our benchmark scenario, we kept that 5‰ value for 
.
To check the validity of our parameters, we simulated our model in absence of intervention (δ = 0, λ1 0 and λ2 ≡ 0) and compare the results with the prediction of Roux et al. (2020). Because in the latter reference they do not model the ICU saturation, we also removed the threshold Umax and the associated additional mortality (which amounts to considering Umax = +∞). Doing this we obtain similar results as those of Roux et al. (2020).
B Sensitivities to the model specifications
In this appendix, we provide sensitivity graphs, where parameters are changed, to assess the robustness of our conclusions. Do not forget that numerical approximations errors can affect a too precise interpretation of the following numerical illustrations.
We consider the following benchmark scenario. The parameters for the dynamics are specified in Table 1. In the control problem, the benchmark situation is the one described in Section 3.2. To be precise, for the numerical implementation, we use the cost function is the one defined in (25) with the following choice of parameters: α = 0, wsanitary = 105, wecon = 1, wprevalence = 1, wimmunity = 1, wICU = 5. 104 and T = 700. Hence the optimal control problem for which we compute an approximate solution numerically is:
with 
the set of measurable functions from [0, T] to [0, 1].
B.1 Impact of the initial virus prevalence 
B.2 Impact of the basic reproductive number ℛ0
B.3 Impact of balancing sanitary and socio-economic costs
B.4 Impact of the vaccination arrival date anticipation α
B.5 Impact of additional constant effort in virologic detection λ1
Here, we add virologic effort for a constant value of λ1 and we compare the result of the optimization over δ for several values of λ1. The optimal control problem for which we compute an approximate solution numerically is:
with 
the set of measurable functions from [0, T] to [0, 1].
B.6 Impact of additional rising effort in virologic detection λ1
Here, we add virologic effort for a value of λ1 which increases linearly in time, 
. The optimal control problem for which we compute an approximate solution numerically is:
with 
the set of measurable functions from [0, T] to [0, 1].
B.7 Optimizing over effort in virologic detection without lockdown intervention
Here, we compare the situation without intervention at all and the situation with intervention through λ1. The optimal control problem for which we compute an approximate solution numerically is:
with 
the set of measurable functions from [0, T] to [0, 1].
B.8 Optimizing over both lockdown intervention δ and effort in virologic detection λ1
Here, we compare the situation with optimal lockdown policy versus the situation in which we optimize over both lockdown and virologic effort. The latter corresponds to the following optimal control problem, for which we compute an approximate solution numerically:
with 
the set of measurable functions from [0, T] to [0, 1].
B.9 Impact of additional effort in immunity detection λ2
Here, we add virologic effort for a constant value of λ2 and we compare the result of the optimization over δ for several values of λ2. The optimal control problem for which we compute an approximate solution numerically is:
with 
the set of measurable functions from [0, T] to [0, 1].
B.10 Optimizing over effort in immunity detection λ2 without lockdown intervention
Here, we compare the situation without intervention at all and the situation with intervention through λ2. The optimal control problem for which we compute an approximate solution numerically is:
with 
the set of measurable functions from [0, T] to [0, 1].
B.11 Optimizing over both lockdown intervention δ and effort in immunity detection λ2
Here, we compare the situation with optimal lockdown policy versus the situation in which we optimize over both lockdown and immunity detection. The latter corresponds to the following optimal control problem, for which we compute an approximate solution numerically:
with 
the set of measurable functions from [0, T] to [0, 1].
B.12 Optimizing over lockdown intervention δ, and efforts in virologic detection λ1 and immunity detection λ2
Here, we compare the optimization on δ only versus the optimization (24) over (δ, λ1, λ2).
C Numerical resolution of the optimal control problem
We discuss in this section the numerical resolution of the optimal control problem (12) of interest.
The original problem has infinite horizon, but for numerical purpose, we focus on the equivalent version over a sufficiently large time horizon T. The proximity between both optimal control problems is well established in the literature, see e.g. Section 4.1 in Elie (2008), while we observe empirically very few impact by the choice of maturity T, as soon as T is chosen large enough in order for the epidemic phase to be over (i.e. such that the proportion of infected people is small enough at time T). With respect to the random time of vaccination τ, such approximation corresponds to optimizing over the time interval [0, τ ∧ T] instead of [0, τ].
Similarly, the problem (12) is an optimal control problem with sate constraint on the ICU capacity, and we choose to replace for ease of numerical tractability the state constraint by a penalization cost with a sufficiently large weight wICU > 0. Hence, we focus numerically on an approximate version of the original optimal control, which is given by:
where the approximate objective function 
is given by
C.1 Optimality condition by Pontryagin’s maximum principle
To alleviate the notation and use more standrad notations from the litterature, the problem can be recast in the following form, where a denotes the control and X the state: Minimize
subject to
We interpret 
3 and we take
where
Moreover, we take
where we denoted ρUR(u) = γUR(u)u and ρUD(u) = γUD(u)u in order to alleviate the notation.
Let H be the Hamiltonian defined by
Pontryagin’s maximum principle leads, at least informally, to the following optimality condition: if â = (ât)t is optimal, then for every t ∈ [0, T],
where 
solve the forward-backward ODE system:
The backward equation can be written as follows (dropping the hats to alleviate the notation):
C.2 Numerical algorithm
The numerical method is an iterative procedure to compute the optimal control 
= 
. Starting from an initial guess a(0), at iteration k ≥ 0, in order to compute a(k +1), we first compute the solution (X(k), Y(k)) to the forward-backward ODE system corresponding to (26) but with control a(k) instead of â, i.e.,
We then set
where θ(k) > 0 is a step size (which may depend on the iteration in order to adjust the convergence rate).
Finally, we define, for each component i of the control,
where π[0,1] denotes the projection on the interval 0, 1.
If the optimization is to be performed over only one component of the control (e.g. over δ only), then the updates are done only for this component instead of the whole vector of controls.
Footnotes
The authors thank Hélène Guérin, Olivier l’Haridon, Olivier Pietquin and Gabriel Turinici for stimulating discussions and feed-backs on early versions of that work, as well as participants of the Machine Learning Paris seminar. R.E. and V.C.T. are partly supported by the Bézout Labex, funded by ANR, reference ANR-10-LABX-58. V.C.T. is also part of ANR Cadence (ANR-16-CE32–0007), by the Chair “Modélisation Mathématique et Biodiversité” of Veolia Environnement-Ecole Polytechnique-Museum National d’Histoire Naturelle-Fondation X.
↵1. In all other figures of this article, but that one, we consider a 700 day horizon. Here, the epidemic phase is over within two months so we shorten the time frame on Figure 2 for ease of readability.
↵2. On some graphs, we have a long term value of ℜtabove 1, but it is obtained whenever close to the terminal date T and due to the numerical approximation of the infinite horizon control problem by a finite horizon one.
References
