Modeling COVID-19 Nonpharmaceutical Interventions: Exploring periodic NPI strategies ==================================================================================== * Raffaele Vardavas * Pedro Nascimento de Lima * Lawrence Baker ## Abstract We developed a COVID-19 transmission model used as part of RAND’s web-based COVID-19 decision support tool that compares the effects of nonpharmaceutical public health interventions (NPIs) on health and economic outcomes. An interdisciplinary approach informed the selection and use of multiple NPIs, combining quantitative modeling of the health/economic impacts of interventions with qualitative assessments of other important considerations (e.g., cost, ease of implementation, equity). This paper provides further details of our model, describes extensions, presents sensitivity analyses, and analyzes strategies that periodically switch between a base NPI level and a higher NPI level. We find that a periodic strategy, if implemented with perfect compliance, could have produced similar health outcomes as static strategies but might have produced better outcomes when considering other measures of social welfare. Our findings suggest that there are opportunities to shape the tradeoffs between economic and health outcomes by carefully evaluating a more comprehensive range of reopening policies. ## 1 Introduction Coronavirus disease 2019 (COVID-19) is unprecedented in terms of scale and speed, affecting millions worldwide. Until recently, vaccines and effective treatments for COVID-19 were unavailable. National leaders have had to take extraordinary measures to mitigate the virus’s spread and prevent health care systems from being overwhelmed. Policymakers have implemented a range of nonpharmaceutical public health interventions (NPIs). These interventions include partial closings (e.g., schools and non-essential businesses, prohibiting large gatherings, quarantining the most vulnerable) and complete lockdown (e.g., placing all residents under stay-at-home orders). The goal of NPIs is to delay and reduce the peak number of cases per day, reduce pressure on health services, and allow time for vaccines to be distributions [1]. If NPIs are relaxed too soon a new wave of infections may occcur. However, NPIs have wide-ranging effects on the health, economy, and social well-being of populations, which has led to growing pandemic fatigue and a decline in adherence to NPIs since they were first initiated [2, 3]. Decision-makers are faced with tough decisions, such as how to sequence, relax, and possibly reinstate mitigation measures. Exacerbating these decisions are significant uncertainties, including new variants and behavioral responses to extended interventions. Mathematical and simulation models of COVID-19 transmission dynamics are invaluable tools to help decision-makers forecast and compare intervention outcomes, predict the timing of peaks in cases and deaths, medical supply needs, and if and when we should expect additional waves. They enable projection and comparison of population-level outcomes over hypothetical scenarios. Model outcomes include the incidence and prevalence of the infection over time and for different population groups. The hypothetical scenarios can consist of the impact of varying pharmaceutical and nonphar-maceutical public health interventions, distributing vaccines, and the emergence of new strains. We developed a COVID-19 transmission Population-Based Model (PBM) used as part of a web-based COVID-19 decision support tool that compares the effects of different nonpharmaceutical public health interventions (NPIs) on health and economic outcomes. An interdisciplinary approach informed the selection NPI portfolios, combining quantitative modeling of the health/economic impacts with qualitative assessments of cost, ease of implementation and equity. An in-depth description of our approach was previously published as a RAND report describing how the PBM, the economic model, and a systematic assessment of NPIs informed the web-tool [4]. We expanded our original model [4] to account additional uncertainties and consider an expanded set of NPI strategies. In this paper, we consider periodic NPI strategies. Recent research has demonstrated that high-frequency periodic NPIs [5] have the potential to mitigate COVID-19 resurgences while providing more predictability and alleviating the damaging effects on economic activity and social well-being. We use our updated model to explore if a periodic strategy could have provided benefits compared to fixed strategies that would keep *R**t* close to one. We find that a periodic strategy can dominated fixed strategies, improving both health and days spent under restrictions. The paper is structured as follows. First, we provide an overview of our model structure. Then, we briefly analyze a set of illustrative scenarios, including periodic and fixed strategies, identifying if the periodic strategies used by other modelers modelers [5] produce similar results in our model. Finally, we provide detailed information on our mathematical model and present sensitivity analyses. ## 2 Model Overview Theory-based epidemiological models use a theoretical understanding of biological and social processes to represent a disease’s clinical and epidemiological course. The most typical model considers the population in four different disease states: susceptible, exposed, infected, and Removed (SEIR). Our PBM incorporates several extensions to the SEIR model of disease transmission. It is formulated deterministically by coupled ODEs and integrated numerically by solvers for stiff problems [6–9]. We extend the SEIR framework to better describe COVID-19 transmission by adding additional disease states and considering population strata based on age and chronic conditions. The PBM models the effects of different NPIs on health outcomes, from partial closings to complete lockdown. Unlike many other COVID-19 models, we simulate the impact of NPIs on different mixing modes (such as home, school, and work) separately, allowing us to model various interventions flexibly. Our PBM also includes population strata and specify mixing pattern heterogeneities across the population strata and for each mode. These heterogeneities included in our model allow us to set the NPI more specifically, with mixing rates reduced deferentially by mixing mode. Our model is designed to help policymakers understand the effects of NPIs, weigh tradeoffs among them, make decisions about which NPIs to implement, and estimate how long-term interventions should be enforced to control the virus. Figure 1 shows a simplified illustration of the disease states included in the first basic version of our PBM. The model includes a pre-symptomatic highly infectious state, which leads to either an asymptomatic state or a state with mild symptoms, a fraction of which continue to severe disease. Most of those who develop severe symp-toms are hospitalized. Non-hospitalized severely-symptomatic either recover or die. ![Figure 1:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/03/21/2021.02.28.21252642/F1.medium.gif) [Figure 1:](http://medrxiv.org/content/early/2021/03/21/2021.02.28.21252642/F1) Figure 1: COVID-19 PBM disease states The hospitalized state includes compartments for both the main hospital and the ICU, where individuals are admitted if they develop critical symptoms. Capacities can be set for the hospital and ICU beyond which no additional patients can be added. At each of the infectious states, individuals can be tested for COVID-19. Each compartment is composed of ten population strata, five age groups, and two health states (those with and without at least one chronic condition). These strata allow the model to simulate how the disease impacts different population groups, including differences in population size, mixing mortality rate, and the proportion who are asymptomatic. Disease progression rates are based on figures given in the literature and are sampled from distributions if uncertain. The force of infection (the rate that susceptible individuals become exposed) is characterized by how many infectious people are in each state, the transmissibility of each state, and levels of mixing. We estimate transmissibilities for each state based on biological and social factors. For instance, viral loads are highest early in the disease [10], so these states have higher biological transmissibility. We assume that those who receive positive tests or exhibit symptoms have lower social transmissibility because they take measures to limit the exposure of others. The total transmissibility of a state is the product of the biological and social transmissibilities, and the population-weighted sum of transmission is proportional to the number of new infections. In our model, NPIs are portfolios of restrictions mandated at the state level, as described in table 1. The set of NPIs levels used by each state is characterized by a discrete set of intervention levels ranging from 1 (no intervention) to 6 (close schools, bars, restaurants, and nonessential businesses; and issue a shelter-in-place order for everyone but essential workers). Each intervention level is associated with mixing matrices that describe how strata interact with each other in six different settings: household, work, school, commercial, recreation, and other. Interventions are modeled as changing the level of mixing which occurs in each of these settings. For instance, closing schools reduces school and work mixing but increases home mixing. Given the specified model structure, the NPI time-series, and the mixing matrices, we calibrate our model for each state using deaths time series. Appendix A provides a detailed description of the mathematical formulation of our model. View this table: [Table 1:](http://medrxiv.org/content/early/2021/03/21/2021.02.28.21252642/T1) Table 1: Nonpharmaceutical intervention levels. ## 3 Exploring Periodic NPI Strategies In this section, we present an illustrative retrospective analysis of policies that can be tested with the model, using the state of California as an example. These analyses and findings do not constitute predictions. Yet, they illustrate that alternative plausible NPI strategies could have produced improved outcomes during 2020, in the absence of vaccines. The purpose of this analysis is two-fold. First, it demonstrates how our model can be used to trace many-objective trade-off curves to support the analysis of reopening strategies. Second, this analysis demonstrates that a periodic switching of NPIs might have placed society in a better position in these trade-off curves - that is, it could have represented a pareto-improvement. This section explores two types of strategies that could have been followed to manage NPIs in the year 2020. The first set of strategies are “fixed” NPI levels. This type of strategy holds the NPI level constant over time. Although this strategy has not been followed in California explicitly, the NPI mandates imposed in California are best approximated in our model by setting the NPI level to three. This NPI level was stable between July of 2020 through the end of the year, and this represents our baseline scenario and is the scenario used to calibrate our model. Fixed NPIs are not, however, the only way to control the pandemic. Alternatively, one could use periodic strategies to curb transmission [5]. A periodic NPI can represent a strategy wherein society goes into more severe periods of NPIs then relaxes to lower levels of stringency. This strategy’s rationale is that those newly infected during the relaxed periods would take a few days before becoming infectious themselves. The enforcement of stringent NPIs would then limit the time and possibilities for the virus to spread further from these infectious individuals during the time before they either recover or are hospitalized. This periodical switching would systematically reduce transmissions and force the dynamics of the epidemics to be controlled by the strategy’s frequency. In essence, a periodic strategy uses the natural timescales of disease progression and infectivity to induce a synchronization phase that helps align the times when people are more likely to be infected together, allowing for the social distancing NPIs to be more effective. An example of a similar strategy includes schools adopting a hybrid learning model wherein students go to school every other week. Similarly, restaurants could open for indoor dining periodically. In absence of vaccines, such policies may be desirable as they would provide stability, regularity, and increased predictability for businesses to plan against. The policies could, in principle, help suppress the transmission of the virus and simultaneously reduce uncertainties in economic activity. Figure 2 illustrates the dynamics of periodic and fixed NPI strategies. The fixed NPI strategies represented in the figure suggest that under the NPI level three, *R**t* closely followed one and increased towards the end of December, driven by our model’s seasonal effect. Because *R**t* was close to one in the model, a departure from the current NPI level would be expected to produce a significant departure from the *R**t* = 1. Therefore, a policy that reopens the state (F-1) would be expected to produce a spike in prevalence and subsequently in the number of deaths, and a more stringent, constant policy (F-5) would be expected to reduce the number of deaths. ![Figure 2:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/03/21/2021.02.28.21252642/F2.medium.gif) [Figure 2:](http://medrxiv.org/content/early/2021/03/21/2021.02.28.21252642/F2) Figure 2: Model Dynamics with Fixed and Periodic NPIs. Fixed NPI strategies are coded with an “F” followed by the intervention level included in the NPI. Periodic NPI strategies are coded with a “P” followed by the maximum NPI level in the periodic strategy and the period in days. In that case, P-3-14 means that the state will switch between the NPI level 3 and 1 every 14 days. As figure 2 shows, the periodic strategy P-5-14 switches between NPI levels 1 and 5 every two weeks. This switching causes *R**t* to oscillate such that prevalence does not increase unchecked. As a consequence, the number of deaths is controlled. The choice of two weeks is based on the typical timescale describing the disease progression for the majority of infected people. However, other choices for the periodicity could be explored. When judging alternative strategies, policymakers often have to weigh multiple criteria to make decisions, so one needs to translate model outcomes to meaningful criteria. One criterion could be the number of days of school closures, which has been an important concern during the COVID-19 pandemic. However, the number of days of school closures does not distinguish scenarios in which non-essential businesses are closed for long periods, so other proxies for welfare loss are needed. One approach could be to use weights for each NPI level, such that those weights are proportional to the marginal daily welfare loss induced by each NPI level. If one defines those weights such that one day under lockdown is equivalent to one, and one day under no restrictions is set to zero, one can compute a proxy to social welfare that can be used to judge alternative strategies. Our weighted lockdown days metric corresponds to this criterion. There are other plausible ways to compute NPI costs. NPIs arguably induce income loss. Our third metric uses the income loss under each NPI level estimated by a general equilibrium model [11] to account for the economic consequences of NPI restrictions. Although all these proxies are imperfect measures of social welfare loss induced by NPIs, our conclusions do not rely on their precision, but on the assumption that NPI costs are increasing in the level of restriction. This structural assumption allows us to illuminate trade-offs and reveal pareto-dominated strategies that rely on the structure of the epidemiological model1. Figure 3 demonstrates that using a small set of alternative measures can support those decisions and reveal pareto-dominated strategies. Strategy F-3, our baseline strategy, is pareto-dominated by a wide range of strategies that oscillate between the NPI level of 5 and 1, using many periods, which is in line with prior research [5]. ![Figure 3:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/03/21/2021.02.28.21252642/F3.medium.gif) [Figure 3:](http://medrxiv.org/content/early/2021/03/21/2021.02.28.21252642/F3) Figure 3: Tradeoff surface implied by alternative policies. The vertical axis presents the number of Deaths / 100k at the end of the simulation run (Feb. 2021) in California, and the horizontal axis contains several proxies that represent alternative criteria to evaluate the costs of NPIs. Across all these criteria, we find that periodic NPIs tend to dominate fixed NPIs. The plot demonstrates that strategies with fixed NPIs generally are dominated by periodic NPI strategies. However, there are limitations to our analysis. First, we do not consider practicality: these periodic strategies might be regarded as unfeasible, impractical, or undesirable by policymakers and the public. This consideration is particularly important because the strength of the periodic NPIs relies on the ability to abruptly reduced mixing, which can only be achieved with a high level of compliance. Further, people are may to shift their mixing to the open periods reducing or even canceling the mitigation effects on transmission intended by the periodic NPI policy. Nevertheless, we use them in this paper as an example to illustrate that alternative NPIs strategies, if implemented with high levels of compliance, might pareto-dominate fixed strategies and might shift the trade-off surface among health and economic/social outcomes. ## 4 Conclusion The scenarios presented in this paper serve the purpose to illustrate that alternative policies using periodic NPIs to manage the COVID-19 pandemic might have produced the same health outcomes while allowing essential activities, such as in-person education, to have happened in a controlled manner. While we do not advocate for any particular strategy, this brief analysis demonstrated that alternative NPI strategies have the potential to shift the trade-off curves among the relevant outcomes. Including social welfare loss measures induced by NPIs in analyses seeking to inform COVID-19 reopening decisions is essential. Only including health outcomes in those analyses leaves the task of weighing other concerns to the policymaker, who may or may not be able to do so consistently. Metrics of welfare loss induced by NPIs can be either derived directly from the model outcomes (e.g., days of school closures), or use a weighted sum based on the NPIs stringency level, potentially using economic models in our prior work[4]. Even if the analysis estimates are not precise or could become less precise with time, they can still be useful. As figure 3 demonstrated, if one ignores all the horizontal axes under the argument that those estimates might be imprecise, policymakers might not be properly informed that alternative policies dominate some policies. This statement and the pattern seen in the trade-off curves do not rely on the precision of economic estimates but on the theory-based epidemiological model structure. The trade-off curves we present also should not be seen as static. Many other factors that have been held constant in our analysis might also shift this curve. Widespread adoption of high-quality masks, for example, would shift every point inwards, making society systematically better off. The emergence of new, more transmissible variant strains can shift the curve outwards. A more stringent strategy to eliminate community spread and prevent re-seeding (such as New Zealand’s strategy) can remove the health-economic trade-offs curve completely if successfully implemented. Adaptation measures to prevent transmission within schools would also shift this curve, strengthening periodic strategies even more attractive to allow in-person education. Moreover, the introduction of vaccines also shapes this trade-off curve over time. As vaccination roll-out advances, the marginal benefit of an additional day under stringent NPIs will decrease. Accounting for the uncertainties mentioned before and the vaccination dynamics will be essential to guide society to a new normal through a robust reopening plan. ## Data Availability All modeling data is available upon request. ## Contributions Raffaele Vardavas (Ph.D.) led the effort in conceptualizing and formulating the model structure and led the overall modeling effort. Pedro Nascimento de Lima (MSc) led the model implementation and model analysis effort. Lawrence Baker (MSc) led the effort in informing the model with data and parameter values found in the literature. All authors contributed equally to this research work and to this article. ## Disclaimer This report provides additional technical details of a peer-reviewed RAND report describing RAND’s COVID-19 Decision Support Tool for State and Local Policymakers. This report uses the simulation model to explore the dynamics of a retrospective counterfactual COVID-19 transmission mitigation policy for 2020 based on intermittent applications of NPIs. The results and conclusions drawn on how effective this policy design could have been if implemented based on our simulation model have not been peer-reviewed. Conclusions drawn from this working paper do not necessarily represent the opinions of the RAND Corporation. ## A The Mathematical Formulation of the Model ### A.1 Model disease states and progression Individuals in our population are divided into fourteen key compartments listed in Table 2. We assume that individuals in the *P* and *I**A* compartments are fully asymptomatic and thus are unaware of being infectious. In our model, individuals in *I**Sm* have mild symptoms, including a dry cough and a fever, while those in *I**Ss* are assumed to have severe symptoms that include shortness of breath in addition to a dry cough and a fever. The sum of the population in all of the states gives the total population *N*. In our model we assume that each state variable gives the proportion of the population belonging to that state. Therefore, instead of tracking the dynamics of each compartment’s population sizes, we track the population densities. We express this as Σ*X* *X* (*t*) = *N* = 1, where *X* ∈ {*S, E, X**I*, *R**A*, *R**S*, *D*} labels the population compartments and *X**I* ∈ {*P, I**Sm*, *I**A*, *I**Ss*, *Y**Sm*, *Y**A*, *Y**Ss*, *H, H**ICU*} labels the subset of compartments that are infectious. View this table: [Table 2:](http://medrxiv.org/content/early/2021/03/21/2021.02.28.21252642/T2) Table 2: Disease states included in the model. The dependence on time *t* is implicitly assumed. In the PBM, the susceptible that get infected enter state *E* at a rate known as the force of infection *λ* described in the section A.5. As for all the transition rates in our PBM, *λ* is specified as a per-person transition probability per unit time. Figure 4 illustrates disease progression in the early stages of COVID-19 infection. They progress to the presymptomatic infectious state *P* at a rate *v*. As an individual-level interpretation of this transition, the mean duration, also known as the mean dwelling or sojourn time that newly infected individuals stay in disease state *E* is given by *v*−1. Those in the presymptomatic infec-tious state *P* either remain asymptomatic and transition to state *I**A* at a rate *γ**A*, or develop mild symptoms and transition to state *I**Sm* at a rate *γ**S*. At the individual level, the mean duration that infected individuals stay in disease state *P* is given by [*γ**S* + *γ**A*]−1, and the probability of developing mild symptoms is given by *γ**S* · [*γ**S* + *γ**A*]−1. We assume that testing the presymptomatic for COVID-19 results in a false negative outcome. Consequently, all those who are presymptomatic are unaware of their infected and infectious state. The formulation of the first three ODEs describing our PBM are ![Figure 4:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/03/21/2021.02.28.21252642/F4.medium.gif) [Figure 4:](http://medrxiv.org/content/early/2021/03/21/2021.02.28.21252642/F4) Figure 4: Initial disease progression stages ![Formula][1] ![Formula][2] ![Formula][3] The asymptomatic and unaware of having been infected, *I**A* progress to the recovered state *R* at rate *ξ**A*. The asymptomatic are assumed to stay infectious until they recover. Figure 5 illustrates the disease progression of the asymptomatic. A proportion *ζ**A*(*t*) · [*ζ**A*(*t*) + *ξ**A*]−1 of the asymptomatic are diagnosed with having COVID-19. This proportion of individuals transition to state *Y**A*. ![Figure 5:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/03/21/2021.02.28.21252642/F5.medium.gif) [Figure 5:](http://medrxiv.org/content/early/2021/03/21/2021.02.28.21252642/F5) Figure 5: As described in section A.3, the testing rate *ζ**A*(*t*) is not constant but implicitly depends on time, due to dynamic testing policies. These individuals most likely get tested because they suspect or are informed by health-care workers engaged in contact-tracing to have recently been in contact with someone diagnosed with COVID-19. Those who enter disease stage *Y**A* progress to the recovered state *R* at rate ![Graphic][4]. The transition rate ![Graphic][5] is faster than *ξ**A* and accounts for the elapsed duration of mild symptoms prior to being diagnosed. These diag-nosed asymptomatic are aware of having been infected and of being infectious. Therefore, they are assumed to engage in increased social distancing behavior described in the section A.3 on the disease transmission model. The ODEs describing the asymptomatic disease progression are ![Formula][6] ![Formula][7] Figure 6 illustrates the disease progression of the symptomatic. The non-diagnosed mildly symptomatic, *I**Sm* either progress to the recovered state *R* at a rate *ξ**m* or develop severe symptoms and progress to state *I**Ss* at a rate *ν*. Some are tested at a rate *ζ**S*(*t*) and diagnosed with having COVID-19. These transition to state *Y**Sm* and become aware of their infection state. As before, *ζ**S*(*t*) is not a constant but is assumed to implicitly depends on time. Our model assumes that those mildly symp-tomatic that get diagnosed will also progress to the recovered state *R* at a rate ![Graphic][8] or develop severe symptoms and transition to state *Y**Ss* at rates *ν**. These rates are respectively faster than *ξ* and *ν* to account for the elapsed duration of mild symptoms prior to being diagnosed. The ODEs describing the non-hospitalized with mild symptoms are ![Figure 6:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/03/21/2021.02.28.21252642/F6.medium.gif) [Figure 6:](http://medrxiv.org/content/early/2021/03/21/2021.02.28.21252642/F6) Figure 6: Model flow of disease progression of the infected symptomatic with transitions that lead to the recovered state *R* and the died state *D* shown respectively in panels a and b. ![Formula][9] ![Formula][10] Most of the non-diagnosed severely symptomatic *I**Ss* are hospitalized at a rate *hA**H*. When they enter the hospital they are diagnosed with COVID-19. The percentage of severe cases hospitalized remains constant as long as the hospital has not reached its capacity in terms of available beds. The dichotomous variable *A**H* indicates whether the hospital is accessible to COVID-19 patients (*A**H* = 1) or whether it has reached its bed capacity and no longer accepts new COVID-19 patients (*A**H* = 0). If the hospital is at capacity, the severely symptomatic patients that would have otherwise been hospitalized are tested at a rate (1 − *A**H*)*ζ**S*(*t*). Once the hospital is no longer at capacity, these diagnosed patients are hospitalized at a rate *h** *A* which is faster than rate *h* as these diagnosed patients are likely to have waited longer to be hospitalized than the non-diagnosed. A minority of the severely symptomatic never access the hospital. Some of these patients recover but most die at home. We assume that the transition rates to the recovered state *R* is *ξ**s* and to death *D* is *µ**s*. These rates also apply to those diagnosed and severely symptomatic as the majority of those in state *Y**Ss* are diagnosed while having mild symptoms. The ODEs describing the non-hospitalized with severe symptoms are ![Formula][11] ![Formula][12] Patients that are hospitalized may develop critical symptoms. Like for the hospital, the ICU can also reach capacity, at which point it takes no more patients until it has free beds. Provided there are available ICU beds, these hospitalized critical patients transition into the ICU at a rate *𝒳A**ICU*. As with hospital accessibility, we use the dichotomous variable *A**ICU* to indicate whether the ICU is accessible (*A**ICU* = 1) to COVID-19 critical patients or whether it has reached capacity (*A**ICU* = 0). Hospitalized patients that do not require the ICU either transition to the recovered state *R* at a rate *ξ**H*, or transition to death *D* at a rate *µ**H*. The transition rate *µ**H* is assumed to be small because very few patients die in the hospital without having accessed the ICU first, provided that the ICU is not at capacity. We assume that patients who recover in the ICU move immediately to the recovered compartment rather than back to the hospital. This ensures that individuals do not make multiple trips to the ICU. The transition rate to the recovered state *R* is *ξ**ICU*. Therefore, the time spent in the *H**ICU* compartment represents both the ICU and the time spent recovering after intensive care in the hospital. The actual transition rates from the hospitalized to the recovered state *R* and death *D* depend on whether the ICU is accessible. When the ICU is closed, we assume that all those who required ICU access will die until the ICU is reopened. Hence, the general transition rate from *H**ICU* to death *D* is *µ**H* +*𝒳*(1− *A**ICU*). The ODEs describing the hospitalized are ![Formula][13] ![Formula][14] When the hospital is at capacity (*A**H* = 0), patients that develop critical symptoms are more likely to die at home. This requires modifying the rate *µ**s* based on the hospital accessibility indicator variable *A**H*. The implementation of our PBM can take this into account where the death rate *µ**s* in state *I**Ss* is increased to *µ**s* + *h𝒳*/(*𝒳* + *ξ**H*). A similar increase in the death rate applies in state *Y**Ss*. However, unlike for the ICU, the duration of having no accessibility to the hospital is likely to be short because hospitals can adapt spaces and create new bed accommodations in a way that is not possible for the ICU. Hence, the ODEs we present and describe here do not consider this increased mortality rate. Therefore, the ODEs describing the recovered and those that die are ![Formula][15] ![Formula][16] ![Formula][17] We assume that the expected time for the diagnosed and asymptomatic *Y**A* to transition to the recovered state *R* is shorter than the expected time for the same transition to the recovered state of the undiagnosed asymptomatic *I**A*. Similarly, the expected time for *Y**Sm* to transition to either state *Y**Ss* or the recovered state *R* is shorter than the expected time for those in *I**Sm* to transition to either state *I**Ss* or the *R* is. We model this by setting ![Graphic][18] and *ν** = *κ**ξ**ν* where the value of *κ**ξ* is sampled in a range of values greater than one. As for interpretation, when (*κ**ξ*−1)/*κ**ξ* is multiplied by the expected duration before the next clinical disease progression, it gives the expected time duration when an infected person is likely to be diagnosed with COVID-19. It is important to note that we have used a single compartment to represent each disease state. Since our model tracks populations’ flow between compartments rather than individuals, implicitly, the sojourn or dwelling time distribution for each compartment is exponential. Hence, it is a Markov process with the following individual-level interpretation. The sojourn times and the transition rate for the next disease progression do not depend on the history or path taken in how an individual reached a specific disease state, nor does it depend on the elapsed time they have spent in each disease state. This property can have profound consequences in the disease progression dynamics, ultimately affecting transmission. The boxcar method can alleviate the strong assumption of exponential dwelling time distributions substituting the exponential distribution with an Erlang distribution, which is more pathologically realistic [12]. This method requires breaking each disease state into a series of concatenated compartments of the same disease state and shorter internals. By making the shorter intervals equal in duration we obtain an Erlang dwelling time distribution. Conversely, the shorter internals could have different duration to approximate other types of dwelling time distributions. For example, a Weibull distribution can be approximated by a series of concatenated compartments with decreasing sojourn times [13]. A model that uses the box-car method retains its Markov property and has more realism in terms of progression rates. However, using this method the model becomes computationally expensive and has diminishing benefits for disease models like our PBM, which has many disease compartments, each with relatively short dwelling times. ### A.2 Loss of immunity The description of our model so far follows the general framework of an SEIR model. However, research indicates that immunity to COVID-19 is not permanent nor enduring and can last less than a year. To simulate this, we adopt an SEIRS model, whereby those who recover can become susceptible again. Since the rate of loss of immunity is slow with a time scale of months or years compared to the disease progression rates with time scales of days, having those who have recovered transition directly to the susceptible state is too rudimentary. This is because, as mentioned in the previous section, progression rates from one compartment to the next assume an exponential dwelling time distribution, and this assumption for the dwelling time of being recovered and immune is highly inaccurate. Hence we adopt a simple first-order boxcar correction whereby we include an intermediate or buffer recovered compartment *R**B*, which recovered people transition into before losing immunity and returning the susceptible population pool *S*. Hence, equation 1, 12 and 13 are modified, and we have an ODE describing the dynamics of the *R**B* compartment ![Formula][19] ![Formula][20] ![Formula][21] ![Formula][22] where *ρ**N* is the rate of loss of naturally acquired immunity. Extending the first order boxcar method to higher orders can accomplish more realistic distributions of natural immunity duration. It would require increasing the number of intermediate buffers *R**B* and correspondingly decrease the transition multiplicative factor from 2 to *n* + 1 where here *n* represents the number of intermediate buffers. In our model, those who lose immunity return to the susceptible state and are treated as any other susceptible. This means that our model does not consider a better prognosis due to residual immunity from the previous infection for people infected a second time. Hence, this effect leads to more pessimistic long-term and multiple-year model dynamics. ### A.3 Testing Our model assumes a daily per-person rate *ζ*(*t*) of being tested for COVID-19 on the population. People in different disease states have different demands for getting tested. For example, we assume that people that are aware of having had COVID-19 and have recovered do not seek a test. We assume that people tested in the exposed and presymptomatic infectious state are not diagnosed with COVID-19 as their test results in a false negative outcome. The specific rates *ζ**S*(*t*) and *ζ**A*(*t*) respectively represent the daily per-person detection and diagnosis rates for the symptomatic and asymptomatic populations. Those with mild and severe symptoms are more likely to get tested than those who are asymptomatic. Hence *ζ**S*(*t*) > *ζ**A*(*t*). The rate *ζ**S*(*t*) only applies to those not yet hospitalized who have either mild or severe symptoms. We assume that those who are hospitalized get tested if they did not previously test positive. The rate at which the asymptomatic population seeks a test can be considered the same for the susceptible population. However, due to testing-and-tracing efforts, it is instead likely to be marginally larger, and hence *ζ**A*(*t*) > *ζ*(*t*). Under the assumption of unconstrained testing rates, these parameters are treated as constant and depend on the base-case demand for testing. This means that anyone who seeks a test is tested. Therefore, we assume testing capacity can accommodate the growth of the epidemic, including the exponential phase. Our PBM can model both the initial and later stages of the pandemic. The assumption of unconstrained testing rates is best suited to model the later stages of the epidemic. The results presented in this paper consider unconstrained testing rates. However, for completeness, we proceed to describe the capability of our PBM in modeling the early phase of the pandemic where the number of available testing kits was limited. The implementation of our PBM can consider settings with constraints in the daily number of available testing kits. This number is assumed to start low and grow linearly to a predefined maximum daily testing rate capacity. Thus, under these settings, the actual rates describing the testing and diagnosis rates from each disease state are reduced based on the capacity constraint. We assume that people who have been hospitalized take priority and are tested first. The remaining number of testing kits is then used to test those with severe symptoms. After that, the remaining number of testing kits is used to test those with mild symptoms, followed by the asymptomatic. This approach requires specifying a constant proportion of testing kits used to successfully identify COVID-19–positive compared with those that are COVID-19–negative. ### A.4 Additional Outputs Our PBM extracts and combines incidence rates that flow into different compartments to track output cumulative quantities by population strata that do not affect the dynamics. These outputs include 1. True cumulative case counts: ![Formula][23] 2. Reported cumulative case counts: ![Formula][24] 3. Cumulative number of people tested: ![Formula][25] 4. Reported recovered: ![Formula][26] We assume that all diagnosed COVID-19 cases that do not result in death are reported as recovered cases. 5. Reported deaths: ![Formula][27] We assume that all diagnosed COVID-19 cases that result in death are reported. All suspect deaths due to COVID-19 of those with severe symptoms that did not get diagnosed and did not progress to the hospital are assumed to be unreported. 6. Reported Case Fatality Rate (CFR): ![Formula][28] 7. The Infection Fatality Rate: ![Formula][29] ### A.5 Modeling SARS-CoV-2 transmission Following the approach taken in standard compartmental models of infectious diseases, the force of infection *λ* describes SARS-CoV-2 transmission. The force of infection is characterized by how infectious people in each disease state infect others. In our formulation, the force of infection is the product of two parameters. The first parameter is the contact mixing rate, representing the number of daily contacts people make with others. The second parameter is biological transmissibility, which defines the probability of transmission between an infectious and a susceptible person when they come in contact. Each disease state would have a different content mixing rate and transmissibility. We express the force of infection as ![Formula][30] where the coefficient ![Graphic][31] represents the social mixing contact rate and ![Graphic][32] represents the transmissibility of infectious people in disease state *X**I*. The expression for the force of infection simplifies by assuming an effective contact rate *c**eff* and an effective *β**eff* transmissibility. The product of these two *c**eff* *β**eff* is assumed to characterize the rate of infections caused by an undiagnosed asymptomatic infected person in either state *P* or *I**A*. Rates of infections in the other disease states are characterized using ![Graphic][33] coefficients that give the multiplicative effect on infectivity with respect to the primary infectious state or an asymptomatic state. For example, *m**Ss* gives the overall average multiplicative infectivity of a symptomatic severe individual relative to an asymptomatic individual. This multiplicative factor combines the effect of decreased social mixing with increased biological transmissibility. We choose the asymptomatic untested individual as our reference because they are unaware of their positive status and thus do not change their social mixing behavior acting as though they are not infected. Hence ![Graphic][34]. We estimate the values of the multiplicative factor of the other disease stages by considering how the transmissibility and the social mixing contact rate change relative to the presymptomatic case. Changes in viral load are used to estimate the transmissibility of each of the disease states. Studies have shown that viral loads peak in the primary infectious stage and decrease monotonically after the onset of symptoms. Hence, we assume the inequalities ![Graphic][35]. In terms of mixing rates we assume that ![Graphic][36]. See section B.2.3 for more details on relative infectivity. Those who are in the presymptomatic or asymptomatic state are unaware of their positive status and thus act as though they are not infected. However, those who have symptoms will reduce their contacts as their conditions become more severe or receive a positive test result. Likewise, tested asymptomatic people also will begin to practice increased social distancing to protect their social contacts. Under ideal conditions, the physical contact rate that leads to disease transmission between health care workers and COVID-19 patients *c**H* and *c**ICU* would be close to zero. However, given the shortage of personal protective equipment during the early stages of the epidemic, contact safety precautions in health care settings may not have always been perfectly adhered to. We consider a range of assumptions for *c**H* and *c**ICU*. The value of the effective infectivity *c**eff* *β**eff* and its range is estimated from the basic reproductive number *R*. This number is defined at the individual-level and represents the average number of secondary infections caused by an infectious person during the disease invasion phase. This phase represents the early stage of the epidemic when susceptible individuals surround each infectious person. At the population-level, *R* represents a threshold parameter. Whether its value is larger or smaller than one, this threshold parameter indicates whether an outbreak will invade the population and become an epidemic or whether it is likely to extinguish before becoming a full-blown epidemic. The distinction between the individual-level definition and the basic reproductive number’s thresh-old property does not always align and can be very consequential, leading to incorrect conclusions. A large body of literature provides a detailed discussion on the basic reproductive number, its uses, limitations, and misconceptions [14–18]. In simple mathematical models of infectious diseases, such as Susceptible-Infected-Removed (SIR) or SEIR models, *R* can be expressed as the product of three terms *R* = *cβτ**I* where *τ**I* represents the duration of the infectious phase. This expression assumes that the contact rate *c* and the transmissibility *β* takes the same value for all the infectious compartments. This is the case for the SIR and SEIR models but it is not the case for our COVID-19 PBM. Our COVID-19 model considers more-infectious compartments with different contact mixing and transmissibility values and compartments that branch off from each other. We express *R* in terms of *c**eff* *β**eff* as ![Formula][37] where *τ**eff* represents a typical time-scale that considers the diversity of both disease transmission and progression across the different disease states. It can be interpreted as the effective infectious period for an equivalent SEIR model with a single infectious compartment, with contact rate *c**eff* and the transmissibility *β**eff*. Consequently, in our model *τ**eff* is not equal to the duration of the infectious period *τ**I* because the former accounts for the changes in transmissibilities in each infectious compartment, namely the *m* multiplicative factors. The expression for *τ**eff* is given at disease invasion and hence considers the case where testing rates *ζ**A*(*t*) and *ζ**S*(*t*) are zero. The next section, appendix A.6, describes the next-generation method and gives the mathematical expression for *τ**eff* at disease invasion in terms of the transmission and progression parameters. Hence, by knowing the input transmission and progression parameter values, we can compute the duration *τ**eff*. By knowing the value of this duration and an estimated input value of *R* we can compute the value of *c**eff* *β**eff*. To extract an estimate for the value of *R*, we use the number of reported cases during the epidemic’s early stages. In the disease invasion phase, the growth in cases is exponential. Hence, the log of the case counts and the log of the death counts increase linearly with time. An estimate for the growth rate *r* is obtained by linear regression of the log of these counts with time. Mathematically, *R* is related to *r* by the following expression ![Formula][38] where *τ**E* + *τ**I* represents the typical duration for which a person is infected and is the sum of the duration of the noninfectious incubation phase *τ**E* and the duration of the infectious phase *τ**I* [17]. The value of *τ**I* depends on the disease progression times and, more specifically, on the dwelling or sojourn times of compartments representing infectious states. As disease dynamics progress beyond the disease-invasion phase and natural immunity develops, we use a time-dependent parameter known as the effective reproduction number *R**t*. Tracking the effective reproductive number dynamics is informative because it exhibits the same threshold properties as the basic reproductive number. Hence, if its value is above 1, the disease is spreading and is not under control. At the individual-level *R**t*, it represents the average number of new infections caused by a single infected individual at time *t* in the partially susceptible population. Generally, it is found by multiplying *R* by the proportion of the population that is susceptible. For our model this means substituting equation 30 for *τ**eff* into equation 27 for *R* and multiplying by *S*(*t*). However, equation 30 provides an approximate value for *τ**eff* which is only exact at disease invasion. This is because *R* is defined at disease invasion, and this allows us to remove unnecessary details from the calculation, which do not matter at disease invasion. For example, we assume that no-one is being tested at disease invasion and that the hospital is at capacity. Hence, complexities in behavioral changes due to testing and changes in the dynamics due to hospital capacity constraints can be neglected. However, when computing *R**t* these details become important and dynamically change the typical time-scale *τ**eff* of the disease. We can obtain an equation for *τ**eff* that replaces equation 30 which is more generally valid, and our model can rely on it. However, it is a very lengthy algebraic expression. We discuss this further in appendix A.6. The estimation of the value of the growth rate *r* and hence of *R* using a linear regression approach has limitations. During the disease invasion exponential growth phase, case reports are not very reliable because of backlogs and limited testing capacity. By the time testing rates and capacity have stabilized, many jurisdictions were already in a stage where they had implemented social distancing. A better choice is to use the death count data. However, when the linear regression approach is used to find the *R* for the different U.S. states we have found that some states produced an unacceptably bad fit to the data. For these states, we assumed that the value of *R* is between two and four based on the population density of the state. For these cases, the value of *R* is assumed to range from two for the state with the lowest population density to four for the state with the highest population density. ### A.6 The expression for *τ**eff* To find the expression for *τ**eff*, we apply the next-generation method [19, 20] to the model described by the set of coupled ODEs given in Eqs 1-14. We first rearrange the order of our ODEs and focus only on the equations describing infection states. We then construct the matrices **M** and **V** respectively, describing the disease transmission and progression terms. The equations, together with the matrices, are shown on the next page. A representation of the Jacobian matrix **J*** of our system of ODEs taken at disease-free equilibrium point (i.e., *S* = 1) is given by ![Formula][39] The expression for *R* is found by finding the largest eigenvalue of **J***. At the disease-free equilibrium point, and soon after, at disease-invasion, the hospital and ICU is not at capacity and there is no testing for COVID-19. Hence the expression for *R* is found by setting *A**H* = *A**ICU* = 1 and *ζ**S* = *ζ**A* = 0). Using this simplification we find that ![Formula][40] where the coefficients *a, b* and *c* are given by ![Formula][41] and ![Formula][42] We can also find an expression for the effective reproductive number *R**t*. Generally *R**t* = *R**S*. However, we cannot use equation 30 for *τ**eff* and to get *R**t* because it is only valid at disease invasion where we assumed no testing and full accessibility to the hospital and ICU. We can derive a complete expression for *τ**eff* that considers testing rates and hospital accessibility using the next-generation method. Such expression is algebraically long and complicated, and we choose not to include it here. A Mathematica notebook providing the full expression is available upon request. The ODEs of ourmodel can be reordered as follows: ![Formula][43] Using this order, the matrix describing the transmission terms **M** is given by ![Formula][44] and the matrix **V** describing the progression terms is given by ![Formula][45] We note that although *τ**eff* expresses a time scale, it includes the multiplicative coefficients *m*. Thus, *τ**eff* gives the time scale of the whole infectious state’s Q% when all multiplicative coefficients *m* are set equal to one. Using a more direct approach instead of the next-generation matrix approach, we verified that the expression for Q-.. does indeed give Q% when all multiplicative coefficients D are set equal to one. Because ![Formula][46] ### A.7 The Population Stratified Model The population in our model can be partitioned into subpopulations. We consider multiple population groups or strata within each compartment. Each stratum specifies the population-based on common characteristics, such as demographic, social, economic, and pathological states. The specific strata we consider are described in section B.1. The structure of the population stratified model is expressed as an array of ODEs, where the disease progression dynamics for each stratum are expressed by equations 1-14. The reformulation to a strata-dependent PBM extends the model from the more conventional version of a single-strata compartment model that assumes homogeneous mixing and implicit interactions within the population. The modeled population is described by the proportion of people in each stratum. These proportions are given by the one-dimensional array ***v*** where the element *v**i* represents the proportion of the population belonging to strata *i*. Since we are tracking population densities, the sum of all the elements of the ***v*** is equal to one. We will refer to ***v*** and other strata-specific one-dimensional arrays as vectors in the disease state space *X*. Therefore, each disease state is also expressed as a vector. For example, the vector ***P*** (*t*) represents the presymptomatic state, and the element *P**i* represents the proportion of the presymptomatic population belonging to strata *i*. Some parameter values and settings that enter the model differ across the strata. The heterogeneity introduced by the strata-dependent parameters plays a crucial role in the disease progression and transmission dynamics and is used to better characterize the epidemiology of COVID-19. Differences across strata include parameter values describing pathological transition rates, including the proportion of the infectious in each stratum that stay asymptomatic (i.e., *γ**A* and *γ**S*), the proportion of the symptomatic that develop severe symptoms and need hospitalization (i.e., *ν*), and the fatality rates (i.e., the *µ* parameter values). The flexibility of our model allows users to easily specify other parameters that depend on the strata. For example, protective behaviors such as willingness to get tested can be specified by stratum. The strata-dependent parameters are also expressed as vectors and are used to specify the strataspecific transition rates between disease states. For example, the vector giving the strata-specific transition rates from state ***P*** to state ***I****A* is expressed as *γ**A* ⊙ ***P***, where ⊙ denotes the element-wise multiplication. Equivalently, this can be expressed by matrix multiplication as diag(*γ**A*) · ***P*** where the operator diag represents vector diagonalization. By following this notation, the PBM’s ODEs can be expressed in vector notation, remaining mathematically concise. Heterogeneity in disease transmission is introduced by the strata-dependent mixing contact rates describing the variations in how people belonging to the different population strata mix with each other. This heterogeneity is described by a mixing matrix ***M***. The matrix element *M**i j* denotes the proportion of contacts of individuals in population strata (or row) *i* with those in population strata (or column) *j*. The sum across each row in the mixing matrix is a sum of proportions, and hence equals one (i.e., Σ*j* *M**i j* = 1). In addition to the mixing matrix, we have a normalized contact vector ***κ***. The vector element *κ**i* denotes the proportion of all daily contacts (or duration of contacts) in the population made by individuals in stratum *i*, and hence, the sum of the vector elements is one (i.e., Σ*i* *κ**i* = 1). The matrix multiplication of the diagonalized vector ***κ*** with ***M*** gives the contact matrix ***K***, expressed by ![Formula][47] The contact matrix ***K*** is a symmetric matrix where the sum across the rows is equal to the vector ***κ***. For off-diagonal elements *i* and *j*, the sum *K**i j* +*K**ji* gives the proportion of all daily contacts between strata *i* and *j*. For diagonal elements, the same proportion is given by *K**ii*. Under the disease-free status-quo conditions everyone is susceptible. Hence ***v*** = ***S***, and the overall daily effective contact rate is propositional to ***S***T·***K*** ·***S***. During the disease transmission dynamics, we are instead interested in the contacts between the susceptible population ***S*** and each of the infectious states ***X******I***. Hence the daily effective contact rate between susceptible and infectious people is proportional to ![Graphic][48]. We denote the coefficient of proportionality by *k**λ*. Taken together with transmissibility *β**eff* and the transmissibility multiplier ![Graphic][49], we can express the force of infection vector for our strata dependent PBM as ![Formula][50] Equation 33 replaces equation 26 for our strata-specific PBM and the transition rate from *S* to *E* given in equation 1 is re-expressed as ![Formula][51] The coefficient of proportionality *k**λ* is related to the base-line total daily contact rates. We estimate *k**λ* at disease-invasion by calibration to the observed data and assume that it stays constant during the dynamics. Instead, changes in mixing rates due to social-distancing NPI and behavioral responses are accounted for by changes in the contact matrix ***K***. ### A.8 Mixing Modes People mix in different settings, or mixing modes. Modes have different levels of social interaction and different strata compositions. For instance, in schools mixing is primarily between children, whereas in commercial settings mixing occurs between all ages. Our PBM considers six different modes of social mixing: household, school, work, commerce, leisure, and other. Based on the data described in Section B.1 we can create matrices describing the average daily contacts between each stratum in each mixing mode. We decompose these matrices into a set of row normalized mixing matrices ***M****m*, column normalized contact vectors ***κ****m*, and scalar mode weight *w**m* for each mixing mode labeled by the index *m*. The total contact matrix, *K* is a weighted sum of the mode-specific contact matrices ***K****m*, and given by ![Formula][52] The weights, *w**m* give the proportion of contacts (or duration of contacts) of how people mix over the different mixing modes. Under the disease-free status-quo conditions these weights sum to one, hence Σ*m* *w**m* = 1. ### A.9 Modeling Nonpharmaceutical interventions (NPIs) When the initial outbreak becomes an epidemic, governments may choose to impose nonpharmaceutical interventions (NPIs) which limit how people can mix. The goal of NPIs is to delay and reduce the peak number of cases and hospitalizations per day by shifting individuals to locations where there are few unique contacts. Hence, we model the effects of NPIs by decreasing the weights associated with various mixing modes. For example, an NPI policy that closed schools, restaurants, and bars is modeled by reducing the weights associated with the school, commerce, and leisure mixing modes. Mixing matrices measure the number of unique contacts, not time spent in a mixing mode, therefore weights are not conserved. NPI interventions, such as confining people to their households, reduce the total number of unique contacts such that the mixing mode weights sum to less than one. To model the impact of reduced mixing from NPI level *n* on mode *m*, we define a diagonal matrix ![Graphic][53]. The diagonal elements of ![Graphic][54] specify the reduction in mixing for each stratum in mode *m* relative to the disease-free state. For interventions that apply to all strata (i.e. where each stratum changes their mixing by the same proportion), such as the closure of schools, all diagonal elements of ![Graphic][55] have the same value. In cases where interventions apply to some strata and not others: such as when only front-line essential workers are expected to attend their workplaces, the diagonal elements of ![Graphic][56] take on different values, each specifying the strata-mode specific impact of the NPI. Hence the expression for ***K***{*n*} that accounts for the impact of NPIs is: ![Formula][57] In the disease-free state, specified by NPI-level one (i.e., *n* = 1) the ![Graphic][58] matrix is equal to the identity matrix where all diagonal elements are equal to one. The matrix ![Graphic][59] is square-rooted and then multiplied on either side of the contact matrix ***K****m* so that the contact matrix is symmetric. The matrix ![Graphic][60] represents multiplicative factors that reduce the strength of the interactions along the network edges. It is obtained by the individual-level reductions specified on the network vertices. Two vertices bound each edge, and hence the overall edge-level reduction of the interactions is given by multiplying the two vertex-level reductions. These reductions are represented by the term ![Graphic][61] in mixing on either side of the strength of the edge, and given by the matrix ***K****m*. We allow the effectiveness of NPIs to vary by state. The effectiveness of the NPIs, *θ*, is defined through a calibration process. We narrow the ranges used as priors in the calibration process by exploring a wide range of parameter combinations using a Latin Hypercube Sample and by observing that values outside those ranges consistently produced biased death results. The calibrated contact matrix ![Graphic][62] is: ![Formula][63] ![Graphic][64]. For other values, *θ* approximately geometrically scales the distance between ***K***{*n*} and zero mixing, while leaving ![Graphic][65] for any *θ*. For instance, a value of *θ* = 2 yields a ![Graphic][66] approximately half the size of ***K***{*n*} for *n* ≠ 1. This functional form allows us to account for the physical and cultural differences between states which may yield different NPI effectiveness and compliance levels. After the initial wave of lockdowns in March and April 2020, many states slackened restrictions. However many behaviors to reduce transmission, including mask-wearing, aversion to crowded spaces, and other adaptation measures remained after the lockdowns were lifted. To account for this we compute the final contact matrix ![Graphic][67] as a weighted average of the contact matrix under the highest NPI ![Graphic][68] and the current NPI ![Graphic][69] ![Formula][70] The relative weight *b**h* of the highest NPI matrix is calibrated through the same process as the NPI effectiveness parameter. Finally, to prevent NPIs from changing instantaneously in the model and causing discontinuities inappropriate for an ODE, we make the NPI level *n**c* a continuous stock variable with rate ![Graphic][71], where *n** is a target NPI level and *l* determines how fast the NPI level can be changed. Therefore, the mixing matrix we use is a weighted average between the ceiling NPI level mixing matrix ![Graphic][72] and the floor NPI level ![Graphic][73] weighted by the distance between *n**c* and its ceiling. ### A.10 Modeling NPI-induced changes in household mixing The NPI-specific contact matrices ***K***{*n*} specify average mixing levels across everyone in the population. For some modes this is a reasonable approximation - the sample of individuals in a grocery store is approximately random and is likely different for each visit. However, this is not a good approximation of households, in which the same set of individuals mixes every day. Consider a stringent intervention that confined everyone to their home: the mixing weights for each other mode would drop to zero, but the mixing weights for the household would remain at one (because the number of unique household contacts remains constant). This formulation would model the situation in which all individuals are confined to a single (gigantic) household. To approximate the reality that people are confined to different households, we assume that the number of infections transmitted within the household is proportional to the amount of mixing outside the household. That is: ![Formula][74] ### A.11 Seasonality The PBM considers seasonality in the transmissibility and social mixing. This is modeled by multiplying *c**eff* *β**eff* by a time-varying term denoted by *ϑ*(*t*) that has an average value equal to one over a year. We use a simple sinusoidal function to describe *ϑ*(*t*) given by ![Formula][75] where *s* is a tunable constant. For *s* = 0 we get no seasonal effects and the multiplicative parameter *ϑ* = 1. As *s* → ∞, the multiplicative parameter *ϑ*(*t*) will tend to vary between 0 in early July and 2 in early January with an average value of 1. The sinusoidal function assumes that the seasonal decrease in transmissibility after January mirrors its seasonal increase after July. In reality, changes in transmissibility over the year are not perfectly symmetric as given by a sinusoidal function. Future versions of our model will relax this assumption. In addition to the smooth change in seasonal transmissibility described by equation 40, our model allows users to specify key date such as national holidays and sports events when people are likely to relax social distancing measures, gather with family and friends, or in large groups, and generally be less compliant to the NPIs. ### A.12 Numerical Integration The coupled ODEs defining our PBM are integrated numerically to track the dynamics of the population in each compartment as they change over time. Since the model includes many compartments and a wide range of values defining the rates across the compartments, the numerical integration of the ODEs is a stiff problem [8], whereby the numerical solution has its step size limited more severely by the stability of the numerical technique than by the accuracy of the technique. We implemented in R using the deSolve package [9]. DeSolve solves initial value problems for stiff ODEs using FORTRAN solvers of the Livermore family. There are various types of solvers that can be used [6, 7]. We tested the lsoda, vode, and the fourth-order Runge-Kutta method and found that the lsoda method was the most reliable for our purpose. ## B Informing the model ### B.1 Mixing Matrices and Population Strata We model mixing in the population using mixing matrices. A mixing matrix describes the amount of contact that occurs between each of the population strata. We consider six different mixing locations: household, work, school, commercial, recreational, and other. We use two sources for mixing matrices within the baseline scenario. The first source is a mixing matrix based on the locations of a synthetic population in the city of Portland, Oregon provided by the Network Dynamics and Simulation Science Laboratory (NDSSL) at Virginia Polytechnic Institute and State University [21]. The second data source is based on self-reported survey data. Over the course of one day in eight European countries, 7,290 participants reported 97,074 unique contacts [22]. These results were then extrapolated to create mixing matrices for 152 countries, including the United States [23]. We use the US matrix as our second data source. These sources represent two different methodologies for quantifying social mixing matrices, and both have limitations. The NDSSL data define a contact as two simulated individuals inhabiting the same sublocation at the same time. This method likely over-weights locations where there are many individuals in low mixing environments, such as workplaces. Further, the data are synthetic and represent Portland on a weekday, so might not be representative of the entire United States. The Prem et al. contacts are self-tracked, so they may over-weight contacts in close environments that are more easily remembered, such as the home. Although they are ostensibly representative of the United States, the original data were from European countries. Additionally, the Prem et al. matrices did not differentiate between commercial, recreational, and other mixing. To mitigate these concerns we averaged the mixing matrices from both sources. States typically apply NPI and vaccination policies to specific groups. To ensure that we could represent state policies, we transformed these averaged matrices to represent the nine non-aggregated strata shown in table 3. Creating different matrices for each state would be time-consuming and unnecessary for the analyses we aim to perform with the model. Instead, we created a single set of matrices using the US populations, though the size of each stratum is allowed to vary by state. The averaged matrices contain strata by age group and chronic condition. We aggregated these strata into three age groups (age ≤ 17, 18 ≤ age ≤ 64, age ≥ 65). To create ‘employed’ and ‘not employed’ strata we split mixing in the ‘working age 18 ≤ age ≤ 64 strata. In every mixing mode except work, we split working-age mixing in proportion to the population who were employed or not employed. This is equivalent to assuming that average levels of mixing in the household, school, commercial, recreational, and other settings are the same for employed and unemployed individuals. Work mode mixing was assigned entirely to the employed strata. Based on BLS data [24] approximately 10% of workers are younger than 20 or older than 65, these workers and their work mixing was aggregated into their respective age groups. View this table: [Table 3:](http://medrxiv.org/content/early/2021/03/21/2021.02.28.21252642/T3) Table 3: Model strata and corresponding populations We further split the employed strata into Front-line essential workers (FLEW) and other workers. All mixing was split in proportion to population, equivalent to assuming that FLEW and other workers have similar mixing patterns. We assumed that FLEW would only mix with other FLEW at work and, similarly, that non-FLEW workers would only mix with other non-FLEW workers. This was based on the Cybersecurity & Infrastructure Security Agency (CISA) guidance which indicates that most essential industries are composed entirely of essential workers [25]. We assumed that those workers with chronic high-risk conditions have mixing behaviors identical to their colleagues without high-risk conditions. Some of these assumptions may not be robust. For instance, essential workers may have different distributions of mixing compared to other workers in settings outside work. They may mix with other essential workers more in social settings because personal relationships are linked to work relationships and many social networks exhibit homophily [26]. Frontline essential workers may also have different levels of mixing at work. Many essential jobs, such as grocery store workers or first-responders involve high-levels of contact. However white-collar jobs are not defined by low mixing, offices and group meetings are high-contact environments, but by the ease at which mixing can be reduced during a pandemic. In an analysis of O*NET data [27], RAND colleagues found that white-collar jobs tended to have higher scores on questions that indicated higher work mixing than blue-collar or service jobs. These mixing matrices represent baseline mixing, differences in adaptability are modeled through strata-specific effects of NPIs. Detailed strata enable us to quickly adapt if states change their vaccination policies. However, model calibration and run-times are significantly longer with more strata, especially if the strata are small (due to absolute solver tolerances). To ensure that no stratum was less than 5% of the population, we aggregated the nine strata into five, as shown in Table 3. We aggregated strata that had the same vaccination priority, for instance, FLEW are always vaccinated early irrespective of whether they have a chronic condition. Where strata were aggregated, their mixing and disease severity are the population-weighted average. ### B.2 Model parameters Parameters indicate how people move between states in the model. These include disease progression rates, the proportion of individuals who enter more severe disease states, and the relative infectivity of each stage. Parameter estimates were selected from a review of the literature and with the input of RAND experts. To carry out sensitivity analyses of the parameters and to calibrate the model, we constructed a large set of independent case runs, each with a different and unique combination of model parameter values. Parameter values for the case runs are sampled using a Latin-Hypercube approach [28, 29]. We use either a uniform or a beta-PERT (Program Evaluation and Review Technique) distribution to sample the model parameter value, as specified within a sensitivity analysis range [30]. In the latter case, the reference value is used to specify the mode of the beta distribution used for our parameter value sampling. The model formulas shown are based on the parameters in Figures 4, 5, and 6. Summaries of parameter estimates, sources, and sensitivity are shown in Tables 4, 5, 6 and 7 View this table: [Table 4:](http://medrxiv.org/content/early/2021/03/21/2021.02.28.21252642/T4) Table 4: Disease duration parameter estimates View this table: [Table 5:](http://medrxiv.org/content/early/2021/03/21/2021.02.28.21252642/T5) Table 5: Disease prognosis parameter estimates View this table: [Table 6:](http://medrxiv.org/content/early/2021/03/21/2021.02.28.21252642/T6) Table 6: Relative infectivity parameter estimates View this table: [Table 7:](http://medrxiv.org/content/early/2021/03/21/2021.02.28.21252642/T7) Table 7: Other parameter estimates View this table: [Table 8:](http://medrxiv.org/content/early/2021/03/21/2021.02.28.21252642/T8) Table 8: Shortened parameter labels of the most significant parameters found by the sensitivity analyses. #### B.2.1 Duration parameters Duration parameter estimates are shown in Table 4 and specify how fast individuals advance through the disease phases. Estimates are for the mean duration of the phase length, rather than for any individual’s phase lengths. The incubation phase, or pre-symptomatic phase, is the time from exposure to the virus to the appearance of the first symptoms. We assume that immediately after exposure individuals are not infectious, but that they become infectious before exhibiting symptoms. Some research suggests that the pre-symptomatic phase is the most infectious period [31], we explore this more in Section B.2.3. We assume that the infectious periods are the same length for those with mild disease or who are asymptomatic [31]. As shown in Figure 6 all individuals who develop severe disease must first pass through the mild disease phase. The mean duration of mild disease is similar to the mean delay between symptom onset and hospitalization [32–37], so we assume that individuals are admitted almost immediately to the hospital after entering the severe compartment of the model. We describe hospital and ICU stays through two parameters. The first includes the expected time spent in the hospital at hospitalization, which includes some expectation of ICU admission. The second parameter is the expected time spent in the ICU at ICU admission as a fraction of the first parameter. For COVID-19 a significant proportion do not enter the ICU and ICU stays are long [37, 38], so the second parameter is a large fraction of the first. The parameters are constructed in this manner so that the length of ICU and hospital stays are not independently sampled. #### B.2.2 Prognosis parameters Prognosis parameter estimates are shown in Table 5 and specify what fraction of individuals enter different phases, such as whether individuals with mild disease recover or develop severe disease. We define severe disease as requiring hospitalization and critical disease as requiring ICU admission. The proportion of asymptomatic individuals used in this model is substantially lower than early estimates. Recent systematic reviews have found that many early studies overestimated the fraction of symptomatic individuals because they applied restrictive criteria for symptoms or did not observe participants for long enough to determine if they were asymptomatic or presymptomatic [42–44]. The proportion of symptomatic infections which result in hospitalization has not been reliably estimated in the literature. Available estimates find high rates (19%) but are likely biased due to selection effects which undersample mild cases [45]. Merely looking at the ratio of cumulative hospitalizations to cumulative positive tests in US (among states who report both metrics) as of February 2020 [46] shows a symptomatic hospitalization rate of around 6% (accounting for the proportion symptomatic shown in Table 4). The real figure is likely significantly lower than this number because we observe a higher fraction of hospitalizations due to COVID-19 than we do COVID-19 infections. There is evidence that the fatality rate of those hospitalized has radically decreased since the beginning of the pandemic due to a combination of improved protocols and new treatment regimens. Part of this effect is explained by a change in the age composition of the hospitalized, but accounting for this some researchers estimate a 2 to 3-fold decrease in fatality rate among those admitted to the hospital [47]. This finding suggests that treatment efficacy is an important time-varying confounder in our model. Therefore, we allow a treatment efficacy stock variable *T* that is initiated at zero is modeled as *d T* /*dt* = (*T* *−*T*)/*t**p* where *T* is the treatment efficacy, *T* * is the final treatment efficacy, and *t**p* controls how fast treatment improves. To model this, we assume that a high proportion of those admitted to the ICU die. This formulation allows treatment efficacy to become a time-varying variable, as it has been observed empirically [47]. #### B.2.3 Relative Infectivity parameters The relative infectivity, the daily expected infections of susceptible, varies by disease state. We conceptualize infectivity as having two components: contact mixing, how likely people are to mix with others, and biological transmissibility, the probability of transmission between an infectious and a susceptible person given a contact. The infectivity is the product of these components. We define infectivities relative to the asymptomatic state. To calculate biological transmissibility we rely on viral load data and estimates of the number of infections in the presymptomatic period. We use viral load data from individuals who tested positive [31], assuming that the average test occurs 3 days after initial exposure [35]. Based on relationships between biological transmissibility observed in other infectious diseases, we assume that the functional form for the relative biological transmissibility, *β*1,2 between the viral load in two states, *V L*1 and *V L*2 is: ![Formula][76] Where *δ* is a calibration parameter. Several studies have estimated that the fraction of infections caused during the presymptomatic period is approximately 45% [10, 49]. We choose *δ* = 1.92 such that 45% of infections are caused in the presymptomatic phase once contact mixing is accounted for. We also assume that those who are asymptomatic have biological transmissibility only 75% of those who have mild symptoms based on estimates by the CDC [36] Contact mixing changes by disease phase because individuals may be incapacitated COVID-19 or may voluntarily reduce their mixing to avoid infecting others. We assume that those who are presymptomatic or asymptomatic do not modify their mixing relative to those who have not yet been exposed to the virus. Those with mild symptoms reduce mixing by 40%, those who are tested positive by 80%, those who are severe (but not yet hospitalized) by 90%, and those who are hospitalized by 95%. The reduction for those who are hospitalized is larger than for those who are waiting for hospitalization because hospital staff is expected to have better protective equipment than home carers. The infectivities relative to the asymptomatic phase are shown in Table 6. #### B.2.4 Other parameters We randomly sample several other parameters from distributions. These are shown in Table 7 and pertain to testing rates, seasonality, and the effectiveness of NPIs within that state. See Sections A.3, A.9, and A.11 for more details. ##### Testing and Detection Rates In our most recent model runs, we do not assume constraints on the availability tests. Hence we assume constant rates for *ζ, ζ**S*, and *ζ**A*. During December 2020 and January 2021, the total daily rate of tests in the US ranged between 1.5 and 2 million tests a day, and the positive rate was 10%. We assume the infectious duration of the mild disease is between 4 and 7 days, and between 33% and 66% of the mildly symptomatic infected individuals seek to get tested during this time. Using these numbers, we estimate that the per-person detection rate of the mildly symptomatic *ζ**S* ranges between 0.06 to 0.13 per day. Using the 10% positive rate and the infectious duration of asymptomatic disease ranging between 4 and 7 days, we estimate that the per-person detection rate of the asymptomatic *ζ**A* ranges between 0.01 and 0.012 per day. ##### Seasonality The range of values used for our seasonality parameter *s* was based on estimates used for influenza [51]. However, depending on the climate zone, the range of *s* varied from 0.3 to 10. A more recent study that looked at seasonality patterns for two human coronaviruses (CoVHKU1 and CoVOC43) found a much lower and tighter range [52]. Based on this study, we assume that *s* varies between 0.15 and 0.3. ##### Increased progression rate for tested individuals We estimate that, on average, those who tested with mild symptoms will have experienced symptoms for a few days before being diagnosed. Hence they are more likely to progress to the next disease state than those that do not seek a test. We estimate that the time they have already spent with mild-symptoms is between 25% and 75% of the expected duration of experiencing mild symptoms. Inverting these proportions, we estimate that the range for *κ**ξ* is between 1.33 and 4 with a most likely value of 2. Similar reasoning and estimation apply to the asymptomatic and how long they are infectious before seeking to be tested if they suspect to have had contact with a confirmed case. ## C Sensitivity Analysis This section presents a sensitivity analysis of the model runs generated by an experimental design that considered 2,000 independent cases, each with a unique combination of input parameter values generated by a Latin Hypercube Sampling (LHS) approach. Each case run considers the COVID-19 dynamics for California from March 1st, 2020, to December 31st, 2020. This is the period of interest before the distribution of the vaccines. Because our period of interest has a duration of less than a year, we fix some model parameter values to their mode value. We do not consider them in the sensitivity analysis. Most notably, we fix the loss of immunity rate *ρ*. We focus on the three model outputs produced. These are the cumulative deaths and the cumulative cases diagnosed by the end of 2020, and the hospitalized cases on that date. We use two approaches. The first is to compute the partial rank correlation coefficient (PRCC) [29] and the second is based on Classification And Regression Tree (CART) analysis. The partial correlation between model output and a model input measures the degree of association between the two while controlling for the variability and the effects of the other model inputs. The partial correlation coefficient removes issues between confounding inputs. Therefore, it is generally a better measure than simple correlation and can reveal an association between an input and an output that is the inverse of that suggested by a simple correlation. Figure 7 shows the PRCC values associated with the input parameters. Parameters that have been omitted from the plot have negligible values. ![Figure 7:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/03/21/2021.02.28.21252642/F7.medium.gif) [Figure 7:](http://medrxiv.org/content/early/2021/03/21/2021.02.28.21252642/F7) Figure 7: Partial rank correlation coefficient values associate with each input parameter for our three model outputs, Our second method uses the importance scores of a CART algorithm. CART is a supervised machine learning algorithm that is commonly applied to analyze large data sets [53, 54]. For each model output of interest, CART creates a binary decision tree. This process recursively splits the space of input parameter values into regions that produce comparable output. As a first step, the algorithm uses regressions to find which input parameter and its range in values best explain the model output variance across all the model case runs. Using a cost function, it finds the input parameter’s threshold value that best splits the output data into two distinct groups. Using the CART terminology, this creates the root-node and two first-generation leaf-nodes. If the input parameter for splitting is numerical and continuous, then the algorithm uses a regression-based method. If the input parameter is categorical, then a different approach is used to use a Gini index function instead of a cost function. This index indicates how “pure” a leaf-node is in terms of what different categorical values of the input it includes. During this process, all input parameters and all possible split point values are evaluated. For a given input parameter and split point, the algorithm first splits the data into two separate training samples and runs a regression model for each sub-sample. For each of the two sub-samples, it then uses the sum of the squared difference between the training sub-sample data and the regression’s prediction to generate the cost function. The algorithm then chooses the input parameter and the split point that minimizes the cost function. Thus, the cost function is minimized when the input space is split into two regions that produce distinct outputs. The algorithm repeats this process for each of the two first-generation groups or leaf-nodes. Thus, using the subset of model case runs belonging to each group, we repeat the process and find which input parameters can best split the output data into additional second-generation leaf-nodes. This process continues until a stopping criterion is satisfied. The most common stopping procedure is to specify a minimum number of model case runs that need to be assigned to each leaf node. When the method splits the model cases runs belonging to a given node into two leaf-nodes, and either leaf-node has fewer cases than some minimum threshold number, then the split is not accepted, and the node is taken as a final leaf-node. Alternatively, stopping criteria can be specified based on when the variance of the output across all model cases contained a given leaf-node goes below a specific threshold value. The CART plots shown in figures 8 - 10 show the decision tree for each of the three model outputs and help provide a visual intuition of the importance of each input parameter in splitting the data. ![Figure 8:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/03/21/2021.02.28.21252642/F8.medium.gif) [Figure 8:](http://medrxiv.org/content/early/2021/03/21/2021.02.28.21252642/F8) Figure 8: Cumulative Deaths CART decision tree. ![Figure 9:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/03/21/2021.02.28.21252642/F9.medium.gif) [Figure 9:](http://medrxiv.org/content/early/2021/03/21/2021.02.28.21252642/F9) Figure 9: Cumulative Diagnosed CART decision tree. ![Figure 10:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/03/21/2021.02.28.21252642/F10.medium.gif) [Figure 10:](http://medrxiv.org/content/early/2021/03/21/2021.02.28.21252642/F10) Figure 10: Hospitalized CART decision tree. An overall importance score can be associated with each node calculated over the CART procedures. The importance score is related to how often each input parameter is used to split the output data most optimally over the decision tree’s nodes. It also considers how well the less optimal parameters split the data at each node without the more optimal parameters. Optimally at each node is found by finding the improvement scores associated with each node and measuring the increase in the split’s quality in the data. The parameter with the highest quality best breaks the data into two distinct output sets. The importance score takes into account which generation the node belongs to for the split. Parameters that split the data at the root node or the first-generation node have more weight. Parameters used in later generation data splits carry less weight towards calculating their importance score even if they produce high-quality data splits. Figure 11 shows a plot of the CART importance scores associated with each input parameter. ![Figure 11:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/03/21/2021.02.28.21252642/F11.medium.gif) [Figure 11:](http://medrxiv.org/content/early/2021/03/21/2021.02.28.21252642/F11) Figure 11: CART Importance scores for each model outputs and associated with each input parameter. ## D Limitations Population-level models are best suited to comparing potential interventions, rather than to forecast outcomes. They make strong assumptions about functional forms, individual behavior, and how parameters (such as treatment efficacy) change over time. These assumptions allow them to be informed with relatively little data but limit the extent to which they can reliably reproduce or predict the outbreak [55]. In this section, we detail several other model limitations. ### Unobservable True Cases Ideally, we would observe the true case count. However because cases can only be verified through testing, the true case count is not observed, instead, we use proxies: confirmed cases and deaths. Both these proxies have flaws. Observed cases are confounded by testing rates, increases in test availability have caused surges in confirmed cases leading to the overestimation of *R*. Deaths are also an imperfect proxy due to the delay between infection and death. Deaths may also be confounded by testing in states where regulations require a positive test to report deaths. ### State Mixing The model assumes that no mixing occurs across state lines. This assumption is of little importance while the virus remains widespread in almost every county in the United States. However, as the virus becomes more controlled in some areas of the country, travel bans may become an important tool to stop reseeding events. ### Behavioral Feedback To make accurate predictions about the future or counterfactuals about the past, one needs to understand how populations will react to different circumstances. Compliance is crucial to the effectiveness of NPIs but is likely a function of perceived risk. As cases decrease, even if mandated NPIs remain stringent, compliance may decrease to the point where we are unable to contain the spread of the virus. Conversely, people may limit their mixing voluntarily if they perceive significant personal risks. Our PBM does not contain an endogenous model of mixing behavior and so has limited ability to model the second-order effects in counterfactual scenarios. ## Acknowledgements We wish to thank Drs. Jeanne Ringel, Jennifer Bouey, Courtney Gidengil and Carter Price at the RAND Corporation for their support in advising the authors in the model development and in finding parameter value estimates. We thank Drs. Robert Lempert, Carolyn Rutter at the RAND Corporation and Dr. Jonathan Ozik at the Argonne National Laboratory for their support in advising the authors in model calibration and policy exploration methods. We thank Dr. Chris Bauch, a professor in applied mathematics at the University of Waterloo for his constructive comments that significantly improved our model. This project was internally funded by RAND. We thank Drs. Michael Rich, Anita Chandra and Peter Hussey for supporting our work and securing funding. We also thank the National Cancer Institute (R21CA157571), and the National Institute of Allergies and Infectious Diseases (R01AI118705) for providing support in projects that led to preliminary work and ideas that motivated this project. ## Footnotes * In this version we have (i) shortened the introduction making it more policy relevant (ii) and fixed some formatting issues and typos. * 1 One might estimate the costs of NPIs using a willingness to pay or similar approach. As long as those weights are monotonically increasing (e.g. people are not deriving utility from NPI restrictions), our substantive findings would hold. While estimating more precise welfare costs of NPIs and using those costs as weights might be valuable to compare benefits from NPIs to costs, we doubt that these weights would be stable over time. Still, as long as these weights are monotonically increasing functions of the NPI level *at any point in time* our substantive results would hold. Because these weights are highly uncertain and potentially not constant, we refrain from trying to aggregate all outcomes under a single social welfare metric in our analyses as a traditional Cost-Benefit analysis would do. Instead, we assess pareto-efficiency and seek to find strategies that dominate other strategies across a set of outcomes. * Received February 28, 2021. * Revision received March 21, 2021. * Accepted March 21, 2021. * © 2021, Posted by Cold Spring Harbor Laboratory This pre-print is available under a Creative Commons License (Attribution-NonCommercial-NoDerivs 4.0 International), CC BY-NC-ND 4.0, as described at [http://creativecommons.org/licenses/by-nc-nd/4.0/](http://creativecommons.org/licenses/by-nc-nd/4.0/) ## References 1. [1]. Julia E. Aledort, Nicole Lurie, Jeffrey Wasserman, and Samuel A. Bozzette. Non-pharmaceutical public health interventions for pandemic influenza: An evaluation of the evidence base. BMC public health, 7:208, August 2007. ISSN 1471-2458. doi: 10.1186/1471-2458-7-208. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/1471-2458-7-208&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=17697389&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F03%2F21%2F2021.02.28.21252642.atom) 2. [2]. Matthew A. Crane, Kenneth M. Shermock, Saad B. Omer, and John A. Romley. Change in Reported Adherence to Nonpharmaceutical Interventions During the COVID-19 Pandemic, April-November 2020. JAMA, January 2021. ISSN 1538-3598. doi: 10.1001/jama.2021.0286. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jama.2021.0286&link_type=DOI) 3. [3]. Bella Nichole Kantor and Jonathan Kantor. Non-pharmaceutical Interventions for Pandemic COVID-19: A Cross-Sectional Investigation of US General Public Beliefs, Attitudes, and Actions. Frontiers in Medicine, 7, July 2020. ISSN 2296-858X. doi: 10.3389/fmed.2020.00384. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3389/fmed.2020.00384&link_type=DOI) 4. [4]. Raffaele Vardavas, Aaron Strong, Jennifer Bouey, Jonathan William Welburn, Pedro Nascimento de Lima, Lawrence Baker, Keren Zhu, Michelle Priest, Lynn Hu, and Jeanne S. Ringel. The Health and Economic Impacts of Nonpharmaceutical Interventions to Address COVID-19: A Deci-sion Support Tool for State and Local Policymakers. [https://www.rand.org/pubs/tools/TLA173-1.html](https://www.rand.org/pubs/tools/TLA173-1.html), 2020. 5. [5]. Michelangelo Bin, Peter Y. K. Cheung, Emanuele Crisostomi, Pietro Ferraro, Hugo Lhachemi, Roderick Murray-Smith, Connor Myant, Thomas Parisini, Robert Shorten, Sebastian Stein, and Lewi Stone. Post-lockdown abatement of COVID-19 by fast periodic switching. PLOS Computational Biology, 17(1):e1008604, January 2021. ISSN 1553-7358. doi: 10.1371/journal.pcbi.1008604. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1371/journal.pcbi.1008604&link_type=DOI) 6. [6]. Peter N. Brown, George D. Byrne, and Alan C. Hindmarsh. VODE: A Variable-Coefficient ODE Solver. SIAM Journal on Scientific and Statistical Computing, 10(5):1038–1051, September 1989. ISSN 0196-5204. doi: 10.1137/0910062. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1137/0910062&link_type=DOI) 7. [7]. Alan C. Hindmarsh and Linda R. Petzold. Algorithms and software for ordinary differential equations and differentialalgebraic equations, Part II: Higher-order methods and software packages. Computers in Physics, 9(2):148–155, March 1995. ISSN 0894-1866. doi: 10.1063/1.168540. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1063/1.168540&link_type=DOI) 8. [8]. L. F. Shampine and C. W. Gear. A User’s View of Solving Stiff Ordinary Differential Equations. SIAM Review, 21(1):1–17, January 1979. ISSN 0036-1445. doi: 10.1137/1021001. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1137/1021001&link_type=DOI) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1979GF76900001&link_type=ISI) 9. [9]. Karline Soetaert, Thomas Petzoldt, and R. Woodrow Setzer. Solving Differential Equations in R: Package deSolve. Journal of Statistical Software, 33(1):1–25, February 2010. ISSN 1548-7660. doi: 10.18637/jss.v033.i09. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.18637/jss.v033.i09&link_type=DOI) 10. [10]. Xi He, Eric H. Y. Lau, Peng Wu, Xilong Deng, Jian Wang, Xinxin Hao, Yiu Chung Lau, Jessica Y. Wong, Yujuan Guan, Xinghua Tan, Xiaoneng Mo, Yanqing Chen, Baolin Liao, Weilie Chen, Fengyu Hu, Qing Zhang, Mingqiu Zhong, Yanrong Wu, Lingzhai Zhao, Fuchun Zhang, Benjamin J. Cowling, Fang Li, and Gabriel M. Leung. Temporal dynamics in viral shedding and transmissibility of COVID-19. Nature Medicine, 26(5):672–675, May 2020. ISSN 1546-170X. doi: 10.1038/s41591-020-0869-5. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1038/s41591-020-0869-5&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32296168&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F03%2F21%2F2021.02.28.21252642.atom) 11. [11]. Aaron Strong and Jonathan William Welburn. An Estimation of the Economic Costs of Social-Distancing Policies. May 2020. 12. [12]. Scott Greenhalgh and Carly Rozins. Novel compartmental models of infectious disease transmission. bioRxiv, page 777250, September 2019. doi: 10.1101/777250. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYmlvcnhpdiI7czo1OiJyZXNpZCI7czo4OiI3NzcyNTB2MiI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIxLzAzLzIxLzIwMjEuMDIuMjguMjEyNTI2NDIuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 13. [13]. P. E. Oguntunde, O. A. Odetunmibi, and A. O. Adejumo. On the Sum of Exponentially Distributed Random Variables: A Convolution Approach. /paper/On-the-Sum-of-Exponentially-Distributed-Random-A-Oguntunde-Odetunmibi/25d5435884e2085a76a5d7514b83168f74180554, 2014. 14. [14]. Romulus Breban, Raffaele Vardavas, and Sally Blower. Linking population-level models with growing networks: A class of epidemic models. Physical Review E, 72(4):046110, October 2005. doi: 10.1103/PhysRevE.72.046110. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1103/PhysRevE.72.046110&link_type=DOI) 15. [15]. Romulus Breban, Raffaele Vardavas, and Sally Blower. Theory versus data: How to calculate R0? PloS One, 2(3):e282, March 2007. ISSN 1932-6203. doi: 10.1371/journal.pone.0000282. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1371/journal.pone.0000282&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=17356693&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F03%2F21%2F2021.02.28.21252642.atom) 16. [16]. O. Diekmann, J. A. P. Heesterbeek, and J. A. J. Metz. On the definition and the computation of the basic reproduction ratio R0 in models for infectious diseases in heterogeneous populations. Journal of Mathematical Biology, 28(4):365–382, June 1990. ISSN 1432-1416. doi: 10.1007/BF00178324. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/BF00178324&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=2117040&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F03%2F21%2F2021.02.28.21252642.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1990DG35000001&link_type=ISI) 17. [17]. J.m Heffernan, R.j Smith, and L.m Wahl. Perspectives on the basic reproductive ratio. Journal of The Royal Society Interface, 2(4):281–293, September 2005. doi: 10.1098/rsif.2005.0042. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1098/rsif.2005.0042&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=16849186&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F03%2F21%2F2021.02.28.21252642.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000234342000002&link_type=ISI) 18. [18]. Jing Li, Daniel Blakeley, and Robert J. Smith? The Failure of R 0. Computational and Mathematical Methods in Medicine, 2011, 2011. ISSN 1748-670X. doi: 10.1155/2011/527610. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1155/2011/527610&link_type=DOI) 19. [19]. P. van den Driessche and James Watmough. Reproduction numbers and sub-threshold endemic equilibria for compartmental models of disease transmission. Mathematical Biosciences, 180(1): 29–48, November 2002. ISSN 0025-5564. doi: 10.1016/S0025-5564(02)00108-6. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S0025-5564(02)00108-6&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=12387915&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F03%2F21%2F2021.02.28.21252642.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000179220600004&link_type=ISI) 20. [20].An introduction to compartmental modeling for the budding infectious disease modeler: Letters in Biomathematics: Vol 5, No 1. [https://www.tandfonline.com/doi/full/10.1080/23737867.2018.1509026](https://www.tandfonline.com/doi/full/10.1080/23737867.2018.1509026). 21. [21]. M. V. Marathe. Synthetic Data Products for Societal Infrastructures and Proto-Populations: Data Set 2.0. Technical Report NDSSL-TR-07-003, Network Dynamics and Simulation Science Laboratory, Virginia Polytechnic Institute and State University, 1880 Pratt Dr, Building XV, Blacksburg, VA, 24061, 2014. 22. [22]. Joël Mossong, Niel Hens, Mark Jit, Philippe Beutels, Kari Auranen, Rafael Mikolajczyk, Marco Massari, Stefania Salmaso, Gianpaolo Scalia Tomba, Jacco Wallinga, Janneke Heijne, Malgorzata Sadkowska-Todys, Magdalena Rosinska, and W. John Edmunds. Social Contacts and Mixing Patterns Relevant to the Spread of Infectious Diseases. PLOS Medicine, 5(3):e74, March 2008. ISSN 1549-1676. doi: 10.1371/journal.pmed.0050074. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1371/journal.pmed.0050074&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=18366252&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F03%2F21%2F2021.02.28.21252642.atom) 23. [23]. Kiesha Prem, Alex R. Cook, and Mark Jit. Projecting social contact matrices in 152 countries using contact surveys and demographic data. PLOS Computational Biology, 13(9):e1005697, September 2017. ISSN 1553-7358. doi: 10.1371/journal.pcbi.1005697. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1371/journal.pcbi.1005697&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=28898249&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F03%2F21%2F2021.02.28.21252642.atom) 24. [24].BLS. Employed persons by detailed occupation and age. Technical report, Bureau of Labor Statistics, 2020. 25. [25].CISA. Guidance on the Essential Critical Infrastructure Workforce. [https://www.cisa.gov/publication/guidance-essential-critical-infrastructure-workforce](https://www.cisa.gov/publication/guidance-essential-critical-infrastructure-workforce), December 2020. 26. [26]. M. E. J. Newman. Assortative Mixing in Networks. Physical Review Letters, 89(20):208701, October 2002. doi: 10.1103/PhysRevLett.89.208701. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1103/PhysRevLett.89.208701&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=12443515&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F03%2F21%2F2021.02.28.21252642.atom) 27. [27]. Andrew Lauland, Benjamin Preston, Michelle Miro, Liam Regan, Scott Stephenson, Aaron Strong, Jon Welburn, and Jeffrey Wenger. National Critical Functions COVID-19 Risk Assessment: Lessons Learned. Technical Report DRR-A210-46, DHS, 2020. 28. [28]. C. J. Sallaberry, J. C. Helton, and S. C. Hora. Extension of Latin hypercube samples with correlated variables. Reliability Engineering & System Safety, 93(7):1047–1059, July 2008. ISSN 0951-8320. doi: 10.1016/j.ress.2007.04.005. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.ress.2007.04.005&link_type=DOI) 29. [29]. Alexander Hoare, David G. Regan, and David P. Wilson. Sampling and sensitivity analyses tools (SaSAT) for computational modelling. Theoretical Biology and Medical Modelling, 5(1):4, February 2008. ISSN 1742-4682. doi: 10.1186/1742-4682-5-4. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/1742-4682-5-4&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=18304361&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F03%2F21%2F2021.02.28.21252642.atom) 30. [30]. W. Cottrell. Simplified Program Evaluation and Review Technique (PERT). Journal of Construction Engineering and Management, 125(1):16–22, 1999. doi: 10.1061/(ASCE)0733-9364(1999)125: 1(16). [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1061/(ASCE)0733-9364(1999)125:1(16)&link_type=DOI) 31. [31]. Ashish Goyal, E. Fabian Cardozo-Ojeda, and Joshua T. Schiffer. Potency and timing of antiviral therapy as determinants of duration of SARS CoV-2 shedding and intensity of inflammatory response. medRxiv, page 2020.04.10.20061325, April 2020. doi: 10.1101/2020.04.10.20061325. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoibWVkcnhpdiI7czo1OiJyZXNpZCI7czoyMToiMjAyMC4wNC4xMC4yMDA2MTMyNXYyIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjEvMDMvMjEvMjAyMS4wMi4yOC4yMTI1MjY0Mi5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 32. [32]. Chanu Rhee, Sanjat Kanjilal, Meghan Baker, and Michael Klompas. Duration of SARS-CoV-2 Infectivity: When is it Safe to Discontinue Isolation? Clinical Infectious Diseases: An Official Publication of the Infectious Diseases Society of America, August 2020. ISSN 1058-4838. doi: 10.1093/cid/ciaa1249. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/cid/ciaa1249&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=33029620&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F03%2F21%2F2021.02.28.21252642.atom) 33. [33]. Andrew William Byrne, David McEvoy, Aine B Collins, Kevin Hunt, Miriam Casey, Ann Barber, Francis Butler, John Griffin, Elizabeth A Lane, Conor McAloon, Kirsty O’Brien, Patrick Wall, Kieran A Walsh, and Simon J More. Inferred duration of infectious period of SARS-CoV-2: Rapid scoping review and analysis of available evidence for asymptomatic and symptomatic COVID-19 cases. BMJ Open, 10(8), August 2020. ISSN 2044-6055. doi: 10.1136/bmjopen-2020-039856. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYm1qb3BlbiI7czo1OiJyZXNpZCI7czoxMjoiMTAvOC9lMDM5ODU2IjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjEvMDMvMjEvMjAyMS4wMi4yOC4yMTI1MjY0Mi5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 34. [34]. Min-Chul Kim, Chunguang Cui, Kyeong-Ryeol Shin, Joon-Yong Bae, Oh-Joo Kweon, Mi-Kyung Lee, Seong-Ho Choi, Sun-Young Jung, Man-Seong Park, and Jin-Won Chung. Duration of Culturable SARS-CoV-2 in Hospitalized Patients with Covid-19. New England Journal of Medicine, (): null, January 2021. ISSN 0028-4793. doi: 10.1056/NEJMc2027040. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1056/NEJMc2027040&link_type=DOI) 35. [35].CDC. COVID-19 Pandemic Planning Scenarios. [https://www.cdc.gov/coronavirus/2019-ncov/hcp/duration-isolation.html](https://www.cdc.gov/coronavirus/2019-ncov/hcp/duration-isolation.html),February 2020. 36. [36].CDC. Coronavirus Disease 2019 (COVID-19). [https://www.cdc.gov/coronavirus/2019-ncov/hcp/planning-scenarios.html](https://www.cdc.gov/coronavirus/2019-ncov/hcp/planning-scenarios.html), mFebruary 2020. 37. [37]. Christel Faes, Steven Abrams, Dominique Van Beckhoven, Geert Meyfroidt, Erika Vlieghe, and Niel Hens. Time between Symptom Onset, Hospitalisation and Recovery or Death: Statistical Analysis of Belgian COVID-19 Patients. International Journal of Environmental Research and Public Health, 17(20), October 2020. ISSN 1661-7827. doi: 10.3390/ijerph17207560. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3390/ijerph17207560&link_type=DOI) 38. [38]. Semagn Mekonnen Abate, Siraj Ahmed Ali, Bahiru Mantfardo, and Bivash Basu. Rate of Intensive Care Unit admission and outcomes among patients with coronavirus: A systematic review and Meta-analysis. PLOS ONE, 15(7):e0235653, July 2020. ISSN 1932-6203. doi: 10.1371/journal.pone.0235653. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1371/journal.pone.0235653&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F03%2F21%2F2021.02.28.21252642.atom) 39. [39]. Wei-jie Guan, Zheng-yi Ni, Yu Hu, Wen-hua Liang, Chun-quan Ou, Jian-xing He, Lei Liu, Hong Shan, Chun-liang Lei, David S.C. Hui, Bin Du, Lan-juan Li, Guang Zeng, Kwok-Yung Yuen, Ru-chong Chen, Chun-li Tang, Tao Wang, Ping-yan Chen, Jie Xiang, Shi-yue Li, Jin-lin Wang, Zi-jing Liang, Yi-xiang Peng, Li Wei, Yong Liu, Ya-hua Hu, Peng Peng, Jian-ming Wang, Ji-yang Liu, Zhong Chen, Gang Li, Zhi-jian Zheng, Shao-qin Qiu, Jie Luo, Chang-jiang Ye, Shao-yong Zhu, and Nan-shan Zhong. Clinical Characteristics of Coronavirus Disease 2019 in China. New England Journal of Medicine, 382(18):1708–1720, April 2020. ISSN 0028-4793. doi: 10.1056/NEJMoa2002032. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1056/NEJMoa2002032&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32109013&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F03%2F21%2F2021.02.28.21252642.atom) 40. [40]. Lirong Zou, Feng Ruan, Mingxing Huang, Lijun Liang, Huitao Huang, Zhongsi Hong, Jianxiang Yu, Min Kang, Yingchao Song, Jinyu Xia, Qianfang Guo, Tie Song, Jianfeng He, Hui-Ling Yen, Malik Peiris, and Jie Wu. SARS-CoV-2 Viral Load in Upper Respiratory Specimens of Infected Patients. New England Journal of Medicine, 382(12):1177–1179, March 2020. ISSN 0028-4793. doi: 10.1056/NEJMc2001737. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1056/NEJMc2001737&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32074444&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F03%2F21%2F2021.02.28.21252642.atom) 41. [41]. Jennie S. Lavine, Ottar N. Bjornstad, and Rustom Antia. Immunological characteristics govern the transition of COVID-19 to endemicity. Science (New York, N.Y.), January 2021. ISSN 1095-9203. doi: 10.1126/science.abe6522. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6Mzoic2NpIjtzOjU6InJlc2lkIjtzOjEyOiIzNzEvNjUzMC83NDEiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMS8wMy8yMS8yMDIxLjAyLjI4LjIxMjUyNjQyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 42. [42]. Eric A. Meyerowitz, Aaron Richterman, Isaac I. Bogoch, Nicola Low, and Muge Cevik. Towards an accurate and systematic characterisation of persistently asymptomatic infection with SARS-CoV-2. The Lancet Infectious Diseases, (), December 2020. ISSN 1473-3099, 1474-4457. doi: 10.1016/S1473-3099(20)30837-9. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S1473-3099(20)30837-9&link_type=DOI) 43. [43]. Oyungerel Byambasuren, Magnolia Cardona, Katy Bell, Justin Clark, Mary-Louise McLaws, and Paul Glasziou. Estimating the extent of asymptomatic COVID-19 and its potential for community transmission: Systematic review and meta-analysis. Official Journal of the Association of Medical Microbiology and Infectious Disease Canada, 5(4):223–234, December 2020. ISSN 2371-0888. doi: 10.3138/jammi-2020-0030. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3138/jammi-2020-0030&link_type=DOI) 44. [44]. Diana Buitrago-Garcia, Dianne Egli-Gany, Michel J. Counotte, Stefanie Hossmann, Hira Imeri, Aziz Mert Ipekci, Georgia Salanti, and Nicola Low. Occurrence and transmission potential of asymptomatic and presymptomatic SARS-CoV-2 infections: A living systematic review and meta-analysis. PLoS medicine, 17(9):e1003346, September 2020. ISSN 1549-1676. doi: 10.1371/journal.pmed.1003346. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1371/journal.pmed.1003346&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32960881&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F03%2F21%2F2021.02.28.21252642.atom) 45. [45]. Stephanie Bialek, Ellen Boundy, Virginia Bowen, Nancy Chow, Amanda Cohn, Nicole Dowling, Sascha Ellington, Ryan Gierke, Aron Hall, Jessica MacNeil, Priti Patel, Georgina Peacock, Tamara Pilishvili, Hilda Razzaghi, Nia Reed, Matthew Ritchey, and Erin Sauber-Schatz. Severe Outcomes Among Patients with Coronavirus Disease 2019 (COVID-19) — United States, February 12–March 16, 2020. Morbidity and Mortality Weekly Report, 69(12):343–346, March 2020. ISSN 0149-2195. doi: 10.15585/mmwr.mm6912e2. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.15585/mmwr.mm6912e2&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32214079&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F03%2F21%2F2021.02.28.21252642.atom) 46. [46].The COVID Tracking Project. [https://covidtracking.com/](https://covidtracking.com/), access date: 2020-07-01. 47. [47]. Leora I. Horwitz, Simon A. Jones, Robert J. Cerfolio, Fritz Francois, Joseph Greco, Bret Rudy, and Christopher M. Petrilli. Trends in COVID-19 Risk-Adjusted Mortality Rates. Journal of Hospital Medicine, 16(2):90–92, February 2021. ISSN 1553-5606. doi: 10.12788/jhm.3552. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.12788/jhm.3552&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=33147129&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F03%2F21%2F2021.02.28.21252642.atom) 48. [48]. R. A. Armstrong, A. D. Kane, and T. M. Cook. Outcomes from intensive care in patients with COVID-19: A systematic review and meta-analysis of observational studies. Anaesthesia, 75(10): 1340–1349, October 2020. ISSN 1365-2044. doi: 10.1111/anae.15201. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/anae.15201&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F03%2F21%2F2021.02.28.21252642.atom) 49. [49]. Miriam Casey, John Griffin, Conor G. McAloon, Andrew W. Byrne, Jamie M. Madden, David Mc Evoy, Á ine B. Collins, Kevin Hunt, Ann Barber, Francis Butler, Elizabeth A. Lane, Kirsty O’Brien, Patrick Wall, Kieran A. Walsh, and Simon J. More. Pre-symptomatic transmission of SARS-CoV-2 infection: A secondary analysis using published data. medRxiv, page 2020.05.08.20094870, June 2020. doi: 10.1101/2020.05.08.20094870. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoibWVkcnhpdiI7czo1OiJyZXNpZCI7czoyMToiMjAyMC4wNS4wOC4yMDA5NDg3MHYyIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjEvMDMvMjEvMjAyMS4wMi4yOC4yMTI1MjY0Mi5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 50. [50]. T. C. Quinn, M. J. Wawer, N. Sewankambo, D. Serwadda, C. Li, F. Wabwire-Mangen, M. O. Meehan, T. Lutalo, and R. H. Gray. Viral load and heterosexual transmission of human immunodeficiency virus type 1. Rakai Project Study Group. The New England Journal of Medicine, 342(13):921–929, March 2000. ISSN 0028-4793. doi: 10.1056/NEJM200003303421303. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1056/NEJM200003303421303&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=10738050&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F03%2F21%2F2021.02.28.21252642.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000086179700003&link_type=ISI) 51. [51]. R.F. Grais, J.H. Ellis, A. Kress, and G.E. Glass. Modeling the Spread of Annual Influenza Epidemics in the U.S.: The Potential Role of Air Travel. Health Care Management Science, 7(2):127–134, May 2004. ISSN 1572-9389. doi: 10.1023/B:HCMS.0000020652.38181.da. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1023/B:HCMS.0000020652.38181.da&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=15152977&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F03%2F21%2F2021.02.28.21252642.atom) 52. [52]. Stephen M. Kissler, Christine Tedijanto, Edward Goldstein, Yonatan H. Grad, and Marc Lipsitch. Projecting the transmission dynamics of SARS-CoV-2 through the postpandemic period. Science, 368(6493):860–868, May 2020. ISSN 0036-8075, 1095-9203. doi: 10.1126/science.abb5793. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6Mzoic2NpIjtzOjU6InJlc2lkIjtzOjEyOiIzNjgvNjQ5My84NjAiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMS8wMy8yMS8yMDIxLjAyLjI4LjIxMjUyNjQyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 53. [53]. Dan H. Moore. Classification and regression trees, by Leo Breiman, Jerome H. Friedman, Richard A. Olshen, and Charles J. Stone. Brooks/Cole Publishing, Monterey, 1984,358 pages. Cytometry, 8(5):534–535, 1987. ISSN 1097-0320. doi: 10.1002/cyto.990080516. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/cyto.990080516&link_type=DOI) 54. [54]. Leo Breiman. Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author). Statistical Science, 16(3):199–231, August 2001. ISSN 0883-4237, 2168-8745. doi: 10.1214/ss/1009213726. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1214/ss/1009213726&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=WOS:00017284&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F03%2F21%2F2021.02.28.21252642.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000172846900001&link_type=ISI) 55. [55]. David Manheim, Margaret Chamberlin, Osonde A. Osoba, Raffaele Vardavas, and Melinda Moore. Improving Decision Support for Infectious Disease Prevention and Control: Aligning Models and Other Tools with Policymakers’ Needs. RAND Corporation, 2016. ISBN 978-0-8330-9550-3. [1]: /embed/graphic-7.gif [2]: /embed/graphic-8.gif [3]: /embed/graphic-9.gif [4]: /embed/inline-graphic-1.gif [5]: /embed/inline-graphic-2.gif [6]: /embed/graphic-11.gif [7]: /embed/graphic-12.gif [8]: /embed/inline-graphic-3.gif [9]: /embed/graphic-14.gif [10]: /embed/graphic-15.gif [11]: /embed/graphic-16.gif [12]: /embed/graphic-17.gif [13]: /embed/graphic-18.gif [14]: /embed/graphic-19.gif [15]: /embed/graphic-20.gif [16]: /embed/graphic-21.gif [17]: /embed/graphic-22.gif [18]: /embed/inline-graphic-4.gif [19]: /embed/graphic-23.gif [20]: /embed/graphic-24.gif [21]: /embed/graphic-25.gif [22]: /embed/graphic-26.gif [23]: /embed/graphic-27.gif [24]: /embed/graphic-28.gif [25]: /embed/graphic-29.gif [26]: /embed/graphic-30.gif [27]: /embed/graphic-31.gif [28]: /embed/graphic-32.gif [29]: /embed/graphic-33.gif [30]: /embed/graphic-34.gif [31]: /embed/inline-graphic-5.gif [32]: /embed/inline-graphic-6.gif [33]: /embed/inline-graphic-7.gif [34]: /embed/inline-graphic-8.gif [35]: /embed/inline-graphic-9.gif [36]: /embed/inline-graphic-10.gif [37]: /embed/graphic-35.gif [38]: /embed/graphic-36.gif [39]: /embed/graphic-37.gif [40]: /embed/graphic-38.gif [41]: /embed/graphic-39.gif [42]: /embed/graphic-40.gif [43]: /embed/graphic-41.gif [44]: /embed/graphic-42.gif [45]: /embed/graphic-43.gif [46]: /embed/graphic-44.gif [47]: /embed/graphic-45.gif [48]: /embed/inline-graphic-11.gif [49]: /embed/inline-graphic-12.gif [50]: /embed/graphic-46.gif [51]: /embed/graphic-47.gif [52]: /embed/graphic-48.gif [53]: /embed/inline-graphic-13.gif [54]: /embed/inline-graphic-14.gif [55]: /embed/inline-graphic-15.gif [56]: /embed/inline-graphic-16.gif [57]: /embed/graphic-49.gif [58]: /embed/inline-graphic-17.gif [59]: /embed/inline-graphic-18.gif [60]: /embed/inline-graphic-19.gif [61]: /embed/inline-graphic-20.gif [62]: /embed/inline-graphic-21.gif [63]: /embed/graphic-50.gif [64]: /embed/inline-graphic-22.gif [65]: /embed/inline-graphic-23.gif [66]: /embed/inline-graphic-24.gif [67]: /embed/inline-graphic-25.gif [68]: /embed/inline-graphic-26.gif [69]: /embed/inline-graphic-27.gif [70]: /embed/graphic-51.gif [71]: /embed/inline-graphic-28.gif [72]: /embed/inline-graphic-29.gif [73]: /embed/inline-graphic-30.gif [74]: /embed/graphic-52.gif [75]: /embed/graphic-53.gif [76]: /embed/graphic-60.gif