Causal Estimands for Analyses of Averted and Avertible Outcomes due to Infectious Disease Interventions ======================================================================================================= * Katherine M Jia * Christopher B Boyer * Jacco Wallinga * Marc Lipsitch ## ABSTRACT During the coronavirus disease (COVID-19) pandemic, researchers attempted to estimate the number of averted and avertible outcomes due to non-pharmaceutical interventions or vaccination campaigns to quantify public health impact. However, the estimands used in these analyses have not been previously formalized. It is also unclear how these analyses relate to the broader framework of direct, indirect, total, and overall causal effects of an intervention under interference. In this study, using potential outcome notation, we adjust the direct and overall effects to accommodate analyses of averted and avertible outcomes. We use this framework to interrogate the commonly-held assumption in empirical studies that vaccine-averted outcomes via direct impact among vaccinated individuals (or vaccine-avertible outcomes via direct impact among unvaccinated individuals) is a lower bound of vaccine-averted (or avertible) outcomes overall. To do so, we describe a susceptible-infected-recovered-death model stratified by vaccination status. When vaccine efficacies wane, the lower bound fails for vaccine-avertible outcomes. When transmission or fatality parameters increase over time, the lower bound fails for both vaccine-averted and avertible outcomes. Only in the simplest scenario where vaccine efficacies, transmission, and fatality parameters are constant over time, outcomes averted via direct impact among vaccinated individuals (or outcomes avertible via direct impact among unvaccinated individuals) is shown to be a lower bound of overall impact on vaccine-averted (or avertible) outcomes. In conclusion, the lower bound can fail under common violations to assumptions on constant vaccine efficacy, pathogen properties, or behavioral parameters over time. In real data analyses, estimating what seems like a lower bound on overall impact through estimating direct impact may be inadvisable. By classifying estimands for averted and avertible outcomes and examining their relations, this study improves conduct and interpretation of research evaluating impact of infectious disease interventions. KEYWORDS * direct effect * overall effect * vaccine-averted deaths * vaccine-preventable deaths ## 1. INTRODUCTION Recently, researchers have estimated the number of COVID-19 deaths (or infections) averted by vaccination campaigns in the United States,1,2 Israel,3,4 Chile,5 Brazil,6 and Japan.7 Similarly, other studies, including our own8, have attempted to estimate vaccine-avertible deaths—defined as the number of deaths that could have been averted by vaccination, but were not because of a failure to vaccinate the unvaccinated.8,9 Most empirical studies quantify direct protection conferred by vaccination among vaccinated individuals, but they typically assume the overall impact of vaccination is larger due to indirect protection.4–6 Indeed, these studies often claim that their estimates represent a lower bound on the overall impact of vaccination, although this claim has not been carefully verified. To support this claim, many of these studies draw, either implicitly or explicitly, on the causal effect framework developed by Halloran and Struchiner,10 in which the overall effect of vaccines can be decomposed into direct and indirect components.11 However, the Halloran and Struchiner framework has not yet been formally extended to cover the specific estimands targeted in vaccine-averted and avertable analyses, which estimate *impact*. Existing effect estimands are defined by contrasts of individual risk12,13 and have been used to estimate vaccine efficacy in clinical trials.14,15 However, observational studies, post-licensure studies, and policy-makers are often equally or more interested in quantifying public health *impact* of vaccination in terms of counts16 such as the number of infections, hospitalizations, and deaths in a group of individuals, instead of risk in each individual. Motivated by recent empirical studies on vaccine-averted and avertible COVID-19 deaths, this article seeks to fill these gaps by: (1) Clearly defining *impact* estimands as corollaries of the direct and overall effect estimands for averted and avertible outcomes (i.e., counts), and (2) determining the conditions under which direct impact is a lower bound on overall impact. To ground our discussion, we introduce a susceptible-infected-recovered-death (SIRD) model stratified by vaccination status to investigate the direct impact and overall impact of vaccination under different scenarios. The rest of the article is structured as follows: In Section 2, we summarize the definitions of the direct, indirect, total, and overall effects introduced by Hudgens and Halloran,12 and provide an alternative partitioning of overall effect into components that will align better with the estimands targeted by vaccine-averted and avertible analyses. In Section 3, we propose count outcome corollaries for the direct and overall effects, show how they map onto estimands for vaccine-averted and avertible analyses, and formalize the claim that the direct impact constitutes a lower bound on overall impact. Section 4 outlines a transmission model to simulate vaccine-averted and avertible outcomes. In Section 5, we examine the conditions under which outcomes averted via direct impact among vaccinated individuals (or outcomes avertible via direct impact among unvaccinated individuals) is and is not a lower bound of overall impact on vaccine-averted (or avertible) outcomes. ## 2. DIRECT, INDIRECT, OVERALL, AND TOTAL EFFECTS Hudgens and Halloran12 previously defined causal estimands for direct, indirect, overall, and total effects in the two-stage randomized trial, as summarized below. ### Setup and notation Consider a two-stage randomized trial with *m* groups indexed by *i* = 1, …, *m*, such that each group consists of *N* individuals indexed by *j* = 1, …, *N* with a large group size *N*. All groups are assumed to be of same size *N*. Partial interference is assumed: Individuals make contacts (leading to interference) within the same group, but individuals in different groups make no contacts. For ease of exposition, assume interest lies in quantifying the effect of vaccination which is a one-time event before the start of an outbreak. Let *A**ij* = 1 if individual *j* in each group *i* is vaccinated and *A**ij* = 0 otherwise. Let ***A******i*** = (*A**i*1, *A**i*2, …, *A**iN*) and ***A******i***,**−*j*** = (*A**i*1, *A**i*2, …, *A**ij*−1, *A**ij*+1, …, *A**iN*), hereafter referred to as allocation programs.11 Let **a*****i*** and **a*****i***,**−*j*** denote possible realizations of ***A******i*** and ***A******i***,**−*j***, respectively. Let 𝒜(*N*) denote the set of all possible 2*N* vaccine allocations for a group of size *N*, for which **a*****i*** ∈ 𝒜(*N*). Let *Y**ij*(**a*****i***) denote the potential binary outcome for individual *j* in group *i* with allocation program **a*****i*** and let *Y**ij*(**a*****i***,**−*j***, *a*) denote the potential binary outcome when individual *j* has vaccination status *a* and the rest of group *i* has vaccination status **a*****i***,**−*j***. #### Individual, group, and population average potential outcomes Hudgens and Halloran12 define the *marginal individual average potential outcome* as ![Formula][1] and the *individual average potential outcome*12 as: ![Formula][2] where ![Graphic][3] is the probability distribution of vaccine allocation program ***A******i*** with parameter *α* ∈ [0,1] representing the proportion vaccinated within group *i*. Specifically, ![Graphic][4] is the probability distribution of ***A******i*** conditional on ![Graphic][5]. Note that here we use type A parameterization, which gives same effect definitions as type B parameterization suggested by VanderWeele and Tchetgen Tchetgen11 when *N* is large. Definitions of ![Graphic][6], type A, and type B parameterizations are given in eAppendix 1. Hudgens and Halloran12 further define (marginal) group average potential outcomes ![Graphic][7] and ![Graphic][8] by averaging over individuals within groups. They also define (marginal) population average potential outcomes12 ![Graphic][9] and ![Graphic][10]. ### Population Average Direct, Indirect, Overall, and Total Causal Effects Hudgens and Halloran define the *population average direct casual effect*12 as ![Graphic][11], which compares ![Graphic][12] when an individual is unvaccinated versus when vaccinated, holding fixed the proportion vaccinated (*α*). They define the *population average indirect casual effect*12 as ![Graphic][13], which compares ![Graphic][14] for an unvaccinated individual in a group with *α* proportion vaccinated versus with *α*′ proportion vaccinated and is hereafter referred to as indirect effect for the unvaccinated. As noted in the literature,17 indirect effect can be analogously defined when an individual is vaccinated: ![Graphic][15], which is hereafter referred to as indirect effect for the vaccinated. They also define the *population average total causal effect*12 as ![Graphic][16] which compares ![Graphic][17] when an individual is unvaccinated in a group with *α* proportion vaccinated versus when the individual is vaccinated in a group with *α*′ proportion vaccinated. Following from the definition, *TE*(*α, α*′) can be decomposed as *DE*(*α*′) + *IE**Unvax*(*α, α*′) or *DE*(*α*) + *IE**vax*(*α, α*′). Finally, they define the *population average overall casual effect*12 as ![Graphic][18], which compares ![Graphic][19] for a typical individual in a group with *α* proportion vaccinated versus with *α*′ proportion vaccinated. ### Overall Effect Partitioning Previously, Hudgens and Halloran12 showed that when *α* = 0, *OE*(*α, α*′) is the weighted sum *α*′ ⋅ *TE*(*α, α*′) + (1 − *α*′) ⋅ *IE**Unvax*(*α, α*′). In this study, for the purpose of establishing the direct impact as a lower bound of the overall impact that is shown in Section 3, we generalize the partitioning of *OE*(*α, α*′) to any *α*′ > *α* in Theorem 1. *Theorem 1 (overall effect partitioning)* ![Formula][20] Theorem 1 is proved in eAppendix 2 and graphically illustrated in Figure 1. Theorem 1 expresses *OE*(*α, α*′) as a weighted average of three effects: 1) *IE**vax*(*α, α*′), 2) *TE*(*α, α*′), and 3) *IE**Unvax*(*α, α*′). Intuitively, if individuals are classified by their vaccination status under a pair of counterfactuals wherein the group has *α* or *α*′ proportion vaccinated (*α*′ > *α*), then *IE**vax*(*α, α*′) is in operation for proportion *α* of individuals who are vaccinated under both counterfactuals; *TE*(*α, α*′) is in operation for proportion *α*′ − *α* of individuals who are vaccinated under *α*′ but unvaccinated under *α*; and *IE**Unvax*(*α, α*′) is in operation for proportion 1 − *α*′ of individuals who are unvaccinated under both counterfactuals. ![Figure 1.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2024/11/11/2024.07.24.24310946/F1.medium.gif) [Figure 1.](http://medrxiv.org/content/early/2024/11/11/2024.07.24.24310946/F1) Figure 1. Graphical illustration on partitioning overall effect. The two rectangles represent a pair of counterfactuals wherein the group has *α* or *α*′ proportion vaccinated (*α*′ > *α*). Individuals fall into three categories based on their vaccination status under the counterfactuals: 1) The dotted region represents those who are unvaccinated under both counterfactuals (hereafter referred to as “never-vaccinated”) and for whom *IE**Unvax*(*α, α*′) is in operation; 2) the gridded region represents those who are unvaccinated under *α* but vaccinated under *α*′ and for whom *TE*(*α, α*′) is in operation (hereafter referred to as “additionally-vaccinated”); and 3) the stripped region represents those who are vaccinated under both counterfactuals (hereafter referred to as “always-vaccinated”) and for whom *IE**vax*(*α, α*′) is in operation. Theorem 1 shows that *OE*(*α, α*′) is a weighted average of three effects: 1) *IE**Unvax*(*α, α*′), 2) *TE*(*α, α*′), and 3) *IE**vax*(*α, α*′), each weighted by the proportion of individuals for whom the effect is in operation respectively: 1) 1 − *α*′ for the never-vaccinated, 2) *α*′ − *α* for the additionally-vaccinated, and 3) *α* for the always-vaccinated. ## 3. ESTIMANDS FOR ANALYSES OF AVERTED AND AVERTIBLE OUTCOMES Up until now, all effect estimands (except the indirect effect for vaccinated)17 have been previously defined by Hudgens and Halloran in a two-stage randomized trial. To define estimands for analyses of averted and avertible outcomes, we use and expand on the terminology of two-stage randomization. Our goal is to be explicit about the definition of estimands in recent observational studies, to assess the applicability of the lower bound argument, and to place these observational studies in the context of target trial emulation, which is playing a growing role in causal inference. The original overall effect (OE) and direct effect (DE) are defined for individual risk,10 and their magnitudes are not comparable because OE and DE use different denominators. However, empirical observational studies often assume vaccination prevents more cases overall than directly for a count outcome, by assuming that multiplying OE with total population size is greater than multiplying DE with the number of vaccinated individuals in the presence of indirect protection for unvaccinated individuals.4–9 To verify this assumption, Section 3 defines two estimands for averted and avertible outcomes—direct impact and overall impact—that align with the targeted estimands in the literature on vaccine-averted and avertible outcomes,1–9 by multiplying the original effect estimands with the number of individuals for whom the effect is in operation. Sections 4 and 5 then verify conditions under which direct impact is or is not a lower bound of overall impact. This paper adds to the existing literature by: (1) Introducing an additional time index *t* to examine outcomes at multiple timepoints post-vaccination, (2) defining the direct impact and overall impact (i.e., corollaries of the direct and overall effect estimands for count outcomes) to be used in averted and avertible outcome analyses, (3) defining the relationships between direct impact and overall impact, and (4) most importantly, using analytical and simulation approaches to identify conditions under which direct impact may or may not be a lower bound of overall impact. ### Notation To examine the change of direct impact and overall impact over time, we incorporate time index *t* so that ![Graphic][21] denotes the population average cumulative incidence of having developed the outcome by time *t*, and similarly for ![Graphic][22] and all the effect estimands defined in Section 2. *t* = 0 denotes start of follow-up. In principle, *t* can be of any timescale, while *t* is in the unit of days in the simulation in Section 5. Now let *α*1 denote a particular vaccination proportion chosen to be implemented in the group, let *α*2 denote some hypothetical higher vaccination proportion (i.e., *α*2 > *α*1), and let *α* denote some hypothetical lower vaccination proportion (i.e., *α* < *α*1). ### Motivating Examples and Causal Questions During the COVID-19 pandemic, determining the total number of infections (or deaths) averted by vaccination has been of great public health interest.1–7,18–20 Vaccine-averted infections (or deaths) is an impact estimand based on the causal question: How many infections (or deaths) have been averted under the particular proportion vaccinated (*α*1) compared to the counterfactual in the absence of vaccination (*α* = 0)? Alternatively, researchers have also shown interest in estimating the vaccine-avertible deaths, that is those that could have been averted by vaccination but were not because of a failure to vaccinate.8,9 The causal question in this case is: How many infections (or deaths) could have been averted under full vaccination (*α*2 = 1), but were not averted given the particular proportion vaccinated (*α*1)? In general, estimands can be defined by comparing the number of infections (or deaths) in a typical group under the particular proportion vaccinated (*α*1) versus when the group has a lower (*α*) or higher (*α*2) proportion vaccinated.9 We term these as *impact* estimands because previous literature has referred to vaccine-averted infections (or deaths) as a (population) impact of vaccination.5,7,18,19 ### Overall Impact For *α* < *α*1, overall impact is defined as: ![Formula][23] For *α*2 > *α*1, overall impact is defined as: ![Formula][24] Overall impact (i.e., overall effect multiplied with population size) directly answers the two aforementioned causal questions on vaccine-averted and avertible outcomes. Therefore, overall impact is an estimand for capturing the vaccine averted and avertible outcomes. For reasons that will become clearer later, we force overall impact to have the same sign when *α*1 is compared to a lower (*α*) or higher (*α*2) hypothetical value. Consequently, arguments of overall impact (*δ**O*) are ordered according to the order of the values (e.g., *α* comes before *α*1 where *α* < *α*1). Note that mathematical modelling studies implicitly refer to overall impact when estimating vaccine-averted deaths by simulating the epidemic trajectory under the counterfactual with a hypothetical proportion vaccinated (e.g., *α* = 0) and comparing it with the estimated number under the particular vaccination campaign (*α*1). Furthermore, by Theorem 1 and then substituting *TE*(*t, α*, *α*1) = *DE*(*t, α*1) + *IE**Unvax*(*t, α*, *α*1), we have the decomposition: ![Formula][25] for *α*1 > *α*. The first term on the right-hand side of the last line of equation (2.1) scales *IE**vax*(*t, α*, *α*1) by number vaccinated under *α*. The second term scales *DE*(*t, α*1) by the additional number vaccinated under *α*1 compared to *α*. The third term scales *IE**Unvax*(*t, α*, *α*1) by number unvaccinated under *α*. Similarly, by Theorem 1 and then substituting *TE*(*t, α*1, *α*2) = *DE*(*t, α*1) + *IE**vax*(*t, α*1, *α*2), we also have the decomposition: ![Formula][26] for *α*2 > *α*1. The first term on the right-hand side of the last line of equation (2.2) scales *IE**vax*(*t, α*, *α*1) by number vaccinated under *α*2. The second term scales *DE*(*t, α*1) by the additional number vaccinated under *α*2 compared to *α*1. The third term scales *IE**Unvax*(*t, α*, *α*1) by number unvaccinated under *α*2. ### Direct Impact Because the overall impact estimand (*δ**O*) is defined as a contrast under two different vaccination proportions, at minimum researchers need to observe two noninteracting groups where each is “assigned” to one of the vaccination strategies of interest to estimate overall impact.12 Ideally, vaccination would be assigned via a two-stage randomized trial. However, these trials are rarely conducted because they are expensive and often hard to justify.21 Instead, most empirical studies only observe a single group under one vaccination proportion (*α*1), in which case it is only feasible to estimate an impact corollary of the direct effect. Examples include observational studies comparing discrete hazards or incidence rates between vaccinated and unvaccinated individuals based on national vaccine data systems.4–9 These studies multiply the direct effect with number vaccinated (or unvaccinated) to estimate outcomes averted (or avertible) via direct impact among vaccinated (or unvaccinated) individuals,4–9 and then generally assume that it is a lower bound of overall impact on averted (or avertible) outcomes in the entire population.4–6,8 In this section, we formalize this definition of direct impact and show how it relates to the overall impact. Some empirical observational studies have used commonly available data to estimate deaths averted via direct impact among vaccinated individuals4–7 based on formulas similar to *N* ⋅ (*α*1 − *α*) ⋅ *DE*(*t, α*1) by setting *α* = 0, while other studies have estimated deaths avertible via direct impact among unvaccinated individuals8,9 based on formulas similar to *N* ⋅ (*α*2 − *α*1) ⋅ *DE*(*t, α*1) by setting *α*2 to be greater than *α*1 (Note these studies have also considered increases in proportion vaccinated over time, such as occurred during a vaccination campaign, which here we ignore for simplicity). Following the literature,4–9,18 let the direct impact (*δ**D*) for any *α, α*′ ∈ [0,1] be *δ**D*(*t, α, α*′) = *N* ⋅ |*α*′ − *α*| ⋅ *DE*(*t, α*′). In particular, for *α*1 > *α*, we have ![Formula][27] and for *α*2 > *α*1, ![Formula][28] Note that *δ**D* (*t, α*, *α*1) or *δ**D* (*t, α*2, *α*1) is not a meaningful causal estimand by itself because *δ**D* (*t, α*1) is conditional on *α*1 only and does not account for the change in direct effect under the counterfactual when the proportion vaccinated is *α* or *α*2 instead of *α*1. Importantly, now direct impact can be a lower bound of overall impact. This is because Theorem 1 provides a useful decomposition of *OE* into *TE, IE**Unvax*, and *IE**vax*, leading to the relationship between *OE* and |*α*′ − *α*| ⋅ *DE* in equations 2.1 and 2.2, such that we can map direct impact onto *N* ⋅ |*α*′ − *α*| ⋅ *DE*(*t, α*′) and show that direct impact can be a lower bound of overall impact. That is, equations (2.1) and (2.2) now represent: ![Formula][29] for *α*1 > *α*, and ![Formula][30] for *α*2 > *α*1. Based on equations 1.1 and 1.2, overall impact has the same sign when *α*1 is compared to a lower (*α*) or higher (*α*2) hypothetical value. Therefore, we can formalize the assumption often made in the empirical vaccine-averted and avertible death studies—that is, direct impact is a lower bound of overall impact by considering Claim 1: ![Formula][31] and ![Formula][32] If Claim 1 is true, direct impact, which can be estimated using commonly available data,4–9,18 is a useful lower bound of overall impact that is relevant for policy-making and retrospective policy evaluation requiring samples from a population of groups as in two-stage randomized trials.12 Following the empirical studies estimating vaccine-averted or avertible outcomes, we consider two special cases of Claim 1. In Claim 1a, a particular vaccination proportion *α*1 is compared to a hypothetical of no vaccination (*α* = 0) to quantify vaccine-averted outcomes. Claim 1a (vaccine-averted outcomes): ![Formula][33] In words, Claim 1a asserts that for *α*1 > *α* = 0, outcomes averted via direct impact of current vaccination among the vaccinated individuals *δ**D*(*t*, 0, *α*1) is a lower bound of total vaccine-averted outcomes among both vaccinated and unvaccinated individuals *δ**O*(*t*, 0, *α*1). In Claim 1b, *α*1 is compared to a hypothetical of near-universal vaccination (*α*2 = 0.9 > *α*1) to quantify vaccine-avertible outcomes, following the literature.9 Claim 1b (vaccine-avertible outcomes): ![Formula][34] In words, Claim 1b asserts that for *α*2 = 0.9 > *α*1, outcomes avertible via direct impact of current vaccination among some unvaccinated individuals *δ**D*(*t*, 0.9, *α*1) is a lower bound of vaccine-avertible outcomes among both vaccinated and unvaccinated individuals *δ**O*(*t, α*1, 0.9) comparing current proportion vaccinated (*α*1) to a proportion of *α*2 = 0.9. Note that we do not compare to full vaccination (*α*2 = 1) because direct effect would be undefined if the group is fully vaccinated, and there would not be an epidemic if the group is fully vaccinated with a highly effective vaccine. In the following sections, we use a susceptible-infected-recovered-death (SIRD) model to verify the conditions under which the Claims may or may not hold. Based on the effect partitioning results in equation 4.1 or 4.2, Claims 1a and 1b would be true if *IE**vax* and *IE**Unvax* are non-negative. However, it is not immediately intuitive when we might expect that to be the case. Therefore, in the next two sections we describe a transmission model that we use to check Claims 1a and 1b under various scenarios (Section 4), and then describe conditions under which the direct impact is or is not a lower bound for the overall impact, using both analytical and simulation-based approaches (Section 5). ## 4. TRANSMISSION MODEL ### The SIRD Model with Vaccination at Baseline To model the impact of vaccination on infection and death, an SIRD model is used to represent a well-mixed group in a two-stage randomized trial assuming partial interference.12 To simulate direct impact and overall impact, we simulate a typical group with a large size under a pair of counterfactual vaccination proportions. In this simulation, the (marginal) group average potential outcome is equivalent to (marginal) population average potential outcome because there is only one group in the population. The model consists of four states for a vaccinated or unvaccinated individual—susceptible, infectious, recovered, and death due to infection. We assume that the group has been randomly assigned with a vaccination policy wherein the proportion vaccinated is *α*, and individuals have been randomly assigned with vaccination status *a* at baseline (*a* = 1 for vaccinated and *a* = 0 for unvaccinated; for equation [5], subscript *v* denotes vaccinated and subscript *u* for unvaccinated). The vaccine is “leaky” in protecting against infection and infection-related death—that is, vaccinated individuals have the susceptibility reduced by a factor *θ* against infection (i.e., vaccine efficacy against infection [*VE**infection*] is (1 − *θ*) • 100%) and have the susceptibility reduced by another factor *κ* against death (i.e., vaccine efficacy against death given infection [*VE**death*|*infection*] is (1 − *κ*) • 100%). Individuals mix homogeneously such that each vaccinated or unvaccinated susceptible individual is equally likely to contact any infectious individual. Vaccinated and unvaccinated infectious individuals are equally contagious. The transmission dynamics are: ![Formula][35] where *γ* = recovery rate, ![Graphic][36] is the hazard rate of infection, with *β* the number of effective contacts made by a typical infectious individual per unit time, and *μ* = probability of death due to infection. In equation (5), *S**U,α*(*t*) and *S**v,α*(*t*) denote, respectively, the number of susceptible individuals who are unvaccinated and vaccinated, *I**U,α*(*t*) and *I**v,α*(*t*) for the infectious individuals, *R**U,α*(*t*) and *R**v,α*(*t*) for the recovered individuals who are no longer at risk, and *D**U,α*(*t*) and *D**v,α*(*t*) for those who died due to infection. *N*(*t*) denotes the sum of all compartments at time *t*. eFigure 1 shows the model flowchart, and eTable 1 shows the parameter values used in simulation. ### Software All model simulations and visualization are conducted using R 4.2.2 (R Foundation for Statistical Computing, Vienna, Austria).22 All models are implemented using R package *odin*.23 Code is available at [https://github.com/katjia/population\_level_effects](https://github.com/katjia/population_level_effects). ## 5. WHEN IS DIRECT IMPACT A LOWER BOUND OF OVERALL IMPACT? ### Rationale The SIRD model has many simplifying assumptions compared to real-world settings. If one can show a counterexample to Claim 1 based on the simplest SIRD model, then Claim 1 is not guaranteed to be true in more general and realistic models. To identify counterexamples of Claim 1, we consider common violations to assumptions on time-invariant parameters so that indirect effects can be negative: (1) Number of effective contacts may increase over time due to meteorological factors,24 lifting of non-pharmaceutical interventions,25 and seasonal variation in social contacts, (2) infection-fatality risk may increase over time,26 (3) waning immunity clearly occurred in the Delta and Omicron waves of the COVID-19 pandemic.27,28 If Claim 1 fails under these common violations, it would be overly optimistic for the empirical studies to assume that Claim 1 holds in reality. ### Scenarios We check the Claims under 5 scenarios (Table). Scenario 1 refers to the SIRD model in equation (5) with time-invariant parameters. eTable 1 lists the model parameters for which one or two parameters vary under each Scenario separately: Scenario 2 increases the number of effective contacts made by a typical infectious individual per day (*β*) from 0.15 to 0.6 from Day 300 onwards; Scenario 3 increases probability of death due to infection (*μ*) from 0.01 to 0.1 from Day 300 onwards; Scenario 4 allows both *VE**infection* and *VE**death*|*infection* to wane linearly after Day 100 reaching 0% at Day 300; and Scenario 5 combines Scenarios 2 (increasing *β*) and 4 (waning VEs). View this table: [TABLE 1.](http://medrxiv.org/content/early/2024/11/11/2024.07.24.24310946/T1) TABLE 1. Scenarios under which the Claims may or may not hold a ### Proof of Claim 1 at the end of outbreak under Scenario 1 For Scenario 1 (i.e., time-invariant parameters), in eAppendix 4, we prove that Claim 1 (i.e., direct impact is a lower bound of overall impact for any two proportions vaccinated) holds in the SIRD model at the end of outbreak (i.e., at *t* → ∞). ### Simulations For *t* < ∞, we first verify Claims 1a and 1b (special cases of Claim 1) through simulation based on parameters specified in eTable 1 and initial conditions in eTable 2. We specify *α* = 0 versus *α*1 = 0.7 in the pair of trajectories to verify Claim 1a and *α*1 = 0.7 versus *α*2 = 0.9 to verify Claim 1b. If Claims 1a and 1b both hold for the specified parameters, Latin hypercube sampling is conducted to generate alternative sets of proportions vaccinated and model parameters to verify the full Claim 1. If only one of Claim 1a or 1b holds, Latin hypercube sampling is conducted to verify that the Claim holds under alternative proportion vaccinated and model parameters (e.g., trying alternative values for *α*1 while fixing *α* = 0 for Claim 1a). Briefly, Claim 1 only holds under Scenario 1 (i.e., time-invariant parameters), but it does not hold under any other Scenarios. The Table summarizes the results. Figures 2 and 3 show trajectories of direct impact and overall impact throughout the epidemic to verify Claims 1a and 1b, respectively. Figures 2 and 3 show that Claims 1a and 1b hold under Scenario 1. Moreover, given alternative sets of proportions vaccinated and model parameters, Latin hypercube sampling verifies that Claim 1 holds under Scenario 1 (eAppendix 5). Under Scenario 2 where *β* increases, Claims 1a and 1b do not hold: Direct impacts are not lower bounds of overall impacts (Figures 2 and 3) due to negative indirect effects (eFigures 3 to 5). Under Scenario 3 where *μ* increases, Claims 1a and 1b hold for infections but not deaths due to negative indirect effects for death (eFigures 3 to 5). Under Scenario 4 where VEs wane, only Claim 1a (vaccine-averted outcomes) holds (Figure 2). Given different values for *α*1 (while holding constant *α* = 0) and alternative sets of model parameters, Latin hypercube sampling verifies that Claim 1a holds under Scenario 4 (eAppendix 5). However, Claim 1b (vaccine-avertible outcomes) does not hold (Figure 3) due to negative indirect effects (eFigures 4 and 5). Finally, under Scenario 5 where *β* increases and VEs wane, Claims 1a and 1b do not hold. ![Figure 2.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2024/11/11/2024.07.24.24310946/F2.medium.gif) [Figure 2.](http://medrxiv.org/content/early/2024/11/11/2024.07.24.24310946/F2) Figure 2. Direct impact and overall impact given *α* = 0 and *α*1 = 0. 7 to verify Claim 1a. Scenario 1, all parameters are time-invariant; Scenario 2, the number of effective contacts made by a typical infectious individual per day (*β*) increases from 0.15 to 0.6 at Day 300; Scenario 3, probability of infection-related death (*μ*) increases from 0.01 to 0.1 at Day 300; Scenario 4, vaccine efficacies against infection and death start to wane linearly after Day 100 reaching 0% at Day 300; and Scenario 5, the combination of Scenarios 2 and 4. ![Figure 3.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2024/11/11/2024.07.24.24310946/F3.medium.gif) [Figure 3.](http://medrxiv.org/content/early/2024/11/11/2024.07.24.24310946/F3) Figure 3. Direct impact and overall impact given *α*1 = 0. 7 and *α*2 = 0. 9 to verify Claim 1b. Scenario 1, all parameters are time-invariant; Scenario 2, the number of effective contacts made by a typical infectious individual per day (*β*) increases from 0.15 to 0.6 at Day 300; Scenario 3, probability of infection-related death (*μ*) increases from 0.01 to 0.1 at Day 300; Scenario 4, vaccine efficacies against infection and death start to wane linearly after Day 100 reaching 0% at Day 300; and Scenario 5, the combination of Scenarios 2 and 4. ## 6. DISCUSSION Motivated by recent research on estimating deaths averted by COVID-19 vaccination, this study adjusts estimands defined by Hudgens and Halloran to accommodate analyses on averted and avertible outcomes due to infectious disease interventions, thereby enabling researchers to distinguish the estimands when conducting and interpreting related studies. An epidemic model is simulated to verify the commonly held claim that outcomes averted (or avertible) via direct impact among the vaccinated (or unvaccinated) individuals is a lower bound of the overall impact on averted (or avertible) outcomes. Based on the SIRD model, we show that the lower bound fails when transmission or fatality parameter increases, or vaccine efficacies wane, implying that the lower bound is not guaranteed to hold for more general and realistic models. Consequently, it would be overly optimistic for empirical studies4–9 to assume that they have estimated a lower bound of true number of averted (or avertible) outcomes through estimating the direct impact. When indirect effects for the vaccinated and unvaccinated are both positive (that is, higher vaccination coverage yields positive number of averted or avertible outcomes), the direct impact is a lower bound of overall impact (equations 4.1 and 4.2). In general, the indirect effects may be positive because vaccination reduces the infectious individuals at a given time, such that infection-naïve individuals are less likely to be infected.29 However, as we have shown above, there are scenarios under which direct impact is not guaranteed to be a lower bound of overall impact due to negative indirect effects (Scenarios 2–5). First, when the number of effective contacts made by a typical infectious individual per day (*β*) increases over time (Scenario 2), overall impact on infection decreases (and can be negative) because at the early stage of outbreak, the less-vaccinated group has many infected and recovered with sterilizing immunity; while the more-vaccinated group has more susceptible (i.e., infection-naïve) individuals who have escaped the earlier infections and will experience a higher force of infection at the later time (eFigure 6). *β* is affected by probability of transmission per contact and number of contacts per day. Transmission probability depends on meteorological factors such as absolute humidity for the influenza A virus,24 behavioral factors such as usage of personal protective equipment,30 and biological factors such as changes in host immunity, and evolution of strains (a proper examination of the impact of co-circulating strains on the indirect effect requires modeling the cross immunity).31 The number of contacts can be affected by implementation or discontinuation of non-pharmaceutical interventions and seasonal variation in social contacts. Second, when the probability of infection-related death (*μ*) increases over time (Scenario 3), overall impact on death decreases because the extensively vaccinated group(s) has more who escape the earlier infections and then experience the high fatality at the later stage of outbreak (eFigure 6). It is plausible for lethality of pathogens to increase over time: Disease severity was found to increase in the autumn wave of the 1918 flu pandemic compared to the spring-summer wave of the same pathogen in that year. Some evidence from the A/H1N1 pandemic in 2009 also suggests that lethality of a subtype may increase in the second wave compared to the first wave.26 Increasing lethality implies that vaccination at the beginning of outbreak can lead to a negative overall impact by postponing cases. On the other hand, if the fatality rate increases with the infection peak due to the sudden shortage of healthcare resources, the overall impact on death will be more positive because vaccines delay infection and flatten the epidemic curve. Likewise, if infection-fatality rates decline progressively due to improvements in care,32,33 then vaccination that delays the epidemic can have amplified positive overall impact. Third, the overall impact of increased vaccination coverage decreases and may become negative when vaccine efficacies wane (Scenario 4). In particular, the vaccination proportion of *α*2 = 0.9 may result in more infections and deaths than the proportion of *α*1 = 0.7. eFigure 7 shows the epidemic curves given *α*1 = 0.7 and *α*2 = 0.9 proportions vaccinated. When *α*1 = 0.7, the epidemic peaks earlier and slows down due to the build-up of recovered individuals with sterilizing immunity. Consequently, return of full susceptibility (i.e., loss of protection) among vaccinated individuals is too late to rescue the epidemic. However, when *α*2 = 0.9, epidemic is delayed and a pool of individuals who are at risk of infection is built up, such that return of susceptibility among vaccinated individuals can rescue the epidemic. The current study has some limitations. First, vaccination is assumed to be a one-time event at baseline before the start of outbreak, but in reality, vaccine rollout is continuous over time and may occur during outbreak. A formalization of the causal effects under a time-varying regime (e.g., by specifying a sequence of proportions vaccinated ![Graphic][37] over time) will be left to future work. Second, throughout, we have considered the case where vaccination occurs at random. However, in most empirical settings there may be strong confounding due to staged rollout of vaccines as well as differences in vaccine acceptance by behavioral and health characteristics. Such confounding if uncontrolled threatens the validity of inferences about the magnitude of effects, whether or not a bound is valid in ideal (unconfounded) circumstances. Third, to illustrate the counterexamples to the Claims, the current study uses a stylized model with multiple simplifications. For example, this model does not consider multiple risk groups, which may have had heterogenous susceptibility to adverse outcomes and heterogeneous mixing patterns, leading to a negative indirect effect for the unvaccinated such that the Claims do not hold. Vaccination for a subgroup could cause negative indirect effect in other subgroups by increasing risk for more severe complications. For example, empirical evidence showed that low rubella vaccination coverage in children increased rubella incidence in the 15 years or over and incidence of congenital rubella in newborns.34 Mathematical modelling suggested that infant vaccination against varicella could increase the rate of reactivation (i.e., zoster) in the entire population.35,36 The examples suggest that the potential for negative indirect effects may be greater when subgroups differ in important properties, such as (in the rubella and varicella examples) risk for adverse outcomes. Another limitation of our model is that in the scenarios considered, changes in lethality and transmission were assumed to occur at a fixed time, whereas in reality they might well occur either in response to pathogen evolution37 or to behavioral changes that are affected by the epidemic trajectory. However, our goal was not to describe the details of a particular epidemic, but to describe qualitatively the conditions under which infectious disease interventions may not prevent more cases overall than directly. Finally, the SIRD model does not consider deaths due to other factors, meaning that simulation experiments based on this model are applicable to studies whose outcome of interest is a consequence of infection. Over a short time frame (e.g., 1 year), it is acceptable to consider deaths due to infection only, when other causes of deaths are negligible. We also did not consider possible adverse events after vaccination, although adverse events have important policy implications. Our focus here is on defining estimands for the protective impact of interventions on averting disease outcomes. Future studies can extend the estimands to investigate adverse events due to interventions. In conclusion, this study examines the commonly held assumption of empirical vaccine-averted and avertible analyses that direct impact is a lower bound on overall impact due to indirect protection. We show that the use of direct impact as a lower bound of overall impact in averted and avertible outcome studies is reliable only under very strong and often unrealistic assumptions. Therefore, empirical studies should not assume this relation because parameters may vary over time in reality. Alternatively, if researchers want to estimate the averted and avertible outcomes, a transmission model should be used to capture the overall impact, thereby improving clarity about what is estimated despite the expense of making additional modelling assumptions. ## Data Availability This is a simulation study with code available at [https://github.com/katjia/population\_level\_effects](https://github.com/katjia/population_level_effects). [https://github.com/katjia/population\_level\_effects](https://github.com/katjia/population_level_effects) ## eAppendix 1. Type A versus type B parameterizations In the main text, the potential outcomes ![Graphic][38] and ![Graphic][39] are defined under type A parameterization. eAppendix 1 will define type A and B parameterizations. We assume a large group size *N* such that results are equivalent under both types of parameterizations.1 Let *ρ*(***A******i*** = **a****i****;** *α*) denote the probability that group *i* receive allocation program **a****i** given parameter *α*. Definition 1 (A) According to VanderWeele and Tchetgen Tchetgen,1 a parameterization is said to be of type A with parameter *α* = (*N, K*) for group *i* if the allocation program ***A******i*** is randomly allocated conditional on ![Graphic][40] with probability mass function: ![Formula][41] **(B)** A parameterization is said to be of type B with parameter *α* if the allocation program ***A******i*** is randomly assigned to individuals in group *i* according to the known Bernoulli probability mass function: ![Formula][42] According to VanderWeele and Tchetgen Tchetgen,1 Hudgens and Halloran 2 defined causal effects under Type A parameterization. Definition 2 The *marginal individual average potential outcome* of Hudgens and Halloran 2 is: ![Formula][43] where ![Graphic][44] and *ρ*(***A******i*** = ***s*;** *α*) follows Definition 1 (A) under type A parameterization. The *individual average potential outcome* 2 is: ![Formula][45] where ![Graphic][46] and *ρ* (***A*** ***i*, −*j*** **= *s***, *A**ij* *= a; α*) follows Definition 1 (A) under type A parameterization. Alternatively, VanderWeele and Tchetgen Tchetgen1 proposed the definitions of potential outcomes under Type B parameterization. Definition 3 Under type B parameterization, the *marginal individual average potential utcome*1 is: ![Formula][47] where ![Graphic][48] and *ρ*(***A******i*** = ***s*;** *α*) follows Definition 1 (B) under type B parameterization. The *individual average potential outcome*1 is: ![Formula][49] where ![Graphic][50] and *ρ*(***A******i***,**−*j*** = ***s*;** *α*) follows Definition 1(B) under type B parameterization. ## eAppendix 2. Proof of Theorem 1 (overall effect partitioning) ![Formula][51] The first line follows by definition; the second follows from line 1 of proof of Theorem 3 from VanderWeele and Tchetgen Tchetgen1; the third distributes terms, the fourth adds and subtracts ![Graphic][52] and ![Graphic][53] the fifth and sixth rearrange terms; and the last applies definitions of *IE**vax*, *TE*, and *IE**Unvax*. The proof can also be shown visually as illustrated in Figure 1. ## eAppendix 3. The susceptible-infected-recovered-death (SIRD) model ### 1. Model structure eFigure 1 shows the flowchart for the transmission model described in equation (5) of the main text. ![eFigure 1.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2024/11/11/2024.07.24.24310946/F4.medium.gif) [eFigure 1.](http://medrxiv.org/content/early/2024/11/11/2024.07.24.24310946/F4) eFigure 1. Model flowchart. ### 2. Model parameters To verify Claim 1a, a pair of trajectories are simulated with same parameters except for the proportions vaccinated (*α* = 0 versus *α*1 = 0.7). Similarly, to verify Claim 1b, another pair of trajectories are simulated with *α*1 = 0.7 versus *α*2 = 0.9. Epidemic trajectories are simulated based on the SIRD model in equation (5)—first under Scenario 1 (i.e., time-invariant parameters) and then Scenarios 2 to 5 (i.e., time-dependent parameters). eTable 1 specifies parameter values for simulations. All models are implemented using R package *odin*,3 whereas the actual solution of the differential equations is conducted with the *deSolve* package using numerical solvers “lsoda” and “ode45.”4 View this table: [eTABLE 1.](http://medrxiv.org/content/early/2024/11/11/2024.07.24.24310946/T2) eTABLE 1. List of parameters for simulations View this table: [eTable 2.](http://medrxiv.org/content/early/2024/11/11/2024.07.24.24310946/T3) eTable 2. List of initial conditions in a group wherein the proportion vaccinated is *α* ## eAppendix 4. Proving Claim 1 under Scenario 1 in the SIRD model at *t* → ∞ ### 1. Definitions Let 0 ≤ *α* < *α*1 < *α*2 ≤ 1. Recall Claim 1: ![Formula][54] and ![Formula][55] By expanding equations (4.1) and (4.2) and suppressing the time notation (*t*), we have: ![Formula][56] for *α*1 > *α*, and ![Formula][57] for *α*2 > *α*1. We first give an intuitive explanation of *δ**D*(*α*, *α*1) being a lower bound of *δ**O*(*α*, *α*1). Recall that individuals fall into three categories based on vaccination status under the counterfactuals with *α* or *α*1 proportion vaccinated: 1) Those who are unvaccinated under both counterfactuals (referred to as “never-vaccinated” and represented by the dotted region in Figure 1); 2) those who are unvaccinated under *α* but vaccinated under *α*1 (referred to as “additionally-vaccinated” and represented by the gridded region in Figure 1); and 3) those who are vaccinated under both counterfactuals (referred to as “always-vaccinated” and represented by the stripped region in Figure 1). If we update risk of the “always-vaccinated” under *α* from ![Graphic][58] to ![Graphic][59] and update risk of the “never-vaccinated” under *α* from ![Graphic][60] to ![Graphic][61], then we have an updated right-hand side of equation (6.1): ![Formula][62] which is reduced to *δ**D*(*α*, *α*1) and is lower than the actual overall effect because the original risk ![Graphic][63] in the first term of the last line of equation (6.1) is higher than the updated ![Graphic][64] (to be proved in Section 3), and the original risk ![Graphic][65] in the last term of the last line of equation (6.1) is also higher than the updated ![Graphic][66] (to be proved in Section 3). Therefore, *δ**D*(*α*, *α*1) in the updated equation is lower than *δ**O*(*α*, *α*1) in the original equation (6.1) and Claim 1 holds for *α*1 > *α*. Similarly, for *α*2 > *α*1, if we update risk of “always-vaccinated” under *α*2 from ![Graphic][67] to ![Graphic][68] and update risk of “never-vaccinated” under *α*2 from ![Graphic][69] to ![Graphic][70], we have on the right-hand side of equation (6.2): ![Formula][71] which is reduced to *δ**D*(*α*2, *α*1) and is lower than the actual overall effect because the original risk ![Graphic][72] in the first term of the last line of equation (6.2) is lower than the updated ![Graphic][73] (to be proved in Section 3), and the original risk ![Graphic][74] in the last term is also lower than the updated ![Graphic][75] (to be proved in Section 3). Therefore, *δ**D*(*α*2, *α*1) in the updated equation is lower than *δ**O*(*α*1, *α*2) in the original equation (6.2) and Claim 1 holds for *α*2 > *α*1. Next, we will formally prove Claim 1 under Scenario 1 at *t* → ∞. ### 2. Sufficient Conditions for Claim 1 to hold Based on equation (4.1), Claim 1 holds for all *α* < *α*1 when *IE**vax*(*t, α*, *α*1) and *IE**Unvax*(*t, α*, *α*1) are both non-negative. Similarly, based on equation (4.2), Claim 1 holds for all *α*2 > *α*1 when *IE**vax*(*t, α*1, *α*2) and *IE**Unvax*(*t, α*1, *α*2) are both non-negative. ### 3. Proof of Claim 1 under Scenario 1 in the Susceptible-infected-recovered (SIR) Model at *t* → ∞ When probability of death due to infection is zero, the SIRD model in equation (5) reduces to a SIR model stratified by vaccination status. *IE**Unvax*(∞, *α*, *α*1) is positive for *α*1 > *α* in the SIR model by proving that ![Graphic][76], the final epidemic fraction in the unvaccinated individuals (i.e., the fraction of the unvaccinated individuals that became infected at the end of the outbreak), decreases with *α*. Similarly, *IE**vax*(∞, *α*, *α*1) is positive by proving that ![Graphic][77], the final epidemic fraction in the vaccinated individuals, also decreases with *α*. ### 3.1 Proof that ![Graphic][78]**and**![Graphic][79]**decrease with *α*** To prove that ![Graphic][80] and ![Graphic][81] decrease with the proportion vaccinated *α*, we prove that ![Graphic][82], the average final epidemic fraction, decreases with *α*. ![Graphic][83] is the weighted average of ![Graphic][84] and ![Graphic][85].5 Assuming that 1) the initial proportion susceptible is close to one, 2) the total group size *N* is near infinite, and 3) transmission events follow a Poisson process, the average final epidemic size is: ![Formula][86] where *θ* = 1 − *VE**infection*/100% given a leaky vaccine, and *R* is the basic reproduction number, defined as the expected number of new infections caused by a single infected individual during the infectious period in a completely susceptible population. Define two separate functions with independent arguments, *Z* and *α* (Miller5 used a similar type of argument to arrive at the final epidemic size): ![Formula][87] ![Formula][88] Here the equality LHS(⋅) = RHS(⋅) is not assumed; in particular, RHS(⋅) takes two independent arguments, *Z* and *α*. Plot LHS(*Z*) and RHS(*Z, α*) in one graph, where *R* = 4, *θ* = 0.2, and *α* = 0, 0.3 and 0.7. ![Figure5](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2024/11/11/2024.07.24.24310946/F5.medium.gif) [Figure5](http://medrxiv.org/content/early/2024/11/11/2024.07.24.24310946/F5) LHS(*Z*) = RHS(*Z, α*) when *Z* = 0, which is a trivial solution. We will now prove that if a positive (non-trivial) root exists such that LHS(*Z*) = RHS(*Z, α*), the value of this root decreases when *α* increases. We first show that the first derivative of RHS(*Z, α*) w.r.t *Z* at zero is greater than 1, which is the derivative of LHS(*Z*). That is: ![Formula][89] where *R**e* is the effective reproduction number when proportion *α* of the group is vaccinated with a leaky vaccine.6 The above inequality holds because *R**e* must be greater than 1 in order for any epidemic to occur, meaning that for some *ϵ* > 0, RHS(*ϵ, α*) > *LHS*(*ϵ*). In other words, RHS(*Z, α*) exceeds LHS(*Z*) for small values of *Z*. Next, we prove that RHS(*Z, α*) is a concave function by computing the second derivative w.r.t *Z*: ![Formula][90] for *α* ∈ [0,1]. Because the second derivative is negative for *α* ∈ [0,1], RHS(*Z, α*) is a concave function of *Z* for any possible values of *α*. Taken together, the first and second derivatives indicate that RHS(*Z, α*) is above LHS(*Z*) when *Z* is greater than but close to zero, and RHS(*Z, α*) bends towards LHS(*Z*) (i.e., the diagonal) and by assumption (that there is a nontrivial root of LHS(*Z*) = RHS(*Z, α*)) coincides with the diagonal at some value of *Z* (See plot above). Finally, consider two values *α*1 > *α* giving rise to two curves RHS(*Z, α* = *α*) and RHS(*Z, α* = *α*1). Let *α*1 = 0.7 and *α* = 0.3 as the plot illustrates. Because 0 < *θ* < 1, [*exp*(−*θR**Z*) − *exp*(−*R**Z*)] in equation (9) is always positive at any given *Z* > 0. Therefore, RHS(*Z, α* = *α*) > RHS(*Z, α* = *α*1) at any given *Z* > 0 when *α*1 > *α*. Suppose the upper curve RHS(*Z, α*) meets LHS(*Z*) at *Z* = *z*. Then RHS(*z*, *α*1) < *z* = *LHS*(*z*). Given that all expressions are continuous and that RHS(*ϵ, α*) > *LHS*(*ϵ*), there must have been some value *z*1 to the left of *z* where RHS(*z*1, *α*1) crossed LHS(*z*1). In other words, RHS(*z*1, *α*1) = LHS(*z*1) for some value *z*1 < *z*, proving that the root *z*1 of RHS(*z*1, *α*1) = LHS(*z*1) is less than the root *z* of RHS(*z*, *α*) = LHS(*z*) for *α*1 > *α*, QED. These non-trivial roots are unique because RHS(*Z, α*) is concave. Any value *Z* that satisfies LHS(*Z*) = RHS(*Z, α*) will satisfy equation (7). Suppose *z* is a positive root for LHS(*Z*) = RHS(*Z, α* = *α*) such that ![Graphic][91], and similarly suppose *z*1 is a positive root for LHS(*Z*) = RHS(*Z, α* = *α*1) such that ![Graphic][92]. Therefore, we have ![Graphic][93] when *α*1 > *α*. Given that ![Graphic][94],5 ![Graphic][95] decreases with *α* because ![Graphic][96] decreases with *α*. Similarly, given that ![Graphic][97],5 ![Graphic][98] decreases with *α*. ### 3.2 Proof of *IE**Unvax*(∞, *α*, *α*1) > 0 and *IE**Vax*(∞, *α*, *α*1) > 0 for *α*1 > *α* In Section 3.1, we proved that ![Graphic][99] decreases with *α*. Therefore, ![Graphic][100] for *α*1 > *α*. We also proved that *π*1,*α* (∞) decreases with *α*, and therefore ![Graphic][101] for *α*1 > *α*. Since *IE**Unvax*(∞, *α*, *α*1) and *IE**vax*(∞, *α*, *α*1) are positive, Claim 1 holds for any *α*1 > *α* at *t* → ∞. The proof in Section 3 also applies to any *α*2 > *α*1 in showing that *IIE**Unvax*(∞, *α*1, *α*2) and *IIE**vax*(∞, *α*1, *α*2) are positive, such that Claim 1 holds under Scenario 1 in the Susceptible-infected-recovered (SIR) Model at *t* → ∞. ## 4. Proof of Claim 1 under Scenario 1 in the Susceptible-infected-recovered-death (SIRD) Model at *t* → ∞ Now consider the SIRD model with non-zero probability of death due to infection. Based on the transmission dynamics in equation (5), those who died of infection are fixed proportion of those who became infected at the end of epidemic. We have: ![Formula][102] for the unvaccinated individuals and ![Formula][103] for the vaccinated individuals. For the unvaccinated individuals, by substituting the right-hand side of equation (10.1) into the equation of *IE**Unvax,death*, we have: ![Formula][104] We show that *IE**Unvax,death*(∞, *α*, *α*1) has the same direction as *IE**Unvax,infection*(∞, *α*, *α*1). If *IE**Unvax,infection*(∞, *α*, *α*1) is positive (Section 3), so is *IE**Unvax,death*(∞, *α*, *α*1). Similarly, for the vaccinated individuals, by substituting the right-hand side of equation (10.2) into the equation of *IE**vax,death*, we have: ![Formula][105] If *IE**vax,infection*(∞, *α*, *α*1) is positive (Section 3), so is *IE**vax,death*(∞, *α*, *α*1). In sum, *IE**vax*(∞, *α*, *α*1) and *IE**Unvax*(∞, *α*, *α*1) are positive for both death and infection given *α*1 > *α* (similarly, *IE**vax*(∞, *α*1, *α*2) and *IE**Unvax*(∞, *α*1, *α*2) are positive for *α*2 > *α*1). Therefore, Claim 1 holds under Scenario 1 in the Susceptible-infected-recovered-death (SIRD) model at *t* → ∞. ## eAppendix 5. Latin hypercube sampling to verify Claim 1 under Scenario 1 and Claim 1a under Scenario 4 ### 1. Verifying Claim 1 under Scenario 1 As Figures 2 and 3 show, Claims 1a and 1b hold under Scenario 1 given the particular proportions vaccinated (*α* = 0 versus *α*1 = 0.7 for Claim 1a and *α*1 = 0.7 versus *α*2 = 0.9 for Claim 1b) and parameter values specified in eTable 1. To test the robustness of Claim 1 under alternative parameter combinations, Latin hypercube sampling (LHS) is used to randomly draw 1,000 sets of proportions vaccinated and parameters. Initial conditions are same as in eTable 2. LHS samples are first generated from uniform distributions using the *lhs* package7 and then scaled to match the range of each parameter as shown in eTable 3. View this table: [eTable 3.](http://medrxiv.org/content/early/2024/11/11/2024.07.24.24310946/T4) eTable 3. Parameter ranges for Latin hypercube sampling under Scenarios 1 and 4 To exclude results due to roundoff errors during numerical integration of the differential equations, Claim 1 is disproven based on the criteria *δ**O* − *δ**D* < −10−7at any time *t*, rather than the conventional criteria of < 0. From the LHS procedure, Claim 1 holds under Scenario 1 for all the 1,000 iterations. eFigure 2 shows direct impact and overall impact for 50 random LHS samples. ### 2. Verifying Claim 1a under Scenario 4 Figure 2 also shows that Claim 1a holds under Scenario 4 given the parameter values specified in eTable 1. Similarly, LHS is conducted to verify if Claim 1a holds under Scenario 4 with different values for *α*1 (while holding constant *α* = 0) and alternative sets of model parameters; parameter ranges are specified in eTable 3. From the LHS procedure, Claim 1a holds under Scenario 4 for all the 1,000 iterations (eFigure 2). ![eFigure 2.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2024/11/11/2024.07.24.24310946/F6.medium.gif) [eFigure 2.](http://medrxiv.org/content/early/2024/11/11/2024.07.24.24310946/F6) eFigure 2. Direct impact and overall impact to verify Claim 1 under Scenario 1 and to verify Claim 1a under Scenario 4 from 50 Latin hypercube samples. **eAppendix 6. Trajectories of indirect effects for the vaccinated and unvaccinated**. Based on equation (4.1), Claim 1a does not hold when *IE**Unvax*(*t*, 0, 0.7) is negative, as shown in eFigure 3. Similarly, based on equation (4.2), Claim 1b may not hold when *IE**Unvax*(*t*, 0.7,0.9) or *IE**vax*(*t*, 0.7,0.9) is negative, as shown in eFigures 4 and 5, respectively. ![eFigure 3.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2024/11/11/2024.07.24.24310946/F7.medium.gif) [eFigure 3.](http://medrxiv.org/content/early/2024/11/11/2024.07.24.24310946/F7) eFigure 3. Indirect effect for the unvaccinated *IE**Unvax*(*t*, 0, 0. 7). Scenario 1, all parameters are time-invariant; Scenario 2, the number of effective contacts made by a typical infectious individual per day (*β*) increases from 0.15 to 0.6 at Day 300; Scenario 3, probability of infection-related death (*μ*) increases from 0.01 to 0.1 at Day 300; Scenario 4, vaccine efficacies against infection and death start to wane linearly after Day 100 reaching 0% at Day 300; and Scenario 5, the combination of Scenarios 2 and 4. ![eFigure 4.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2024/11/11/2024.07.24.24310946/F8.medium.gif) [eFigure 4.](http://medrxiv.org/content/early/2024/11/11/2024.07.24.24310946/F8) eFigure 4. Indirect effect for the unvaccinated *IE**Unvax*(*t*, 0. 7, 0. 9). ![eFigure 5.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2024/11/11/2024.07.24.24310946/F9.medium.gif) [eFigure 5.](http://medrxiv.org/content/early/2024/11/11/2024.07.24.24310946/F9) eFigure 5. Indirect effect for the vaccinated *IE**Vax*(*t*, 0. 7, 0. 9). ## eAppendix 7. Trajectories of proportion susceptible by vaccination status over time ![eFigure 6.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2024/11/11/2024.07.24.24310946/F10.medium.gif) [eFigure 6.](http://medrxiv.org/content/early/2024/11/11/2024.07.24.24310946/F10) eFigure 6. Proportion susceptible among vaccinated and unvaccinated individuals under 0%, 70%, or 90% proportions vaccinated. Scenario 1, all parameters are time-invariant; Scenario 2, the number of effective contacts made by a typical infectious individual per day (*β*) increases from 0.15 to 0.6 at Day 300; Scenario 3, probability of infection-related death (*μ*) increases from 0.01 to 0.1 at Day 300; Scenario 4, vaccine efficacies against infection and death start to wane linearly after Day 100 reaching 0% at Day 300; and Scenario 5, the combination of Scenarios 2 and 4. ## eAppendix 8. Epidemic curves under Scenario 4 ![eFigure 7.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2024/11/11/2024.07.24.24310946/F11.medium.gif) [eFigure 7.](http://medrxiv.org/content/early/2024/11/11/2024.07.24.24310946/F11) eFigure 7. Epidemic curves under Scenario 4 given *α*1 = 0. 7 and *α*2 = 0. 9 proportions vaccinated. The dashed line represents the number of effectively susceptible persons = *S**u*(*t*) + *θ*(*t*) ⋅ *S**v*(*t*) where *S**u*(*t*) is the number of unvaccinated susceptibles and *θ*(*t*) ⋅ *S**v*(*t*) is the number of vaccinated susceptibles multiplied with *θ*(*t*) = 1 − *VE**infection*(*t*)/100%. The dashed lines show that more individuals at risk of infection (weighted by that risk) are built up under *α*2 = 0.9 than *α*1 = 0.7 before the epidemic peaks. ## Footnotes * Methods and results remain the same. Abstract, introduction, and conclusion have been revised to highlight the contributions and update the key takeaways. * Received July 24, 2024. * Revision received November 11, 2024. * Accepted November 11, 2024. * © 2024, Posted by Cold Spring Harbor Laboratory This pre-print is available under a Creative Commons License (Attribution-NonCommercial 4.0 International), CC BY-NC 4.0, as described at [http://creativecommons.org/licenses/by-nc/4.0/](http://creativecommons.org/licenses/by-nc/4.0/) ## REFERENCES 1. 1.Vilches TN, Moghadas SM, Sah P, et al. Estimating COVID-19 Infections, Hospitalizations, and Deaths Following the US Vaccination Campaigns During the Pandemic. JAMA Netw Open. 2022;5(1):e2142725. doi:10.1001/jamanetworkopen.2021.42725 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jamanetworkopen.2021.42725&link_type=DOI) 2. 2.Schneider EC, Shah A, Sah P, et al. Impact of U.S. COVID-19 Vaccination Efforts: An Update on Averted Deaths, Hospitalizations, and Health Care Costs Through March 2022. To the Point. 2022. Available at: doi:10.26099/d3dm-fa91. Accessed December 10, 2023. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.26099/d3dm-fa91&link_type=DOI) 3. 3.Gavish N, Yaari R, Huppert A, Katriel G. Population-level implications of the Israeli booster campaign to curtail COVID-19 resurgence. Sci Transl Med. 2022;14(647):eabn9836. doi:10.1126/scitranslmed.abn9836 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1126/scitranslmed.abn9836&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=35412326&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F11%2F11%2F2024.07.24.24310946.atom) 4. 4.Haas EJ, McLaughlin JM, Khan F, et al. Infections, hospitalisations, and deaths averted via a nationwide vaccination campaign using the Pfizer–BioNTech BNT162b2 mRNA COVID-19 vaccine in Israel: a retrospective surveillance study. The Lancet Infectious Diseases. 2022;22(3):357–366. doi:10.1016/S1473-3099(21)00566-1 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S1473-3099(21)00566-1&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=34562375&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F11%2F11%2F2024.07.24.24310946.atom) 5. 5.Brault A, Hart A, Uribe P, et al. Direct impact of COVID-19 vaccination in Chile: averted cases, hospitalizations, ICU admissions, and deaths. BMC Infect Dis. 2024;24(1):467. doi:10.1186/s12879-024-09304-1 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/s12879-024-09304-1&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=38698324&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F11%2F11%2F2024.07.24.24310946.atom) 6. 6.Santos CVBD, Noronha TGD, Werneck GL, Struchiner CJ, Villela DAM. Estimated COVID-19 severe cases and deaths averted in the first year of the vaccination campaign in Brazil: a retrospective observational study. The Lancet Regional Health - Americas. 2023;17:100418. doi:10.1016/j.lana.2022.100418 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.lana.2022.100418&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=36575682&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F11%2F11%2F2024.07.24.24310946.atom) 7. 7.Kayano T, Sasanami M, Kobayashi T, et al. Number of averted COVID-19 cases and deaths attributable to reduced risk in vaccinated individuals in Japan. The Lancet Regional Health - Western Pacific. 2022;28:100571. doi:10.1016/j.lanwpc.2022.100571 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.lanwpc.2022.100571&link_type=DOI) 8. 8.Jia KM, Hanage WP, Lipsitch M, et al. Estimated preventable COVID-19-associated deaths due to non-vaccination in the United States. Eur J Epidemiol. 2023; 38(11):1125–1128. doi:10.1007/s10654-023-01006-3 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s10654-023-01006-3&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=37093505&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F11%2F11%2F2024.07.24.24310946.atom) 9. 9.Zhong M, Glazer T, Kshirsagar M, et al. Estimating Vaccine-Preventable COVID-19 Deaths Among Adults Under Counterfactual Vaccination Scenarios in The United States: A Modeling Study Using Observational Data. J Pharm Pharmacol Res. 2023;07(03). doi:10.26502/fjppr.079 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.26502/fjppr.079&link_type=DOI) 10. 10.Halloran ME, Struchiner CJ. Study Designs for Dependent Happenings: Epidemiology. 1991;2(5):331–338. doi:10.1097/00001648-199109000-00004 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1097/00001648-199109000-00004&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=1742381&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F11%2F11%2F2024.07.24.24310946.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1991GL56600004&link_type=ISI) 11. 11.VanderWeele TJ, Tchetgen Tchetgen EJ. Effect partitioning under interference in two-stage randomized vaccine trials. Statistics & Probability Letters. 2011;81(7):861–869. doi:10.1016/j.spl.2011.02.019 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.spl.2011.02.019&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=21532912&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F11%2F11%2F2024.07.24.24310946.atom) 12. 12.Hudgens MG, Halloran ME. Toward Causal Inference With Interference. Journal of the American Statistical Association. 2008;103(482):832–842. doi:10.1198/016214508000000292 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1198/016214508000000292&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=19081744&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F11%2F11%2F2024.07.24.24310946.atom) 13. 13.Halloran ME, Struchiner CJ, Longini IM. Study Designs for Evaluating Different Efficacy and Effectiveness Aspects of Vaccines. American Journal of Epidemiology. 1997;146(10):789–803. doi:10.1093/oxfordjournals.aje.a009196 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/oxfordjournals.aje.a009196&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=9384199&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F11%2F11%2F2024.07.24.24310946.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1997YF88200001&link_type=ISI) 14. 14.Polack FP, Thomas SJ, Kitchin N, et al. Safety and Efficacy of the BNT162b2 mRNA Covid-19 Vaccine. N Engl J Med. 2020;383(27):2603–2615. doi:10.1056/NEJMoa2034577 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1056/NEJMoa2034577&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=33301246&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F11%2F11%2F2024.07.24.24310946.atom) 15. 15.Frenck RW, Klein NP, Kitchin N, et al. Safety, Immunogenicity, and Efficacy of the BNT162b2 Covid-19 Vaccine in Adolescents. N Engl J Med. 2021;385(3):239–250. doi:10.1056/NEJMoa2107456 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1056/NEJMoa2107456&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=34043894&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F11%2F11%2F2024.07.24.24310946.atom) 16. 16.Standaert B, Dort T, Toumi M. Vaccine Efficacy, Effectiveness, Or Impact: Which One To Choose In Economic Evaluations Of Vaccines? Value in Health. 2017;20(9):A754. doi:10.1016/j.jval.2017.08.2115 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.jval.2017.08.2115&link_type=DOI) 17. 17.Chakladar S, Rosin S, Hudgens MG, et al. Inverse probability weighted estimators of vaccine effects accommodating partial interference and censoring. Biometrics. 2022;78(2):777–788. doi:10.1111/biom.13459 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/biom.13459&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=33768557&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F11%2F11%2F2024.07.24.24310946.atom) 18. 18.Kayano T, Nishiura H. Assessing the COVID-19 vaccination program during the Omicron variant (B.1.1.529) epidemic in early 2022, Tokyo. BMC Infect Dis. 2023;23(1):748. doi:10.1186/s12879-023-08748-1 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/s12879-023-08748-1&link_type=DOI) 19. 19.Watson OJ, Barnsley G, Toor J, Hogan AB, Winskill P, Ghani AC. Global impact of the first year of COVID-19 vaccination: a mathematical modelling study. The Lancet Infectious Diseases. 2022;22(9):1293–1302. doi:10.1016/S1473-3099(22)00320-6 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S1473-3099(22)00320-6&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=35753318&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F11%2F11%2F2024.07.24.24310946.atom) 20. 20.Shoukat A, Vilches TN, Moghadas SM, et al. Lives saved and hospitalizations averted by COVID-19 vaccination in New York City: a modeling study. The Lancet Regional Health - Americas. 2022;5:100085. doi:10.1016/j.lana.2021.100085 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.lana.2021.100085&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=34746912&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F11%2F11%2F2024.07.24.24310946.atom) 21. 21.Magosi LE, Zhang Y, Golubchik T, et al. Deep-sequence phylogenetics to quantify patterns of HIV transmission in the context of a universal testing and treatment trial – BCPP/Ya Tsie trial. eLife. 2022;11:e72657. doi:10.7554/eLife.72657 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.7554/eLife.72657&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=35229714&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F11%2F11%2F2024.07.24.24310946.atom) 22. 22.R Core Team. R: A Language And Environment for Statistical Computing. Accessed May 31, 2024. [https://www.R-project.org/](https://www.R-project.org/) 23. 23.FitzJohn R. odin: ODE Generation and Integration. Published online 2024. [https://github.com/mrc-ide/odin](https://github.com/mrc-ide/odin) 24. 24.Shaman J, Kohn M. Absolute humidity modulates influenza survival, transmission, and seasonality. Proc Natl Acad Sci USA. 2009;106(9):3243–3248. doi:10.1073/pnas.0806852106 [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoicG5hcyI7czo1OiJyZXNpZCI7czoxMDoiMTA2LzkvMzI0MyI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDI0LzExLzExLzIwMjQuMDcuMjQuMjQzMTA5NDYuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 25. 25.Li Y, Campbell H, Kulkarni D, et al. The temporal association of introducing and lifting non-pharmaceutical interventions with the time-varying reproduction number (R) of SARS-CoV-2: a modelling study across 131 countries. The Lancet Infectious Diseases. 2021;21(2):193–202. doi:10.1016/S1473-3099(20)30785-4 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S1473-3099(20)30785-4&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=33729915&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F11%2F11%2F2024.07.24.24310946.atom) 26. 26.Simonsen L, Chowell G, Andreasen V, et al. A review of the 1918 herald pandemic wave: importance for contemporary pandemic response strategies. Annals of Epidemiology. 2018;28(5):281–288. doi:10.1016/j.annepidem.2018.02.013 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.annepidem.2018.02.013&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=29530388&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F11%2F11%2F2024.07.24.24310946.atom) 27. 27.Chemaitelly H, Tang P, Hasan MR, et al. Waning of BNT162b2 Vaccine Protection against SARS-CoV-2 Infection in Qatar. N Engl J Med. 2021;385(24). doi:10.1056/NEJMoa2114114 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1056/NEJMoa2114114&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=34614327&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F11%2F11%2F2024.07.24.24310946.atom) 28. 28.Menegale F, Manica M, Zardini A, et al. Evaluation of Waning of SARS-CoV-2 Vaccine–Induced Immunity: A Systematic Review and Meta-analysis. JAMA Netw Open. 2023;6(5):e2310650. doi:10.1001/jamanetworkopen.2023.10650 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jamanetworkopen.2023.10650&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=37133863&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F11%2F11%2F2024.07.24.24310946.atom) 29. 29.Pitman R, Fisman D, Zaric GS, et al. Dynamic Transmission Modeling: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force Working Group–5. Med Decis Making. 2012;32(5):712–721. doi:10.1177/0272989X12454578 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/0272989X12454578&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=22990086&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F11%2F11%2F2024.07.24.24310946.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000310163100009&link_type=ISI) 30. 30.Leung NHL, Chu DKW, Shiu EYC, et al. Respiratory virus shedding in exhaled breath and efficacy of face masks. Nat Med. 2020;26(5):676–680. doi:10.1038/s41591-020-0843-2 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1038/s41591-020-0843-2&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32371934&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F11%2F11%2F2024.07.24.24310946.atom) 31. 31.Bushman M, Kahn R, Taylor BP, Lipsitch M, Hanage WP. Population impact of SARS-CoV-2 variants with enhanced transmissibility and/or partial immune escape. Cell. 2021;184(26):6229-6242.e18. doi:10.1016/j.cell.2021.11.026 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.cell.2021.11.026&link_type=DOI) 32. 32.Horwitz LI, Jones SA, Cerfolio RJ, et al. Trends in COVID-19 Risk-Adjusted Mortality Rates. Journal of Hospital Medicine. 2021;16(2):90–92. doi:10.12788/jhm.3552 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.12788/jhm.3552&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=33147129&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F11%2F11%2F2024.07.24.24310946.atom) 33. 33.Dennis JM, McGovern AP, Vollmer SJ, Mateen BA. Improving Survival of Critical Care Patients With Coronavirus Disease 2019 in England: A National Cohort Study, March to June 2020. Critical Care Medicine. 2021;49(2):209–214. doi:10.1097/CCM.0000000000004747 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1097/CCM.0000000000004747&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=33105150&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F11%2F11%2F2024.07.24.24310946.atom) 34. 34.Panagiotopoulos T, Antoniadou I, Valassi-Adam E. Increase in congenital rubella occurrence after immunisation in Greece: retrospective survey and systematic review. BMJ. 1999 Dec 4;319(7223):1462–7. doi: 10.1136/bmj.319.7223.1462. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEzOiIzMTkvNzIyMy8xNDYyIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjQvMTEvMTEvMjAyNC4wNy4yNC4yNDMxMDk0Ni5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 35. 35.Brisson M, Edmunds WJ, Gay NJ. Varicella vaccination: Impact of vaccine efficacy on the epidemiology of VZV. J Med Virol. 2003;70(S1):S31-S37. doi:10.1002/jmv.10317 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/jmv.10317&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=12627484&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F11%2F11%2F2024.07.24.24310946.atom) 36. 36.Brisson M, Gay NJ, Edmunds WJ, Andrews NJ. Exposure to varicella boosts immunity to herpes-zoster: implications for mass vaccination against chickenpox. Vaccine. 2002;20(19-20):2500-2507. doi:10.1016/S0264-410X(02)00180-9 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S0264-410X(02)00180-9&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=12057605&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F11%2F11%2F2024.07.24.24310946.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000176456200010&link_type=ISI) 37. 37.Markov PV, Ghafari M, Beer M, et al. The evolution of SARS-CoV-2. Nat Rev Microbiol. 2023;21(6):361–379. doi:10.1038/s41579-023-00878-2 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1038/s41579-023-00878-2&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=37020110&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F11%2F11%2F2024.07.24.24310946.atom) ## References for eAppendix 1. 1.VanderWeele TJ, Tchetgen Tchetgen EJ. Effect partitioning under interference in two-stage randomized vaccine trials. Statistics & Probability Letters. 2011;81(7):861–869. doi:10.1016/j.spl.2011.02.019 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.spl.2011.02.019&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=21532912&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F11%2F11%2F2024.07.24.24310946.atom) 2. 2.Hudgens MG, Halloran ME. Toward Causal Inference With Interference. Journal of the American Statistical Association. 2008;103(482):832–842. doi:10.1198/016214508000000292 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1198/016214508000000292&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=19081744&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F11%2F11%2F2024.07.24.24310946.atom) 3. 3.FitzJohn R. odin: ODE Generation and Integration. Published online 2024. [https://github.com/mrc-ide/odin](https://github.com/mrc-ide/odin) 4. 4.Soetaert K, Petzoldt T, R. Woodrow S. Solving Differential Equations in R: Package deSolve. Journal of Statistical Software. 2010;33(9):1–25. doi:10.18637/jss.v033.i09 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.18637/jss.v033.i09&link_type=DOI) 5. 5.Miller JC. A Note on the Derivation of Epidemic Final Sizes. Bull Math Biol. 2012;74(9):2125–2141. doi:10.1007/s11538-012-9749-6 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s11538-012-9749-6&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=22829179&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F11%2F11%2F2024.07.24.24310946.atom) 6. 6.Chao DL, Dimitrov DT. Seasonality and the effectiveness of mass vaccination. Mathematical Biosciences and Engineering. 2016;13(2):249–259. doi:10.3934/mbe.2015001 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3934/mbe.2015001&link_type=DOI) 7. 7.Carnell R. lhs: Latin Hypercube Samples. Published online 2022. [https://CRAN.R-project.org/package=lhs](https://CRAN.R-project.org/package=lhs) [1]: /embed/graphic-1.gif [2]: /embed/graphic-2.gif [3]: /embed/inline-graphic-1.gif [4]: /embed/inline-graphic-2.gif [5]: /embed/inline-graphic-3.gif [6]: /embed/inline-graphic-4.gif [7]: /embed/inline-graphic-5.gif [8]: /embed/inline-graphic-6.gif [9]: /embed/inline-graphic-7.gif [10]: /embed/inline-graphic-8.gif [11]: /embed/inline-graphic-9.gif [12]: /embed/inline-graphic-10.gif [13]: /embed/inline-graphic-11.gif [14]: /embed/inline-graphic-12.gif [15]: /embed/inline-graphic-13.gif [16]: /embed/inline-graphic-14.gif [17]: /embed/inline-graphic-15.gif [18]: /embed/inline-graphic-16.gif [19]: /embed/inline-graphic-17.gif [20]: /embed/graphic-3.gif [21]: /embed/inline-graphic-18.gif [22]: /embed/inline-graphic-19.gif [23]: /embed/graphic-5.gif [24]: /embed/graphic-6.gif [25]: /embed/graphic-7.gif [26]: /embed/graphic-8.gif [27]: /embed/graphic-9.gif [28]: /embed/graphic-10.gif [29]: /embed/graphic-11.gif [30]: /embed/graphic-12.gif [31]: /embed/graphic-13.gif [32]: /embed/graphic-14.gif [33]: /embed/graphic-15.gif [34]: /embed/graphic-16.gif [35]: /embed/graphic-17.gif [36]: /embed/inline-graphic-20.gif [37]: /embed/inline-graphic-21.gif [38]: /embed/inline-graphic-22.gif [39]: /embed/inline-graphic-23.gif [40]: /embed/inline-graphic-24.gif [41]: /embed/graphic-21.gif [42]: /embed/graphic-22.gif [43]: /embed/graphic-23.gif [44]: /embed/inline-graphic-25.gif [45]: /embed/graphic-24.gif [46]: /embed/inline-graphic-26.gif [47]: /embed/graphic-25.gif [48]: /embed/inline-graphic-27.gif [49]: /embed/graphic-26.gif [50]: /embed/inline-graphic-28.gif [51]: /embed/graphic-27.gif [52]: /embed/inline-graphic-29.gif [53]: /embed/inline-graphic-30.gif [54]: /embed/graphic-33.gif [55]: /embed/graphic-34.gif [56]: /embed/graphic-35.gif [57]: /embed/graphic-36.gif [58]: /embed/inline-graphic-31.gif [59]: /embed/inline-graphic-32.gif [60]: /embed/inline-graphic-33.gif [61]: /embed/inline-graphic-34.gif [62]: /embed/graphic-37.gif [63]: /embed/inline-graphic-35.gif [64]: /embed/inline-graphic-36.gif [65]: /embed/inline-graphic-37.gif [66]: /embed/inline-graphic-38.gif [67]: /embed/inline-graphic-39.gif [68]: /embed/inline-graphic-40.gif [69]: /embed/inline-graphic-41.gif [70]: /embed/inline-graphic-42.gif [71]: /embed/graphic-38.gif [72]: /embed/inline-graphic-43.gif [73]: /embed/inline-graphic-44.gif [74]: /embed/inline-graphic-45.gif [75]: /embed/inline-graphic-46.gif [76]: /embed/inline-graphic-47.gif [77]: /embed/inline-graphic-48.gif [78]: /embed/inline-graphic-49.gif [79]: /embed/inline-graphic-50.gif [80]: /embed/inline-graphic-51.gif [81]: /embed/inline-graphic-52.gif [82]: /embed/inline-graphic-53.gif [83]: /embed/inline-graphic-54.gif [84]: /embed/inline-graphic-55.gif [85]: /embed/inline-graphic-56.gif [86]: /embed/graphic-39.gif [87]: /embed/graphic-40.gif [88]: /embed/graphic-41.gif [89]: /embed/graphic-43.gif [90]: /embed/graphic-44.gif [91]: /embed/inline-graphic-57.gif [92]: /embed/inline-graphic-58.gif [93]: /embed/inline-graphic-59.gif [94]: /embed/inline-graphic-60.gif [95]: /embed/inline-graphic-61.gif [96]: /embed/inline-graphic-62.gif [97]: /embed/inline-graphic-63.gif [98]: /embed/inline-graphic-64.gif [99]: /embed/inline-graphic-65.gif [100]: /embed/inline-graphic-66.gif [101]: /embed/inline-graphic-67.gif [102]: /embed/graphic-45.gif [103]: /embed/graphic-46.gif [104]: /embed/graphic-47.gif [105]: /embed/graphic-48.gif