The Impact of Vaccination to Control COVID-19 Burden in the United States: A Simulation Modeling Approach ========================================================================================================= * Oguzhan Alagoz * Ajay K. Sethi * Brian W. Patterson * Matthew Churpek * Ghalib Alhanaee * Elizabeth Scaria * Nasia Safdar ## ABSTRACT **Introduction** Vaccination programs aim to control the COVID-19 pandemic. However, the relative impacts of vaccine coverage, effectiveness, and capacity in the context of nonpharmaceutical interventions such as mask use and physical distancing on the spread of SARS-CoV-2 are unclear. Our objective was to examine the impact of vaccination on the control of SARS-CoV-2 using our previously developed agent-based simulation model. **Methods** We applied our agent-based model to replicate COVID-19-related events in 1) Dane County, Wisconsin; 2) Milwaukee metropolitan area, Wisconsin; 3) New York City (NYC). We evaluated the impact of vaccination considering the proportion of the population vaccinated, probability that a vaccinated individual gains immunity, vaccination capacity, and adherence to nonpharmaceutical interventions. The primary outcomes were the number of confirmed COVID-19 cases and the timing of pandemic control, defined as the date after which only a small number of new cases occur. We also estimated the number of cases without vaccination. **Results** The timing of pandemic control depends highly on vaccination coverage, effectiveness, and adherence to nonpharmaceutical interventions. In Dane County and Milwaukee, if 50% of the population is vaccinated with a daily vaccination capacity of 0.1% of the population, vaccine effectiveness of 90%, and the adherence to nonpharmaceutical interventions is 65%, controlled spread could be achieved by July 2021 and August 2021, respectively versus in March 2022 in both regions without vaccine. If adherence to nonpharmaceutical interventions increases to 70%, controlled spread could be achieved by May 2021 and April 2021 in Dane County and Milwaukee, respectively. **Discussion** In controlling the spread of SARS-CoV-2, the impact of vaccination varies widely depending not only on effectiveness and coverage, but also concurrent adherence to nonpharmaceutical interventions. The effect of SARS-CoV-2 variants was not considered. **Primary Funding Source** National Institute of Allergy and Infectious Diseases Key Words * SARS-CoV-2 * COVID-19 * novel coronavirus * agent-based simulation modeling * herd immunity * vaccination * nonpharmaceutical interventions * social distancing measures ## INTRODUCTION With over 27 million cases in the US alone, poorly controlled transmission of SARS-CoV-2 has challenged the capacity of health systems and has resulted in over 460,000 deaths.1 Until recently, the only effective measures for controlling the spread of SARS-CoV-2 have been nonpharmaceutical interventions (NPIs), such as physical distancing and masking. Sustained adherence to these NPIs is variable and difficult to achieve.2 The US Food and Drug Administration recently authorized the use of two, two-dose vaccines and one, single-dose vaccine against SARS-CoV-2 of similarly high efficacy to prevent symptoms of novel coronavirus disease (COVID-19) and hospitalization.3,4 However, if and when widespread use of these vaccines will result in sustained control of the pandemic remains unclear given the speed of vaccine roll out and societal factors such vaccine hesitancy and suboptimal adherence to NPIs. Moreover, there is a need to examine these factors at a regional level given the rolling nature of the pandemic and COVID-19 hot spots around the country. The objective of this study was to use our previously developed agent-based simulation model,2 to examine the impact of vaccine coverage and effectiveness, vaccination capacity, and adherence to NPIs on SARS-CoV-2 burden, as well as to predict how these factors influence control of virus spread in urban communities in the US. ## METHODS We previously developed the COVID-19 Agent-based simulation Model (COVAM)2 to represent the interactions among people that may lead to transmission of SARS-CoV-2 in three urban regions in the US: Dane County in Wisconsin, the Milwaukee metro area in Wisconsin, and New York City (NYC). Briefly, COVAM works as follows: individuals who belong to one of eight possible states representing an individual’s COVID-19-related status interact with each other through which SARS-CoV-2 is transmitted.2 The number of such close interactions in a given day is estimated separately for different regions considering population densities using data from social network literature and calibration.5,6 For each of these daily interactions, there is a possibility that a contagious individual exposes another individual to SARS-CoV-2. The number of daily contacts differs by age group, with older individuals less likely to have such interactions as compared to younger individuals. COVAM considers the possibility that not all individuals infected with SARS-CoV-2 will be tested positive and reported. The basic reproduction number (R0) corresponding to the unmitigated base-case transmission dynamics was 3.34 for Dane County and Milwaukee and 6.68 for NYC due to a large number of daily contacts. A unique feature of COVAM is its ability to represent varying levels of adherence of the local communities to NPIs implemented in different regions. COVAM uses cell-phone mobility data and calibration to estimate a time-dependent adherence to NPIs and implements it explicitly by adjusting the number of contacts per person each day.7–9 For instance, a 70% adherence level to NPIs in Dane County and Milwaukee is implemented by simply reducing the number of daily contacts from 10 per day to 3 per day per person, which slows the transmission of SARS-CoV-2. COVAM uses adherence to NPIs to model several distinct behaviors to mitigate transmission, including mask wearing and reduced number of interactions where individuals do not maintain a 6-feet distance during person-to-person interactions, and estimate future burden of COVID-19. COVAM was calibrated using historical pandemic data from the three urban regions and validated in the short term with data that were not used in model development. Additional details are available in Alagoz et al.2 ### COVAM updates We made two extensions to COVAM to answer the research questions of this study. The original version of COVAM used a fixed level of adherence to NPIs for future projections. However, cell-phone-based data and several studies have shown that the response of local communities to NPIs changed over time; therefore, the assumption of a fixed adherence level may not be realistic.7–10 Specifically, even in the absence of seasonal changes, the observed number of cases and deaths impacted how local communities followed NPIs.10–12 To this end, we added a *dynamic adherence* scenario in which we defined two thresholds that trigger high or low adherence to NPIs. Namely, if the number of daily new cases exceeds a high threshold, then the NPI adherence level increases significantly from observed levels. Similarly, if the number of daily new cases drops below a low threshold, then the adherence level drops significantly. The second extension was the incorporation of vaccination into COVAM. We assumed that some individuals undergo vaccination and as a result may gain protective immunity and become non-susceptible after vaccination representing the *vaccine effectiveness* rate. We assumed that individuals who gain protective immunity after vaccination cannot get infected with COVID-19 and do not transmit the disease to others. We also considered that only a proportion of the population agrees to be vaccinated, and there is a daily capacity for vaccination as a function of the proportion of the population in the community. We also allowed vaccinated individuals (whether they are immunized or not) to have a lower adherence to NPIs. Finally, we calibrated COVAM to the latest data on the reported number of confirmed cases in Dane County, Milwaukee, and NYC using case counts by January 5, 2021, January 5, 2021, and January 14, 2021, respectively and compared COVAM’s predictions against data until February 1, 2021. ### Vaccination scenarios In all runs, we assumed that vaccination started on January 5, 2021, as the first vaccine was administered on December 14, 2020 in the US.13 We used COVAM to evaluate the impact of different aspects of vaccination and NPI adherence using the model inputs as presented in **Table 1**. We used observed adherence levels until February 1, 2021 and assumed that adherence levels after this date are maintained afterwards. The adherence scenarios in **Table 1** are based on the new cases and the adherence levels observed in these regions since the beginning of the pandemic. View this table: [Table 1:](http://medrxiv.org/content/early/2021/03/31/2021.03.22.21254131/T1) Table 1: Description of vaccination scenarios We ran 100 replications for each experiment and report only mean values due to very low standard errors. Unless noted otherwise, we reported the cumulative number of confirmed cases associated with each scenario on December 31, 2021. Our base case scenario assumed that vaccination starts on January 5, 2021; daily vaccination capacity is 0.1% of the population per day (i.e., 540 people/day in Dane County, 1,626 people/day in Milwaukee, and 8,672 people/day in NYC); there is a 20% drop in adherence among vaccinated individuals; vaccine effectiveness is 90%; vaccination coverage is 50%; and the baseline test rate is 75%. ### Controllable spread date analysis Using the implied R0 values for infectious disease models, the herd immunity is theoretically achieved when 70%, 70%, and 85% of the population gains immunization through recovery from COVID-19 or vaccination in Dane County, Milwaukee, and NYC, respectively. However, these herd immunity levels assume that NPIs, including face mask use, are no longer adopted. Therefore, we investigated whether vaccination, along with NPIs, will mitigate the pandemic to a level such that that it will become controllable. For this purpose, we defined a *controllable spread date* as the date after which the number of daily new confirmed cases never exceeds 20 for Dane County, 60 for Milwaukee, and 320 for NYC. For example, if the daily number of new cases never exceeds 20 for Dane County after June 15, 2021, the controllable spread date is set to June 15, 2021. We selected these daily numbers of confirmed cases based on data from Dane County that employs 180 contact tracers to trace COVID-19 cases; therefore, 20 new cases per day would make it feasible to conduct aggressive and efficient contact tracing to completely control the disease in Dane County.14 For the other regions, we scaled up the number considering population size. ### Sensitivity analysis We conducted a parametric sensitivity analysis in which we tested the impact of uncertainty in the drop in adherence to NPIs after vaccination, where the drop in adherence was assumed to be 0% as opposed to 20% in the base case. We also conducted a structural sensitivity analysis in which we assumed that vaccination reduces the risk of infection instead of leading to complete immunity. That is, a 90% vaccine effectiveness scenario reduces the risk of getting infected among all vaccinated individuals by 90% but does not lead to complete immunity. ## RESULTS COVAM accurately predicted the reported number of cases in each urban area in the short term (**Supplement Figure 1**). We first reported the number of confirmed cases over time for different vaccine coverage and adherence to NPI scenarios when vaccine effectiveness is 90% and daily vaccination capacity is 0.1% (**Figure 1**). We found that the total number of confirmed cases was not sensitive to the vaccination coverage in any of the regions as long as communities keep a high level of adherence to NPIs. In general, the dynamic adherence scenario has led to later controllable spread dates compared to those under fixed adherence scenarios for both vaccination and no vaccination cases. ![](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/03/31/2021.03.22.21254131/F1/graphic-2.medium.gif) [](http://medrxiv.org/content/early/2021/03/31/2021.03.22.21254131/F1/graphic-2) ![](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/03/31/2021.03.22.21254131/F1/graphic-3.medium.gif) [](http://medrxiv.org/content/early/2021/03/31/2021.03.22.21254131/F1/graphic-3) ![](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/03/31/2021.03.22.21254131/F1/graphic-4.medium.gif) [](http://medrxiv.org/content/early/2021/03/31/2021.03.22.21254131/F1/graphic-4) Figure 1. Impact of vaccine coverage on the number of confirmed cases in different regions for two different scenarios of adherence to NPIs. In this experiment, we assumed that vaccine effectiveness is 90% and vaccine capacity is 0.1% per day. **Figure 2**. and **Table 2** show how adherence to NPIs change the effect of vaccination on the cumulative number of cases. Assuming vaccine effectiveness is 90%, vaccine coverage is 50%, and daily vaccination capacity is 0.1%, we found that the level of adherence to NPIs had a major impact on the cumulative number of cases (**Figure 2** and **Table 2**), as well as the controllable spread date. When the level of NPI adherence is very high (75% for Dane County and Milwaukee; 90% for NYC), the effect of vaccination on the controllable spread date and number of cumulative cases was minimal compared to no vaccination. For the most recently observed NPI adherence rates (70% adherence for Dane County and Milwaukee; 85% for NYC), vaccination reduced the number of confirmed cumulative cases from 51,784, 185,343, and 1,110,070 to 46,063, 172,592, and 909,823 implying an 11%, 7%, and 18% rate of reduction in the number of confirmed cases for Dane, Milwaukee, and NYC, respectively, and reduced the time to controllable spread by 2-3 months in all three regions (**Table 2**, 0.1% vaccination capacity). View this table: [Table 2:](http://medrxiv.org/content/early/2021/03/31/2021.03.22.21254131/T2) Table 2: Controllable spread date and number of cases on December 31, 2021 for different daily vaccination capacity and adherence to nonpharmaceutical interventions (NPI) scenarios (vaccine effectiveness 90%, vaccine coverage 50%). Numbers in parentheses represent percent reduction relative to no vaccine. ![](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/03/31/2021.03.22.21254131/F2/graphic-7.medium.gif) [](http://medrxiv.org/content/early/2021/03/31/2021.03.22.21254131/F2/graphic-7) ![](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/03/31/2021.03.22.21254131/F2/graphic-8.medium.gif) [](http://medrxiv.org/content/early/2021/03/31/2021.03.22.21254131/F2/graphic-8) ![](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/03/31/2021.03.22.21254131/F2/graphic-9.medium.gif) [](http://medrxiv.org/content/early/2021/03/31/2021.03.22.21254131/F2/graphic-9) Figure 2. Impact of vaccination and adherence to nonpharmaceutical interventions (NPIs) on the number of cases over time. In this experiment, we assumed that vaccine effectiveness is 90%, vaccine coverage is 50%, and daily vaccination capacity is 0.1% per day. The reduction in the number of cases due to vaccination was higher in Dane County compared to Milwaukee. The proportion of cumulative confirmed cases in the population in Dane County and Milwaukee as of January 31, 2021 were 7% and 10%, respectively.15 COVAM considers the possibility that the actual number of cases is higher than the confirmed number of cases due to some patients experiencing very mild disease and the limited testing. As such, COVAM estimated that the proportion of cumulative confirmed and unreported cases in the population in Dane County and Milwaukee as of January 31, 2021 was 8% and 11%, respectively. Therefore, the prevalence of SARS-CoV-2 was higher in Milwaukee compared to Dane County during the vaccination period. **Table 2** also shows that maintaining a high level of adherence to NPIs under the same vaccination scenario reduces the number of confirmed cases and results in a significantly earlier date of controllable spread compared to lower NPI adherence in all three regions. For example, in Dane County, a 60% level of NPI adherence after February 1, 2021 led to 108,001 cases with vaccination and a controllable spread date of September 15, 2021 versus a 70% level of NPI adherence, which led to a total of 46,063 cases and a controllable spread date of May 1, 2021. We also found that daily vaccination capacity had a differential effect on the number of cases and the date on which controllable spread is achieved (**Table 2**). In particular, under the dynamic adherence scenario, vaccination at 0.5% instead of 0.05% of the population per day reduced the number of confirmed cases from 72,062, 250,540, and 1,273,560 to 42,468, 163,134, and 688,177 implying a 41%, 35%, and 46% reduction in Dane County, Milwaukee, and NYC, respectively, and reduced the time to controllable spread by 12, 10, and 16 months, respectively. **Table 3** shows that the effectiveness of vaccination does not have a major impact on the date when controllable spread is achieved, whereas it reduces the number of confirmed cases greatly. Parametric and structural sensitivity analyses recapitulated the overall trends observed in the base-case runs (**Supplement Figures 3-6** and **Supplement Tables 2-5**). View this table: [Table 3:](http://medrxiv.org/content/early/2021/03/31/2021.03.22.21254131/T3) Table 3: Controllable spread date and number of cases on December 31, 2021 for different vaccination effectiveness scenarios (vaccine coverage 50%, vaccination capacity 0.01%). Numbers in parentheses represent percent reduction in the number of cases relative to no vaccine. ## DISCUSSION In this simulation study, we estimated the impact of vaccination on the number of COVID-19 cases in three urban communities using agent-based simulation modeling. We found that controllable spread of SARS-CoV-2 can be achieved sooner than when a large proportion of the population is vaccinated (e.g., 70-80%) as long as there is high adherence to NPIs in the community. We further found that vaccination would reduce the number of COVID-19 cases significantly, but the rate of reduction in the number of cases differs among regions. Finally, we found that the level of adherence to NPIs has a major impact on the number of cases, as well as the date that controllable spread is achieved regardless of vaccine capacity, vaccine effectiveness, or region. In NYC, vaccination would reduce the number of cases from 1,110,070 to 909,823, or by 18%, when the adherence to NPIs is fixed at 85%. Under the same scenario, controllable spread would be achieved on May 30, 2021 as opposed to August 27, 2021 without vaccination. Although the disease spread rate (i.e., R0 value) was identical for Dane County and Milwaukee, the impact of vaccination on the number of cases and timing of controllable spread was more pronounced in Dane County compared to Milwaukee. Of note, according to the COVAM model, 8% of the Dane County population was infected as of January 31, 2021 compared to 11% of the Milwaukee population. Thus, our results imply that vaccination may have a larger impact when the history of COVID-19 infections is lower to start with. Based on this, we suggest to benefit most from vaccination, regions need to keep a high level of adherence to NPIs to keep caseload low, a similar finding as reported in a prior modeling study.16 Relying on vaccination alone to control the spread of COVID-19 over the coming months may delay the timing of controllable disease by several months, and this has important implications for the timing of lifting policies for NPIs that are currently in place. Mainstream media, as well as prior modeling efforts, have focused on the endpoint of herd immunity – the point at which disease transmission is halted and life can “return to normal” from the standpoint of NPIs.17,18 While reaching herd immunity is the ultimate goal of a vaccination campaign, we chose to focus on an intermediate metric: controlled spread while continuing reasonable public health precautions. Our results highlight the importance of continued adherence to NPIs during vaccine rollout. As the vaccine is being delivered, our results suggest that vaccine effectiveness in reducing viral spread is highly dependent on NPIs. If NPI efforts are sustained at current levels during the rollout, then even low levels of vaccine uptake will be able to control viral spread. On the other hand, if vaccine rollout is paired with NPI easing, a higher proportion of the population will need vaccination to stop the accumulation of new cases. Our findings are consistent with those from a few other modeling studies that report reaching herd immunity depends on high adherence to NPIs during vaccination rollout.16,19 The impact of even highly effective vaccines may be diminished if deployed into a population with high viral transmission caused by low adherence to NPIs.16 In addition, the use of NPIs may decrease the level of vaccine coverage needed to achieve herd immunity.19 Our study extends the existing literature by showing that the degree to which NPIs, vaccine coverage, and pace of vaccination impact disease control may vary regionally, and are highly dependent on both population density and the existing cumulative burden of COVID-19 since the pandemic began. Areas of lower density need lower adherence to NPIs to decrease case rates with vaccination, and those with a lower existing burden of COVID-19 have a greater ability to improve their trajectories with vaccination campaigns. Our study has several limitations related to uncertainty in vaccine effectiveness in real-world settings. Authorized COVID-19 vaccines to date have demonstrated high efficacy for preventing COVID-19 illness and hospitalization.3 COVAM assumes that vaccination also prevents transmission of SARS-CoV-2. However, studies to determine the effect of vaccination on acquisition and shedding of SARS-CoV-2 are ongoing. Our model also assumes that COVID-19 vaccines will be effective in preventing transmission of new and future variants of the virus. The B.1.1.7 (UK), B.1.351 (South Africa), and P.1 (Brazil) variants have all been detected in the US,20 and all contain mutations in the spike protein. Both the Pfizer and Moderna vaccines are believed to be effective against the B.1.1.7 variant.21–23 However, a recent study suggests the neutralizing antibodies induced by vaccination are less potent against the B.1.351 and B.1.1.7 variants *in vitro*.24 As long as SARS-CoV-2 is replicating at high levels, new variants can be expected to emerge, underscoring the importance of high-level adherence to NPIs and rapid vaccination with high-level coverage. Furthermore, we do not use a full calibration procedure that is commonly used in simulation modeling to estimate the unobservable inputs of the model, which may have led to suboptimal set of inputs.25–27 Finally, our model does not consider age-based differences in the administration and effectiveness of the vaccines. However, currently used vaccines are administered for individuals over ages 16 only. It is unknown if vaccination of individuals younger than 16 of age will begin by the end of the study’s simulation period or if the effectiveness will be different for younger individuals. In conclusion, this simulation modeling demonstrates that continued high adherence to NPIs along with vaccination results in a shorter time to control the COVID-19 pandemic in US urban areas. Furthermore, our results suggest adhering to NPIs to keep the caseload low until vaccine becomes widely available can lead to greater benefit of vaccination in terms of reduction in the number of COVID-19 cases. ## Supporting information Supplement [[supplements/254131_file03.docx]](pending:yes) ## Data Availability All underlying data is included in the manuscript. ## Author contributions All authors contributed to the design of this study. NS obtained funding for this study. OA coded the simulation model used in this study. OA and GA ran the simulation experiments and analyzed the model outputs. AS, NS, and OA wrote the first draft of this manuscript. All authors critically revised the manuscript for intellectual content. ## Conflict of Interest statement OA is a paid consultant for Bristol-Myers Squibb and Biovector Inc, outside of the submitted work. Other authors have no conflicts of interest to report. ## Role of the funding source The National Institute of Allergy and Infectious Diseases (NIAID) funded this work, and had no role in the design and conduct of the research; collection, analysis, and interpretation of the data; preparation, review, and approval of the manuscript; or the decision to submit the manuscript for publication. ## Access to data and data analyses All authors had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. ## Acknowledgements The authors thank Amanda Young for her help in preparing this manuscript. ## Footnotes * **Support**: This work was supported by the National Institutes of Health under National Institute of Allergy and Infectious Diseases (NIAID) Grant 1DP2AI144244-01. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. * abstract revised * Received March 22, 2021. * Revision received March 30, 2021. * Accepted March 31, 2021. * © 2021, Posted by Cold Spring Harbor Laboratory The copyright holder for this pre-print is the author. All rights reserved. The material may not be redistributed, re-used or adapted without the author's permission. ## References 1. 1.John Hopkins University of Medicine. Coronavirus Resource Center. 2021; [https://coronavirus.jhu.edu/](https://coronavirus.jhu.edu/). Accessed February 9, 2021. 2. 2.Alagoz O, Sethi AK, Patterson BW, Churpek M, Safdar N. Effect of Timing of and Adherence to Social Distancing Measures on COVID-19 Burden in the United States: A Simulation Modeling Approach. Annals of internal medicine. 2020. 3. 3.Baden LR, El Sahly HM, Essink B, et al. Efficacy and safety of the mRNA-1273 SARS-CoV-2 vaccine. New England Journal of Medicine. 2020. 4. 4.FDA. COVID-19 Vaccines: Emergency Use Authorization - Vaccines. 2021; [https://www.fda.gov/emergency-preparedness-and-response/coronavirus-disease-2019-covid-19/covid-19-vaccines#eua-vaccines](https://www.fda.gov/emergency-preparedness-and-response/coronavirus-disease-2019-covid-19/covid-19-vaccines#eua-vaccines). Accessed March 22, 2021. 5. 5.Del Valle SY, Hyman JM, Hethcote HW, Eubank SG. Mixing patterns between age groups in social networks. Social Networks. 2007;29(4):539–554. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.socnet.2007.04.005&link_type=DOI) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000250964800005&link_type=ISI) 6. 6.Lee BY, Brown ST, Cooley P, et al. Simulating school closure strategies to mitigate an influenza epidemic. Journal of Public Health Management and Practice. 2010;16(3):252–261. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1097/PHH.0b013e3181ce594e&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=20035236&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F03%2F31%2F2021.03.22.21254131.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000276873700010&link_type=ISI) 7. 7.GeoDS Lab @ UW-Madison. Mapping Mobility Changes in Response to COVID-19. 2021; [https://geods.geography.wisc.edu/covid19/physical-distancing/](https://geods.geography.wisc.edu/covid19/physical-distancing/). Accessed February 1, 2021. 8. 8.Google. COVID-19 Community Mobility Report. 2021; [https://www.google.com/covid19/mobility/](https://www.google.com/covid19/mobility/). Accessed February 1, 2021. 9. 9.Unacast. Social Distancing Scoreboard. 2021; [https://www.unacast.com/covid19/social-distancing-scoreboard](https://www.unacast.com/covid19/social-distancing-scoreboard). Accessed February 1, 2021. 10. 10.Masters NB, Shih S-F, Bukoff A, et al. Social distancing in response to the novel coronavirus (COVID-19) in the United States. PloS one. 2020;15(9):e0239025. 11. 11.Pedersen MJ, Favero N. Social Distancing during the COVID □ 19 Pandemic: Who Are the Present and Future Noncompliers? Public Administration Review. 2020;80(5):805–814. 12. 12.Xie K, Liang B, Dulebenets MA, Mei Y. The impact of risk perception on social distancing during the COVID-19 pandemic in China. International journal of environmental research and public health. 2020;17(17):6256. 13. 13.BBC. Covid-19: First vaccine given in US as roll-out begins. 2021; [https://www.bbc.com/news/world-us-canada-55305720](https://www.bbc.com/news/world-us-canada-55305720). Accessed January 20, 2021. 14. 14.PBS Wisconsin. Dane County Shifts to Contact Tracing ‘Crisis Model’ Amid Another Record-Setting Day. 2021; [https://pbswisconsin.org/news-item/dane-county-shifts-to-contact-tracing-crisis-model-amid-another-record-setting-day/](https://pbswisconsin.org/news-item/dane-county-shifts-to-contact-tracing-crisis-model-amid-another-record-setting-day/). Accessed January 31, 2021. 15. 15.Wisconsin Department of Health Services. COVID-19: Wisconsin Data. 2021; [https://www.dhs.wisconsin.gov/covid-19/data.htm](https://www.dhs.wisconsin.gov/covid-19/data.htm). Accessed February 1, 2021. 16. 16.Paltiel AD, Schwartz JL, Zheng A, Walensky RP. Clinical Outcomes Of A COVID-19 Vaccine: Implementation Over Efficacy: Study examines how definitions and thresholds of vaccine efficacy, coupled with different levels of implementation effectiveness and background epidemic severity, translate into outcomes. Health Affairs. 2021:doi:10.1377/hlthaff.2020.02054. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1377/hlthaff.2020.02054&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=33211536&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F03%2F31%2F2021.03.22.21254131.atom) 17. 17.Randolph HE, Barreiro LB. Herd immunity: understanding COVID-19. Immunity. 2020;52(5):737–741. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.immuni.2020.04.012&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32433946&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F03%2F31%2F2021.03.22.21254131.atom) 18. 18.McNeil Jr. DG. How Much Herd Immunity Is Enough? New York Times December 24, 2020: [https://www.nytimes.com/2020/2012/2024/health/herd-immunity-covid-coronavirus.html](https://www.nytimes.com/2020/2012/2024/health/herd-immunity-covid-coronavirus.html). 19. 19.Iboi EA, Ngonghala CN, Gumel AB. Will an imperfect vaccine curtail the COVID-19 pandemic in the US? Infectious Disease Modelling. 2020;5:510–524. 20. 20.CDC. Emerging SARS-CoV-2 Variants. 2021; [https://www.cdc.gov/coronavirus/2019-ncov/more/science-and-research/scientific-brief-emerging-variants.html](https://www.cdc.gov/coronavirus/2019-ncov/more/science-and-research/scientific-brief-emerging-variants.html). Accessed February 1, 2021. 21. 21.Wu K, Werner AP, Moliva JI, et al. mRNA-1273 vaccine induces neutralizing antibodies against spike mutants from global SARS-CoV-2 variants. BioRxiv. 2021:[https://doi.org/10.1101/2021.1101.1125.427948](https://doi.org/10.1101/2021.1101.1125.427948). 22. 22.Greaney AJ, Loes AN, Crawford KH, et al. Comprehensive mapping of mutations to the SARS-CoV-2 receptor-binding domain that affect recognition by polyclonal human serum antibodies. bioRxiv. 2021:[https://doi.org/10.1101/2020.1112.1131.425021](https://doi.org/10.1101/2020.1112.1131.425021). 23. 23.Weisblum Y, Schmidt F, Zhang F, et al. Escape from neutralizing antibodies by SARS-CoV-2 spike protein variants. Elife. 2020;9:e61312. 24. 24.Wang P, Lihong L, Iketani S, et al. Increased Resistance of SARS-CoV-2 Variants B. 1.351 and B. 1.1. 7 to Antibody Neutralization. bioRxiv. 2021:[https://doi.org/10.1101/2021.1101.1125.428137](https://doi.org/10.1101/2021.1101.1125.428137). 25. 25.Barker AK, Alagoz O, Safdar N. Interventions to reduce the incidence of hospital-onset Clostridium difficile infection: An agent-based modeling approach to evaluate clinical effectiveness in adult acute care hospitals. Clinical Infectious Diseases. 2018;66(8):1192–1203. 26. 26.Cevik M, Ergun MA, Stout NK, Trentham-Dietz A, Craven M, Alagoz O. Using active learning for speeding up calibration in simulation models. Medical Decision Making. 2016;36(5):581–593. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/0272989X15611359&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=26471190&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F03%2F31%2F2021.03.22.21254131.atom) 27. 27.Codella J, Safdar N, Heffernan R, Alagoz O. An agent-based simulation model for C. difficile infection control. Medical Decision Making. 2015;35(2):211–229. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/0272989X14545788&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=25112595&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F03%2F31%2F2021.03.22.21254131.atom)