Which factors had the greatest impact on the United States COVID-19 outbreak? An ecological assessment of mitigation behavior and policy contributions to reducing SARS-CoV-2 transmission in the US from September 2020 through November 2021
==============================================================================================================================================================================================================================================

* Velma K Lopez
* Sarah Kada
* Pragati V Prasad
* Taylor Chin
* Betsy L. Cadwell
* Jessica M Healy
* Rachel B Slayton
* Matthew Biggerstaff
* Michael A Johansson

## Abstract

*Introduction*: United States’ jurisdictions implemented varied policies to slow SARS-CoV-2 transmission. Understanding patterns of these policies alongside individual’s behaviors can inform effective outbreak response.

*Methods*: We estimated the time-varying reproduction number (Rt), a weekly measure of real-time transmission using US COVID-19 reported cases from September 2020-November 2021. Using two multi- level regression models, we then assessed the association between Rt and policies, personal COVID-19 mitigation behaviors, variants, immunity, and social vulnerability indicators. First, we fit a model with state-level policy stringency according to the Oxford Stringency Index (OSI), a composite indicator reflecting the strictness of COVID-19 policies and strength of pandemic-related communication. Our second model included a subset of specific policies.

*Results*: Implementation of stringent observed policies (defined by OSI) was associated with an Rt reduction of 6.7- 11.6% (95% Credible Interval [CI]). In the Individual Policy Model, mask mandates had a null association with Rt (95% CI: -1.5-0.2%), while other policies were associated with modest reductions: cancellation of public events 95% CI: 1.4-3.7%; restrictions on gathering sizes 95% CI: 0.1- 2.2%; stay-at-home orders 95% CI: 0.3-4.8%. The association between Rt and other covariates was similar in both models. Among personal COVID-19 mitigation behaviors in the OSI Model, Rt was estimated to decrease 18%-26% (95% CI) with a 50% reduction in national airline travel, 2.4-3.3% (95% CI) with a 10% reduction in local movement to recreation and retail locations, and 12-15% (95% CI) with self-reported mask use of 50%. Increased COVID-19 seroprevalence and vaccination were both associated with reduced Rt, 28-32% (95% CI) and 20-23% (95% CI) reductions, if half of people had been previously infected or fully vaccinated, respectively in the OSI Model.

*Conclusion*: SARS-CoV-2 transmission was reduced by layered measures. These results underscore the need for policy, behavior change, and risk communication integration to reduce virus transmission during epidemics.

**Key messages** What is already known on this topic: When the United States responded to the COVID-19 outbreak, jurisdictions took varied approaches to balance economic well-being with health and safety. Yet, the impact of the pandemic was large - millions of people were infected and over a million people died – and the relative roles of policies, policy-independent behavior change, and other factors remains unclear.

What this study adds: A retrospective analysis of policy implementation and SARS-CoV-2 transmission dynamics over a year and a half indicated that social behavioral change was critical for limiting transmission prior to increases in immunity due to infection and vaccination.

How this study might affect research, practice or policy: While policies contributed to slowing the spread of COVID-19, the largest impact on transmission early in the pandemic was due to individual behavior change, highlighting the importance of identifying and communicating effective control practices regardless of specific policies.

Keywords
*   Outbreak response
*   policies
*   personal behavior

## Main Text Introduction

As SARS-CoV-2 began to spread in early 2020, there was limited knowledge about transmission dynamics and uncertainty regarding the most effective mitigation tools and how to best implement them. Governments began to implement travel restrictions and stay-at-home orders to reduce contact between individuals. These non-pharmaceutical intervention (NPI) policies rapidly evolved and eventually encompassed a diverse range of strategies. NPI policies, such as stay-at-home orders, closure of public facilities, limiting gathering size, and closing schools or universities, and their combinations indeed slowed SARS-CoV-2 spread during the early pandemic phase [1–3]. While NPI policies, particularly those targeting containment of the virus, were markedly successful at reducing transmission opportunities, several exogenous factors also played important roles.

Social structure dramatically affects how people interact and cannot be ignored in disease dynamics. Within the United States (US), physical distancing by staying home from work was a policy option available to more wealthy and White individuals rather than frontline, essential worker populations [4], who are more likely to have a lower socioeconomic status and belong to racial minoritized communities [5,6]. Additionally, SARS-CoV-2 testing sites may have been disproportionately available in areas that had a greater proportion of White residents [7–9]. In other words, the opportunity to know one’s infection status and limit interactions with others, if necessary, was only available to a privileged subset of the population. Moreover, weather added further complexity to these dynamics. For example, SARS- CoV-2 transmission may decrease as temperature and specific humidity increases [10]; potentially weakening the viral envelope [11] and impacting when people spend time indoors, i.e., conditions more favorable to viral transmission. Whether COVID-19 mitigation policies effectively reduced transmission thus requires an assessment within the larger context of behavior, social factors, and weather conditions.

COVID-19 mitigation policies are broadly accepted in the literature as successfully reducing SARS-CoV-2 transmission. However, the relative contribution of policy over long periods of time, while accounting for exogenous factors that also drive transmission, is unknown. Disentangling competing drivers of US SARS-CoV-2 dynamics over time can highlight opportunities to improve future public health outbreak responses. US COVID-19 mitigation policies were diverse and heterogeneously implemented [12].

Additionally, the 55 million cases and over 820,000 deaths reported by December 31, 2021 [13], were spatially patterned by existing demographics and health disparities across jurisdictions. Given these patterns, we seek to understand the relative impact of COVID-19 mitigation policies on US SARS-CoV-2 transmission as both the pandemic and pandemic response evolved.

## Methods

We conducted an ecological assessment of jurisdictional-level SARS-CoV-2 transmission in the US. The analysis period (64 weeks; September 6, 2020, to November 27, 2021) and the fifty-one jurisdictions (all US states and the District of Columbia; DC) analysed were selected for data availability.

*COVID-19 Time-varying Reproduction Number Estimation*: We estimated Rt, a weekly measure of real- time transmission, using COVID-19 cases reported to the Centers for Disease Control and Prevention (CDC). Onset dates were back-projected from case report dates using time-specific delays; infection dates were sampled using a log-normal distribution for the incubation periods (log mean = 1.63 and log standard deviation = 0.5 based on published data [14]). For each trajectory, we estimated Rt using the methods described in Cori et al. [15]. We used a 7-day window and an uncertain serial interval (SI) (mean: 5 days, standard deviation: 1 day), with 5 samples from the SI distribution and 5 samples of the Rt posterior for each SI value [16]. We generated 250 Rt samples for each time point and jurisdiction and used the mean estimate on each Wednesday for this analysis.

*Covariate data:* We included data on mitigation policies [17], personal mitigation behaviors [18,19], key variants circulation [20], weather [21,22], immunity indicators [23,24], and vulnerability indicators [25]. Standardized policy data were obtained from the Oxford COVID-19 Government Response Tracker [17]. The dataset includes a composite indicator, the Oxford Stringency Index (OSI), which reflects the overall strictness of COVID-19 policies and strength of pandemic-related communication, as well as individual policies. All data are described in the Supplemental Text. Descriptive statistics to inform model building are shown in Supplemental Figures 1-5. We assessed correlation between covariates by estimating the median pairwise R2 with bivariate regression models (see Supplemental Figure 1).

*Analytical Approach*: We assessed the association between Rt and selected determinants with Bayesian Gaussian multi-level regression models, using a log-link function and jurisdiction- and time-specific intercepts. We assessed two models. The first focused on general government response, using the OSI, a composite indicator of several different policies and communication strategies. Because we were also interested in the individual impacts of some of the specific policies, we built a second model that omitted the general OSI variable and included only a few specific policies that are components of OSI. The second model included four individual policies: cancelation of public events, restrictions on gathering sizes, stay-at-home orders, and mask mandates. Both models were adjusted for the aforementioned covariates, which were selected for inclusion because of their *a priori* relationships with COVID-19 transmission (see Supplemental Text for model statements, samples run, and information about priors). Model convergence was assessed using the Gelman-Rubin convergence diagnostic ![Graphic][1]</img> and model fit was evaluated from the predictive posteriors. Jurisdiction-specific results for the OSI model are presented in Supplemental Figure 6.

We conducted several sensitivity analyses. First, we compared these models to the same two models with naïve priors via leave-one-out (LOO) cross-validation [26] (Supplemental Figures 7) and compared models with different structures for temporal correlation (Supplemental Figure 8). Second, we re-ran the primary models using publicly available Rt estimates from the Centre for Mathematical Modeling of Infectious Diseases COVID modelling group [27] and compared the model results to those presented here (Supplemental Figure 9). Third, we re-ran the individual policy model without the behavior covariates (Supplemental Figure 10).

Analyses were conducted using R (version 4.2.1), with the *rstanarm* package used for primary analyses. R code is available in a public repository ([https://github.com/cdcepi/COVID-19-Mitigation_Rt](https://github.com/cdcepi/COVID-19-Mitigation_Rt)).

This activity was reviewed by CDC to be deemed not human subject research and therefore a human subject review was not required. The study was conducted consistent with applicable federal law and CDC policy§.

## Results

Rt estimates exhibited spatiotemporal variability between September 2020 and November 2021 (Figure 1). All jurisdictions experienced sustained transmission increases (Rt > 1) in late 2020, followed by a period of fluctuations until the summer 2021 Delta wave when many had their highest Rt estimates. Rt dynamics differed substantially between jurisdictions, such as transmission timing and rate increases in late 2020 during the Alpha wave. The lowest mean estimated Rt over this period was for Vermont in May 2021 (Rt=0.66) and the highest value was for DC in November 2021 (Rt=1.62).

![Figure 1.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2024/09/26/2023.07.19.23292882/F1.medium.gif)

[Figure 1.](http://medrxiv.org/content/early/2024/09/26/2023.07.19.23292882/F1)

Figure 1. 
The jurisdiction-specific weekly COVID-19 time-varying reproduction number (Rt), 90% confidence intervals in grey, and time periods reflecting implementation of specific non-pharmaceutical interventions (NPIs). State-wide implementation of stay-at-home orders are in red, restrictions on gatherings in yellow, cancellation of public events in dark blue, and universal masking when physical distancing was not possible in light blue. The solid vertical line represents January 1, 2021 and the dashed horizontal line reflects an Rt value of 1.0.

All individual policies exhibited spatiotemporal variability (Figure 1) and moderate correlation (maximum: 0.47, Supplemental Figure 1B). Overall, stringency of mitigation policies dipped slightly in October 2020 and decreased substantially between March and June 2021 (Figure 2A). Jurisdictional variation in policy stringency persisted, with the lowest median value in South Dakota (0.09; range: 0.06- 0.21) and the highest median value in Hawaii (0.66; range: 0.44-0.76).

![Figure 2.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2024/09/26/2023.07.19.23292882/F2.medium.gif)

[Figure 2.](http://medrxiv.org/content/early/2024/09/26/2023.07.19.23292882/F2)

Figure 2. 
**A**. Distribution of time-varying covariates, including the Oxford Stringency Index (OSI), personal COVID-19 behaviors, proportion of key SARS-CoV-2 variants in circulation, weather, and immunity to SARS-CoV-2. Lines reflect jurisdiction-level observations over time, with the median values across jurisdictions depicted by the bold line. The solid, black vertical line represents January 1, 2021. **B**. Average of the three selected Community Covid-19 Vulnerability Index (CCVI) indicators for each jurisdiction.

Personal COVID-19 mitigation behaviors also varied over time and space (Figure 2A). Reductions in national airline travel and individual mobility showed similar patterns over time, though individual mobility also varied by jurisdiction. Reductions in both were substantial and relatively static until March 2021. In August 2021, all three behavior indicators increased, with new reductions in national airline travel and local mobility, as well as increased reported mask use (Figure 2A). The increased personal mitigation behaviors in August 2021 coincided with the rapid increase in prevalence of the Delta variant (Figure 2A, Supplemental Figure 1A). Expansion of the Alpha variant in early 2021 was generally slower, more heterogeneous, and not correlated with increased mitigation behavior.

There was high heterogeneity in jurisdictional SARS-CoV-2 seroprevalence. By the end of November 2022, seroprevalence ranged from 0.10 in Vermont to 0.46 in Wyoming (Figure 2A). The proportion of the population that was fully vaccinated with an initial vaccine series increased across all jurisdictions beginning with the vaccination distribution in early 2021 and ranged from 0.47 (Alabama) to 0.75 (Vermont) at the end of November 2021.

The average of the five Community COVID-19 Vulnerability Index (CCVI) indicators showed higher vulnerability in southern jurisdictions (Figure 2B).

We fitted two regression models to assess the relationship of these factors with Rt dynamics: the OSI Model and the Individual Policy Model. For both models, we assessed alternative spatiotemporal model structures and found that a model with independent random effects for time and state provided the best fit (see Supplement 8). With moderate values for CCVI indicators (0.5 each) and temperature (12°C), estimated Rt in the absence of mitigation (i.e., the fixed effect intercept) was 2.2 for the OSI Model (95% Credible Interval [CI]: 2.0-2.5) and 2.5 (95% CI 1.9-3.4) for the policy model.

Overall stringency, some of the specific individual policies, each behavioral component, and both immunity indicators were associated with decreased Rt (Figure 3). Implementation of the strictest policies observed in the US relative to least strict policy implementation decreased Rt by 9.2% (95% CI: 6.7- 11.6%). Within the individual policies model, cancellation of public events decreased Rt by 2.6% (95% CI: 1.4-3.7%), restrictions on gathering sizes by 1.2% (95% CI: 0.1-2.2%), and stay-at-home orders by 2.6% (95% CI: 0.3-4.8%). Mask mandates had a mean estimate corresponding to a 0.7% reduction in Rt but did not reach statistical significance (95% CI: -1.5-0.2%). Strong associations were also observed for personal mitigation behaviors in both models. For the OSI Model, Rt decreased by 22% (95% CI: 18%- 26%) if there were a 50% reduction in national airline travel, 2.9% (95% CI: 2.4-3.3%) if local movement to recreation and retail locations decreased by 10%, and 14% (95% CI: 12-15%) if self-reported mask use reached 50%. Seroprevalence and vaccination were both associated with reduced Rt - a 30% (95% CI: 28- 32%) and 22% (95% CI: 20-23%) estimated reduction if half the population had been previously infected or fully vaccinated, respectively.

![Figure 3.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2024/09/26/2023.07.19.23292882/F3.medium.gif)

[Figure 3.](http://medrxiv.org/content/early/2024/09/26/2023.07.19.23292882/F3)

Figure 3. 
**A**. Percent change in the COVID-19 time-varying reproduction (Rt) number from policies, personal COVID-19 behaviors, proportion of key SARS-CoV-2 variants in circulation, weather, immunity to SARS- CoV-2, and variables affecting underlying trends in transmission. The results from two different linear regression models are shown below. Model one, in pink, included the Oxford Stringency Index (OSI). Model two, in purple, included a subset of policies used to comprise the OSI. Both regression models were gaussian with log link function and had jurisdiction and time specific intercepts. The posterior mean is represented by the point and the 95% credible intervals by the bars. **B**. The marginal mean in the COVID- 19 Rt across the range of observed OSI values in jurisdictions with the highest and lowest population density. **C**. The marginal mean COVID-19 Rt, and 95% credible interval show in bands, across the range of observed OSI values in jurisdictions with the highest and lowest racial and ethnic diversity.

Community vulnerability also played an important role in transmission. Both population density and a community’s racial and ethnicity diversity level were associated with increases in Rt (Figure 3). However, the level of policy stringency needed to reduce Rt to less than 1 differed within each of these groups. For example, in jurisdictions with the highest population density (Figure 3B) and the most racial and ethnic diversity (Figure 3C), even the highest OSI values were not sufficient to reduce transmission.

Sensitivity analysis showed consistent estimates for policy impacts even when personal mitigation behaviors were excluded (Supplement 10). Coefficients were similar in direction for both models, with or without informative priors (see Supplement 6), and minor magnitude differences were observed in our sensitivity analyses with alternative Rt estimates (Supplement 9).

We estimated the proportional reduction associated with each time-varying component individually and combined for each jurisdiction and nationally over time using the OSI Model estimates (Figure 4, Figure 5, and Supplemental Figure 6). Over the study period, personal mitigation behaviors were associated with the largest proportional transmission reductions in all jurisdictions before vaccine implementation and remained an important or leading contributor to reduction thereafter (median combined reduction over time across locations: 44%, range: 10-62%) (Figure 5). Immunity was the second most important contributor overall and of growing importance as more people were infected and vaccination coverage increased. These patterns, however, were starkly different between jurisdictions (Figure 4 and Supplemental Figure 6). For example, in November 2021 many jurisdictions had higher estimated reductions associated with previous infections than vaccination (e.g., Wyoming). Meanwhile, other jurisdictions with lower seropositivity or more vaccinations had higher estimated reductions associated with vaccination (e.g., Vermont). At the end of the study period, SARS-CoV-2 seroprevalence and vaccination were associated with a wide range of jurisdictional reductions (8-35% and 24-36%, respectively). Policies and weather were also associated with transmission changes but with less overall estimated magnitude than the effects of behavior and immunity.

![Figure 4.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2024/09/26/2023.07.19.23292882/F4.medium.gif)

[Figure 4.](http://medrxiv.org/content/early/2024/09/26/2023.07.19.23292882/F4)

Figure 4. 
Estimates for jurisdiction-level proportional reductions in the COVID-19 time-varying reproduction number (Rt) over time for the Oxford Stringency Index (OSI) regression model. Colored lines depict the relative reduction for select sets of covariates, with the 95% credible interval in the corresponding-colored bands, and the solid vertical line represents January 1, 2021.

![Figure 5.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2024/09/26/2023.07.19.23292882/F5.medium.gif)

[Figure 5.](http://medrxiv.org/content/early/2024/09/26/2023.07.19.23292882/F5)

Figure 5. 
Average relative contribution of observed covariates on fitted COVID-19 time-varying reproduction number (Rt) across all jurisdictions from the OSI model. The highest values over time (i.e., the top of the pink band) represent Rt estimates including only the effects of local vulnerability and variants. The pink band represents reductions in Rt associated with changing weather. The blue bands represent reductions in Rt associated with infection-acquired immunity (dark blue) and with behavior modification (i.e., masking use, mobility, and national airline travel, combined and depicted with light blue). The green bands represent reductions in Rt associated with policies (light green) and vaccination (dark green). The fitted values from the regression model are represented with the top of the black band. The solid vertical line represents January 1, 2021.

## Discussion

Deciphering SARS-CoV-2 transmission drivers throughout the pandemic can inform future mitigation policies and interventions for respiratory pathogens. Our analysis integrates spatial and temporal patterns of potential transmission determinants to assess associations between those determinants and SARS-CoV-2 transmission. We included a wide range of determinants with the goal of assessing the relative impact of each as the pandemic progressed. On aggregate, while both general and specific policies and behavior were associated with reduced SARS-CoV-2 transmission, we found that vaccination uptake, mask wearing, and reduced airline travel were associated with greater transmission reductions than government policies or mandates. Yet, the proportional reduction on transmission varied overtime; mask wearing and reduced airline travel had large impacts early on, while the relative contribution of immunity - due to previous infection and vaccination - increased as the pandemic continued, and behavior modification decreased. These findings imply that behavioral mitigation was critical to limiting transmission, reinforce the value of vaccines in returning to more normal behavior, and indicate the complexity of relationships between infection dynamics, behavioral choices, and government policies.

Early US COVID-19 mitigation strategies focused on physical distancing and masking policies. Our estimates suggest that physical distancing policies reduced SARS-CoV-2 transmission, with an estimated mean reduction of 1-3% for individual policies and an estimated total reduction of approximately 9% if sets of policies tracked by the OSI were at the highest observed levels in the US. These reductions are comparable with other, short-term, national-level assessments of NPI policies and showed reductions in COVID-19 cases [28–31], transmission [32–34], and deaths [29]; there was some reduction in effectiveness for policies with longer durations [29]. In contrast to our null results for masking policies, jurisdiction-specific studies show short-term effectiveness of mask mandates in reducing cases [35] and hospitalization growth rates [36]. Importantly, most of these early evaluations focused on the period in which the NPIs were in effect and did not assess long-term effects or periods when the NPI was not implemented, as was done here. There is some evidence of limited prolonged reduction in COVID-19 outcomes when NPI policies were lifted [37], with more socially disadvantaged communities experiencing greater rebounds in COVID-19 burden than other communities [38]. Overall, the association between individual policies and transmission reductions was strong but accounted for only a modest overall transmission risk reduction. This finding highlights the need to better understand adherence to the policies and what drivers personal behavior mitigations.

The relationships underlying policy implementation and behavior are intricate. Like others, we found indicators of behavior such as local mobility, national airline travel, and self-reported mask use to be associated with significant reductions in Rt although not necessarily temporally aligned with the corresponding policies [39]. For example, analysis of early mobility data showed that movement patterns changed drastically even before most physical distancing policies were implemented [40], with continued reductions in movement after policies were in place [41]. It is plausible that broad agreement within the physical distancing policies early in the pandemic [42] influenced personal choices to stay home. Similarly, self-reported mask use increased, and local mobility decreased rapidly as the Delta wave grew even though updated recommendations stated that vaccinated individuals could resume pre- pandemic activities without wearing a mask, once again, indicating a behavioral response that was independent of policy. Conversely, one example of temporal alignment between policy and behavior was immediately following the May 2021 guidance update when there was a rapid decrease in reported mask use and increases in local mobility, likely an effect of individuals returning to activities that had previously stopped due to the pandemic.

Before the Delta wave, US vaccination rates were steadily rising, with varying geospatial, socioeconomic, and race/ethnic coverage patterns [43,44]. Many jurisdictions also experienced substantial transmission in 2020 and early 2021, resulting in greater infection-acquired immunity. While we did not adjust for waning immunity [45] or for changing vaccine effectiveness with the appearance of new variants [45,46], we found associations between infection-acquired immunity and vaccination with decreased transmission. It is plausible that if we adjusted our model estimates for waning immunity, the strength of the immunity-related associations would be attenuated. In most states, the relative impact of either infection-acquired immunity or vaccination was as high as the decreasing impact of behavior change by November 2021, when the impact of vaccination was higher than the impact of infection-acquired immunity. There were also distinct jurisdictional differences, with some jurisdictions showing comparable impact by late 2021 (e.g., Alabama, Arkansas) and others showing much higher impact from vaccination than infection-acquired immunity (e.g., Connecticut, Hawaii). Additionally, we did not adjust our model for COVID-19 testing, which may have a non-linear relationship with seropositivity since testing availability and behaviors changed over time.

Social structure drives transmission patterns for all pathogens and the US COVID-19 epidemic highlighted existing social fault lines that influenced not only who in society was more likely to get infected, but also who was more likely to benefit from mitigation measures [47,48]. Here, we accounted for multiple social vulnerability indicators and found higher transmission rates in states with higher population density and greater racial and ethnic diversity. Stronger mitigation was needed to reduce Rt below the critical threshold of 1 for communities with higher vulnerability. This suggest that a one-size- fits all approach to mitigation is not appropriate and that more vulnerable communities may benefit from intentional support. Our findings align with several results from county-level analyses. For example, higher SARS-CoV-2 transmission rates occurred in low-income and racial minoritized communities [49]. Additionally, counties with higher social vulnerability were more likely to become a COVID-19 burden hotspot [50], that is a geographic area with elevated disease incidence [51]. Because structural racism is part of the intersectional factors comprising social vulnerability, it is unsurprising that hotspots were common in US counties with a greater percentage of non-White residents [52–54]. These findings highlight the importance of incorporating social markers of risk in infectious disease transmission models [55].

Our findings do not reflect causal relationships. Establishing causality between mitigation measures and transmission is complicated by a variety of risk factors, from SARS-CoV-2 variants emergence and changes in human behavior to environmental conditions– which all fluctuate over time and space. First, many important SARS-CoV-2 transmission determinants are correlated. To partially address this limitation, we removed highly correlated variables that measured similar factors when possible.

However, we did choose to retain some highly correlated variables, which may have influenced the findings. Determining the impact of these highly correlated variables, however, is not straightforward given the directionality of the conditional associations and the correlations between the covariates themselves. Second, jurisdiction-level policies may differ substantially from policies implemented at a county or city level. Additionally, many different policy variations were implemented (e.g., some jurisdictions required masks universally, whereas others only in certain locations or where physical distancing was not possible), even within jurisdictions, and those variations were not captured here. Third, we applied a regression framework, which assumes log-linear independence between covariates that does not account for the observed correlation between variables. Overall, the potential causal pathways between the predictors and Rt are not individually identifiable at this scale.

Ideally, the wealth of data and intervention diversity in the United States could be used to develop specific recipes for control. However, the diversity and correlation between many contributing factors make precise intervention estimates and/or combinations of interventions infeasible. Here, we conducted an ecological analysis of key strategy types and found that personal mitigation behaviors were more strongly associated with decreased transmission than policies. While most policies may not be sufficient to control COVID-19 on their own, a combination of policies and communication efforts that promote, support, and reinforce behavior change may be an essential pathway for mitigation. The other most impactful intervention was vaccination, a nationwide intervention that was not available early on but became as important as behavior modification for controlling transmission in most jurisdictions by mid to late 2021 [29]. Importantly, transmission was reduced not by a single measure, but by various layered measures; this indicates that no single measure is likely to control SARS-CoV-2 on its own [56]. These findings demonstrate the complexity of the COVID-19 response and SARS-CoV-2 transmission and illustrate the ongoing importance of layered mitigation approaches integrated across public health, government, and communities.

## Supporting information

Supplemental Material [[supplements/292882_file02.pdf]](pending:yes)

## Data Availability

COVID-19 case data are available upon request at [https://data.cdc.gov/Case-Surveillance/COVID-19-Case-Surveillance-Restricted-Access-Detai/mbd7-r32t](https://data.cdc.gov/Case-Surveillance/COVID-19-Case-Surveillance-Restricted-Access-Detai/mbd7-r32t). All other data are in the public domain and referenced in Supplement 1.

## CDC disclaimer

The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the U.S. Centers for Disease Control and Prevention.

## Data Availability

All data produced in the present work are contained in the manuscript

## Author Contributions

MAJ and VKL designed the research question, with assistance from SK and JMH. VKL designed the analysis plan, with components run by VKL, SK, PVP, and TC. VKL wrote the manuscript in consultation with BLC, JMH, RBS, MB, and MAJ. All authors provided critical feedback to help shape the analysis and manuscript. All authors accept full responsibility for the finished work and/or the conduct of the study, had access to the data, and controlled the decision to publish.

## Patient and Public Involvement

Because we used anonymous, public data, we did not include individuals that in the design, conduct, reporting, or dissemination plans of this work.

## Competing Interest Statement

The authors declare no competing interests.

## Funding

N/A

## Ethics Approval

No Ethics Approval needed.

## Acknowledgments

This research is based on survey results from Carnegie Mellon University’s Delphi Group. Thank you to Prabasaj Paul for his thoughtful comments and review of this manuscript.

## Footnotes

*   Additional results show COVID-19 mitigation policy impact within communities with differing levels of social vulnerability.

*   § See e.g., 45 C.F.R. part 46, 21 C.F.R. part 56; 42 U.S.C. §241(d); 5 U.S.C. §552a; 44 U.S.C. §3501 et seq.

*   Received July 19, 2023.
*   Revision received September 26, 2024.
*   Accepted September 26, 2024.


*   © 2024, Posted by Cold Spring Harbor Laboratory

This article is a US Government work. It is not subject to copyright under 17 USC 105 and is also made available for use under a CC0 license

## References

1.  1.Brauner JM, Mindermann S, Sharma M, Johnston D, Salvatier J, Gavenčiak T, et al. Inferring the effectiveness of government interventions against COVID-19. Science [Internet]. 2021 Feb 19 [cited 2022 Dec 1];371(6531). Available from: [https://www.science.org/doi/10.1126/science.abd9338](https://www.science.org/doi/10.1126/science.abd9338)
    
    

2.  2.Flaxman S, Mishra S, Gandy A, Unwin HJT, Mellan TA, Coupland H, et al. Estimating the effects of non-pharmaceutical interventions on COVID-19 in Europe. Nat 2020 5847820. 2020 Jun 8;584(7820):257–61.
    
    

3.  3.Haug N, Geyrhofer L, Londei A, Dervic E, Desvars-Larrive A, Loreto V, et al. Ranking the effectiveness of worldwide COVID-19 government interventions. Nat Hum Behav 2020 412. 2020 Nov 16;4(12):1303–12.
    
    

4.  4.Asfaw A. Racial and Ethnic Disparities in Teleworking Due to the COVID-19 Pandemic in the United States: A Mediation Analysis. Int J Environ Res Public Health 2022 Vol 19 Page 4680. 2022 Apr 13;19(8):4680.
    
    

5.  5.Rogers TN, Rogers CR, VanSant-Webb E, Gu LY, Yan B, Qeadan F. Racial Disparities in COVID-19 Mortality Among Essential Workers in the United States. World Med Health Policy. 2020 Sep 1;12(3):311–27.
    
    

6.  6.Gaitens J, Condon M, Fernandes E, McDiarmid M. COVID-19 and Essential Workers: A Narrative Review of Health Outcomes and Moral Injury. Int J Environ Res Public Health 2021 Vol 18 Page 1446. 2021 Feb 4;18(4):1446.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3390/ijerph18041446&link_type=DOI) 

7.  7. DiMaggio P. Culture and Cognition [Internet]. Vol. 23, Annual Review of Sociology. Annual Reviews; [cited 2018 Jan 5]. p. 263–87. Available from: [https://www.jstor.org/stable/2952552](https://www.jstor.org/stable/2952552)
    
    

8.  8.Grigsby-Toussaint DS, Shin JC, Jones A. Disparities in the distribution of COVID-19 testing sites in black and Latino areas in new York City. Prev Med. 2021 Jun 1;147:106463.
    
    

9.  9.Mody A, Pfeifauf K, Bradley C, Fox B, Hlatshwayo MG, Ross W, et al. Understanding Drivers of Coronavirus Disease 2019 (COVID-19) Racial Disparities: A Population-Level Analysis of COVID-19 Testing Among Black and White Populations. Clin Infect Dis Off Publ Infect Dis Soc Am. 2021 Nov 1;73(9):e2921–31.
    
    

10. 10.Smith TP, Flaxman S, Gallinat AS, Kinosian SP, Stemkovski M, Juliette H, et al. Temperature and population density influence SARS-CoV-2 transmission in the absence of nonpharmaceutical interventions. Proc Natl Acad Sci U S A. 2021 Jun 22;118(25):e2019284118.
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoicG5hcyI7czo1OiJyZXNpZCI7czoxODoiMTE4LzI1L2UyMDE5Mjg0MTE4IjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjQvMDkvMjYvMjAyMy4wNy4xOS4yMzI5Mjg4Mi5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

11. 11.Schoeman D, Fielding BC. Coronavirus envelope protein: current knowledge. Virol J 2019 161. 2019 May 27;16(1):1–22.
    
    

12. 12.Hallas L, Pyarali M, Hale T, Professor A, Brown A, Sudarmawan A, et al. Variation in US states’ responses to COVID-19 [Internet]. 2021 [cited 2022 Dec 2]. Available from: [https://www.bsg.ox.ac.uk/research/publications/variation-us-states-responses-covid-19](https://www.bsg.ox.ac.uk/research/publications/variation-us-states-responses-covid-19)
    
    

13. 13.Centers for Disease Control and Prevention (CDC). CDC COVID Data Tracker: Home [Internet]. [cited 2022 Dec 2]. Available from: [https://covid.cdc.gov/covid-data-tracker/#datatracker-home](https://covid.cdc.gov/covid-data-tracker/#datatracker-home)
    
    

14. 14.McAloon C, Collins Á, Hunt K, Barber A, Byrne AW, Butler F, et al. Incubation period of COVID-19: a rapid systematic review and meta-analysis of observational research. BMJ Open. 2020 Aug 1;10(8):e039652.
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYm1qb3BlbiI7czo1OiJyZXNpZCI7czoxMjoiMTAvOC9lMDM5NjUyIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjQvMDkvMjYvMjAyMy4wNy4xOS4yMzI5Mjg4Mi5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

15. 15.Cori A, Ferguson NM, Fraser C, Cauchemez S. A New Framework and Software to Estimate Time- Varying Reproduction Numbers During Epidemics. Am J Epidemiol. 2013 Nov 1;178(9):1505–12.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/aje/kwt133&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=24043437&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F26%2F2023.07.19.23292882.atom) 

16. 16.Alene M, Yismaw L, Assemie MA, Ketema DB, Gietaneh W, Birhan TY. Serial interval and incubation period of COVID-19: a systematic review and meta-analysis. BMC Infect Dis. 2021 Dec 1;21(1):1–9.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/s12879-021-05950-x&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F26%2F2023.07.19.23292882.atom) 

17. 17.Hale T, Angrist N, Goldszmidt R, Kira B, Petherick A, Phillips T, et al. A global panel database of pandemic policies (Oxford COVID-19 Government Response Tracker). Nat Hum Behav 2021 54. 2021 Mar 8;5(4):529–38.
    
    

18. 18.Salomon JA, Reinhart A, Bilinski A, Chua EJ, la Motte-Kerr W, Rönn MM, et al. The US COVID-19 Trends and Impact Survey: Continuous real-time measurement of COVID-19 symptoms, risks, protective behaviors, testing, and vaccination. Proc Natl Acad Sci U S A [Internet]. 2021 Dec 21 [cited 2022 Dec 2];118(51). Available from: /delphi-epidata/symptom-survey/
    
    

19. 19.COVID-19 Trends and Impact Survey | Delphi Epidata API [Internet]. [cited 2022 Dec 2]. Available from: [https://cmu-delphi.github.io/delphi-epidata/symptom-survey/](https://cmu-delphi.github.io/delphi-epidata/symptom-survey/)
    
    

20. 20.Paul P, France AM, Aoki Y, Batra D, Biggerstaff M, Dugan V, et al. Genomic Surveillance for SARS- CoV-2 Variants Circulating in the United States, December 2020–May 2021. MMWR Morb Mortal Wkly Rep. 2021;70(23):846–50.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.15585/mmwr.mm7023a3&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=34111060&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F26%2F2023.07.19.23292882.atom) 

21. 21.Carslaw D. Import Surface Meteorological Data from NOAA Integrated Surface Database (ISD) • worldmet [Internet]. 2021 [cited 2022 Dec 2]. Available from: [https://davidcarslaw.github.io/worldmet/](https://davidcarslaw.github.io/worldmet/)
    
    

22. 22.Global Hourly - Integrated Surface Database (ISD) | National Centers for Environmental Information (NCEI) [Internet]. [cited 2022 Dec 2]. Available from: [https://www.ncei.noaa.gov/products/land-based-station/integrated-surface-database](https://www.ncei.noaa.gov/products/land-based-station/integrated-surface-database)
    
    

23. 23.García-Carreras B,  T Hitchings MD, Johansson MA, Biggerstaff M, Slayton RB, Healy JM, et al. Accounting for assay performance when estimating the temporal dynamics in SARS-CoV-2 seroprevalence in the U.S. [cited 2022 Dec 2]; Available from: doi:10.1101/2022.09.13.22279702
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoibWVkcnhpdiI7czo1OiJyZXNpZCI7czoyMToiMjAyMi4wOS4xMy4yMjI3OTcwMnYxIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjQvMDkvMjYvMjAyMy4wNy4xOS4yMzI5Mjg4Mi5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

24. 24.COVID-19 Vaccination Trends in the United States,National and Jurisdictional | Data | Centers for Disease Control and Prevention [Internet]. [cited 2022 Dec 2]. Available from: [https://data.cdc.gov/Vaccinations/COVID-19-Vaccination-Trends-in-the-United-States-N/rh2h-3yt2](https://data.cdc.gov/Vaccinations/COVID-19-Vaccination-Trends-in-the-United-States-N/rh2h-3yt2)
    
    

25. 25.Smittenaar P, Stewart N, Sutermaster S, Coome L, Dibner-Dunlap A, Jain M, et al. A COVID-19 Community Vulnerability Index to drive precision policy in the US. medRxiv. 2021 May 20;2021.05.19.21257455.
    
    

26. 26.Vehtari A, Gelman A, Gabry J. Practical Bayesian model evaluation using leave-one-out cross- validation and WAIC. Stat Comput 2016 275. 2016 Aug 30;27(5):1413–32.
    
    

27. 27.Abbott S, Hellewell J, Thompson RN, Sherratt K, Gibbs HP, Bosse NI, et al. Estimating the time- varying reproduction number of SARS-CoV-2 using national and subnational case counts. Wellcome Open Res 2020 5112. 2020 Dec 8;5:112.
    
    

28. 28.Courtemanche C, Garuccio J, Le A, Pinkston J, Yelowitz A. Strong social distancing measures in the united states reduced the covid-19 growth rate. Health Aff (Millwood). 2020;39(7):1237–46.
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F26%2F2023.07.19.23292882.atom) 

29. 29.Lurie MN, Silva J, Yorlets RR, Tao J, Chan PA. Coronavirus disease 2019 epidemic doubling time in the United States before and during stay-at-home restrictions. J Infect Dis. 2020;222(10):1601–6.
    
    

30. 30.Siedner MJ, Harling G, Reynolds Z, Gilbert RF, Haneuse S, Venkataramani AS, et al. Social distancing to slow the US COVID-19 epidemic: Longitudinal pretest–posttest comparison group study. PLoS Med. 2020;17(8 August):1–12.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1371/journal.pmed.1003501&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F26%2F2023.07.19.23292882.atom) 

31. 31.Wellenius GA, Vispute S, Espinosa V, Fabrikant A, Tsai TC, Hennessy J, et al. Impacts of social distancing policies on mobility and COVID-19 case growth in the US. Nat Commun. 2021;12(1):6–12.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1038/s41467-020-20095-2&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=33397903&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F26%2F2023.07.19.23292882.atom) 

32. 32.Liu X, Xu X, Li G, Xu X, Sun Y, Wang F, et al. Differential impact of non-pharmaceutical public health interventions on COVID-19 epidemics in the United States. BMC Public Health. 2021 Dec 1;21(1):1–7.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/s12889-021-10961-z&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=33451323&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F26%2F2023.07.19.23292882.atom) 

33. 33.Dreher N, Spiera Z, McAuley FM, Kuohn L, Durbin JR, Marayati NF, et al. Policy Interventions, Social Distancing, and SARS-CoV-2 Transmission in the United States: A Retrospective State-level Analysis. Am J Med Sci. 2021 May 1;361(5):575–84.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.amjms.2021.01.007&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=33775425&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F26%2F2023.07.19.23292882.atom) 

34. 34.Yang B, Huang AT, Garcia-Carreras B, Hart WE, Staid A, Hitchings MDT, et al. Effect of specific non-pharmaceutical intervention policies on SARS-CoV-2 transmission in the counties of the United States. Nat Commun. 2021 Jun 11;12(1):3560.
    
    

35. 35.Van Dyke ME, Rogers TM, Pevzner E, Satterwhite CL, Shah HB, Beckman WJ, et al. Trends in County-Level COVID-19 Incidence in Counties With and Without a Mask Mandate — Kansas, June 1– August 23, 2020. MMWR Morb Mortal Wkly Rep. 2022 Nov 27;69(47):1777–81.
    
    

36. 36.Joo H, Miller GF, Sunshine G, Gakh M, Pike J, Havers FP, et al. Decline in COVID-19 Hospitalization Growth Rates Associated with Statewide Mask Mandates — 10 States, March– October 2020. MMWR Morb Mortal Wkly Rep. 2021 Feb 12;70(6):212–6.
    
    

37. 37.Singh S, Shaikh M, Hauck K, Miraldo M. Impacts of introducing and lifting nonpharmaceutical interventions on COVID-19 daily growth rate and compliance in the United States. Proc Natl Acad Sci U S A. 2021 Mar 23;118(12):e2021359118.
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoicG5hcyI7czo1OiJyZXNpZCI7czoxODoiMTE4LzEyL2UyMDIxMzU5MTE4IjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjQvMDkvMjYvMjAyMy4wNy4xOS4yMzI5Mjg4Mi5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

38. 38.Chang HY, Tang W, Hatef E, Kitchen C, Weiner JP, Kharrazi H. Differential impact of mitigation policies and socioeconomic status on COVID-19 prevalence and social distancing in the United States. BMC Public Health. 2021;21(1):1–10.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/s12889-021-10961-z&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=33451323&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F26%2F2023.07.19.23292882.atom) 

39. 39.Crane MA, Shermock KM, Omer SB, Romley JA. Change in Reported Adherence to Nonpharmaceutical Interventions During the COVID-19 Pandemic, April-November 2020. JAMA. 2021 Mar 2;325(9):883–5.
    
    

40. 40.Sanchez JN, Reyes GA, Martínez-López B, Johnson CK. Impact of social distancing on early SARS- CoV-2 transmission in the United States. Zoonoses Public Health. 2022;(November 2021):1–11.
    
    

41. 41.Gupta S, Nguyen T, Raman S, Lee B, Lozano-Rojas F, Bento A, et al. Tracking Public and Private Responses to the COVID-19 Epidemic. 101086/716197. 2021 Sep 1;7(4):361–404.
    
    

42. 42.Czeisler MÉ, Tynan MA, Howard ME, Honeycutt S, Fulmer EB, Kidder DP, et al. Public Attitudes, Behaviors, and Beliefs Related to COVID-19, Stay-at-Home Orders, Nonessential Business Closures, and Public Health Guidance — United States, New York City, and Los Angeles, May 5–12, 2020. MMWR Morb Mortal Wkly Rep. 2020;69(24):751–8.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.15585/mmwr.mm6924e1&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F26%2F2023.07.19.23292882.atom) 

43. 43.Berry CR, Fowler A, Glazer T, Handel-Meyer S, MacMillen A. Evaluating the effects of shelter-in- place policies during the COVID-19 pandemic. Proc Natl Acad Sci U S A. 2021;118(15):1–6.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1073/pnas.2023989118&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=33820825&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F26%2F2023.07.19.23292882.atom) 

44. 44.Pingali C, Meghani M, Razzaghi H, Mark ;, Lamias J, Weintraub E, et al. COVID-19 Vaccination Coverage Among Insured Persons Aged ≥16 Years, by Race/Ethnicity and Other Selected Characteristics-Eight. 2021;70(28). Available from: [https://www.cdc.gov/mmwr/mmwr_continuingEducation.html](https://www.cdc.gov/mmwr/mmwr_continuingEducation.html)
    
    

45. 45.Milne G, Hames T, Scotton C, Gent N, Johnsen A, Anderson RM, et al. Does infection with or vaccination against SARS-CoV-2 lead to lasting immunity? Lancet Respir Med. 2021 Dec 1;9(12):1450–66.
    
    

46. 46.Krause PR, Fleming TR, Longini IM, Peto R, Briand S, Heymann DL, et al. SARS-CoV-2 Variants and Vaccines. N Engl J Med. 2021 Jul 8;385(2):179–86.
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F26%2F2023.07.19.23292882.atom) 

47. 47.Egede LE, Walker RJ. Structural Racism, Social Risk Factors, and Covid-19 — A Dangerous Convergence for Black Americans. N Engl J Med. 2020 Sep 17;383(12):e77.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1056/NEJMp2023616&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F26%2F2023.07.19.23292882.atom) 

48. 48.Noppert GA, Zalla LC. Who Counts and Who Gets Counted? Health Equity in Infectious Disease Surveillance. Am J Public Health. 2021 Jun 1;111(6):1004–6.
    
    

49. 49.Bibbins-Domingo K. This Time Must Be Different: Disparities During the COVID-19 Pandemic. Httpsdoiorg107326M20-2247. 2020 Apr 28;173(3):233–4.
    
    

50. 50.Dasgupta S, Bowen VB, Leidner A, Fletcher K, Musial T, Rose C, et al. Association Between Social Vulnerability and a County’s Risk for Becoming a COVID-19 Hotspot — United States, June 1–July 25, 2020. Morb Mortal Wkly Rep. 2020 Oct 10;69(42):1535.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.15585/mmwr.mm6942a3&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F26%2F2023.07.19.23292882.atom) 

51. 51.Lessler J, Azman AS, McKay HS, Moore SM. What is a Hotspot Anyway? Am J Trop Med Hyg. 2017 Jun 6;96(6):1270.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.4269/ajtmh.16-0427&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F26%2F2023.07.19.23292882.atom) 

52. 52.Andersen LM, Harden SR, Sugg MM, Runkle JD, Lundquist TE. Analyzing the spatial determinants of local Covid-19 transmission in the United States. Sci Total Environ. 2021 Feb 1;754:142396.
    
    

53. 53.Maroko AR, Nash D, Pavilonis BT. COVID-19 and Inequity: a Comparative Spatial Analysis of New York City and Chicago Hot Spots. J Urban Health Bull N Y Acad Med. 2020 Aug;97(4):461–70.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s11524-020-00468-0&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F26%2F2023.07.19.23292882.atom) 

54. 54.Kim H, Zanobetti A, Bell ML. Temporal transition of racial/ethnic disparities in COVID-19 outcomes in 3108 counties of the United States: Three phases from January to December 2020. Sci Total Environ. 2021 Oct 15;791:148167.
    
    

55. 55.Zalla LC, Martin CL, Edwards JK, Gartner DR, Noppert GA. A Geography of Risk: Structural Racism and Coronavirus Disease 2019 Mortality in the United States. Am J Epidemiol. 2021 Aug 1;190(8):1439–46.56.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/aje/kwab059&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=33710272&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F26%2F2023.07.19.23292882.atom) 

56. 56.Reason J. Understanding adverse events: human factors. BMJ Quality & Safety. 1995 Jun 1;4(2):80–9.
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjY6IjQvMi84MCI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDI0LzA5LzI2LzIwMjMuMDcuMTkuMjMyOTI4ODIuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9)

 [1]: /embed/inline-graphic-1.gif