Abstract
Using publicly available data, we quantify the impact of mask adherence and mask mandates on COVID-19 outcomes. We show that mask mandates are associated with a statistically significant decrease in daily new cases (−3.24 per 100K), deaths (−0.19 per 100K), and the proportion of hospital admissions (−2.47%) due to COVID-19 between February 1 and September 27, 2020. These effects are large, corresponding to 13% of the highest recorded number of cases, 20% of deaths, and 7% of admission proportion. We also find that mask mandates are linked to a 23.4 percentage point increase in mask adherence in four diverse states, and that mask adherence is associated with improved COVID-19 outcomes. Lastly, using a large novel survey in 68 countries, we find that community mask adherence and attitudes towards masks are associated with a reduction in COVID-19 cases and deaths. Our results have relevant policy implications, indicating the need to maintain and encourage mask-wearing.
As of December 2020, SARS-CoV-2, the virus responsible for COVID-19 has infected at least 66 million people worldwide and caused more than 1.5 million deaths [1]. Numerous studies have analyzed the role played by masks during the COVID-19 pandemic [2–6]: masks have been associated with a reduction in the infection rate among health care workers in a large hospital network [7], mask mandates have helped reduce the number of cases in the United States and in Germany [8–10], and simulations have shown that wearing a mask can protect against droplet infection by preventing the spread of viral particles [11–16]. Despite this evidence, there has been strong resistance against mask-wearing, begging the question of the role played by attitudes towards masks in determining COVID-19 outcomes, and whether mask mandates could lead to an increase in adherence [17–19]. Investigating these questions is a key component of an ongoing effort to determine how to design, implement and sustain the adoption of non-pharmaceutical interventions (NPIs) to curb the spread of the COVID-19 pandemic [20–25].
A significant limitation in investigating the effect of mask mandates, mask adherence, and attitudes towards masks on COVID-19 outcomes has been the lack of a consistent dataset capturing key variables of interest. Existing data on mask mandates sometimes disagree due to variations in start date and in the type of mandates being considered [26–28]. Furthermore, while numerous datasets record daily changes in COVID cases and deaths [29–31], they are rarely accompanied by NPI information. Additionally, longitudinal data on mask adherence has been missing until recently, even though there are significant on-going surveys [32, 33] and computer vision-based efforts to estimate mask adherence from social media images and videos [34–37].
We therefore first create a dataset of state-level mask mandate start and end dates by manually reading each state government’s memos. We describe in detail our process for how we coded the mask mandate start and end dates for different types of mandates in the State-level Mandate Introduction Dates and Types section. We then undertake the task of unifying disparate public data sources* to create a longitudinal dataset that can be used to investigate the effect of mask mandates on COVID-19 deaths and cases, as detailed in Section 1.1.
We then build on recent work using event study designs to estimate treatment effects in the context of the COVID-19 pandemic: the effect of NPIs (e.g. business closures and stay-at-home orders) on the volume of online searches related to unemployment [38], on social distancing in the US [39], on the airline industry [40], on stock markets worldwide [41–44], and on COVID-19 cases at the county level [45]. For background, the effect of the introduction of new policies (here, mask mandates) has been studied with a variety of techniques such as difference-in-differences [46], event studies [47] and regression discontinuity [48].
The event study framework is particularly well-suited for our purposes as it allows for the same state to be used both as treatment and control based on the timing of a mandate, which is critical because states introduced mask mandates at different dates over the course of the pandemic [38]. While the first mandates were implemented in April, the latest ones occurred in November, some states still having had no mandate as of this writing (Table B in Appendix B).
An event study design allows us to estimate the treatment effect associated with mask mandates on COVID-19 outcomes on each day following the introduction of the mandate relative to the day prior to its introduction. Our geographical unit of analysis are states to minimize peer-effects of neighboring counties’ mandates due to the underlying interdependence between county mobility patterns [49], as well as the fact that people living in one county often have to travel to a different county to get medical care, resulting in inconsistent accounting of COVID-19 health outcomes at the county level [50]. By contrast, patients rarely cross state borders to get medical help as health insurance in the US is predominantly state-based. See appendix section 3.2.1 for the full model specification.
Although previous work has investigated the effect of mask mandates on COVID-19 outcomes [7–10], our results are novel because our event study specification simultaneously accounts for the following: all 50 US states and the District of Columbia, longer timescales (up to 50 days after the introduction of a mandate, and for the time period between February 1 and September 27, 2020 §), three COVID-19 outcomes: daily new confirmed daily cases, daily new confirmed deaths, and the proportion of daily new hospitalization admissions due to COVID-19.
To support our result of the effect of mask mandates on COVID-19 outcomes, we investigate the associated increase in mask adherence (i.e. the percentage of people who wear masks in public) following a mask mandate for the four states – Hawaii, Iowa, North Dakota and New Hampshire – for which adherence data [29] is available before and after the introduction of the mandate, as detailed in section 1.2. For the same states, we also investigate the effect of mask adherence itself on COVID-19 cases and deaths. As a final supporting analysis, we leverage a novel dataset [32] comprising 8 waves of survey data spanning July to October 2020 and totaling more than 479,000 responses from 68 countries in 51 languages to estimate the effect of weighted (weights are used to obtain more representative samples) self-reported mask adherence as well as attitudes (towards masks on COVID-19 cases and deaths) at the community level, controlling for population density, human development and mobility, as detailed in section 1.3. The survey dataset (de-identified) is available to academic and non-profit researchers‡.
1 Results
1.1 Mask Mandates and COVID-19 Outcomes
We estimate the effect of mask mandates on three prominent COVID-19 outcomes: 1) number of new confirmed cases, 2) proportion of newly-admitted COVID-19 related hospital admissions (i.e. the number of patients admitted to treat COVID-19 symptoms relative to the total number of admitted patients), and 3) number of deaths. We ensure to only use new cases, hospital admissions and deaths each day, as opposed to cumulative numbers. Outcome variable characteristics (Z-normalization, smoothing, date ranges, etc.) are defined in appendix table 1. Our estimation controls for the level of human mobility (using different types of mobility metrics [51]) as a proxy for the introduction of other non-pharmaceutical interventions, the number of COVID-19 tests being administered [30], seasonal effects (proxied by temperature and precipitation data [52]), and idiosyncratic state and time effects. We regress each COVID-19 outcome on day t in state s to estimate the treatment effects of mask mandates γτ, where τ is the number of weeks before or after the start of a mandate (see Eq. 1 in the Methods section for the full specification).
As show in fig. 1, mask mandates (here defined as the earlier between mandates for all employees or mandates for all members of the public – we later do robustness checks to relax these assumptions) were introduced at day zero, the red vertical line. The y-axis represents the treatment effect associated with mask mandates on the COVID-19 outcome for each day before or after the introduction of a mask mandate relative to the day before the introduction of a mask mandate (as is standard in an event study framework [38] because we normalize γτ =−1 = 0). For all three outcomes, although the pre-treatment effect is statistically indistinguishable from zero for the 6 days preceding the introduction of the mandates, it exhibits a slight upward trend, reflecting the fact that mandates were in many cases introduced as a response to cases, hospitalizations and deaths starting to surge.
Using an event study design, we estimate the treatment effect of the introduction of mask mandates (shown on the vertical red line) on Z-scored population-normalized COVID-19 daily new confirmed cases, daily new hospitalization admissions proportion, and deaths across all 50 states and D.C. over the time period between February 1 and September 27, 2020. Shaded area represents 95% confidence interval, with standard errors clustered at the state level. (A) It takes 11 days for the number of cases to start decreasing. After 50 days, the number of cases decreased by −0.45 [−0.68,-0.26] standard deviation, which corresponds to a decrease of 3.24 cases per 100K or 13% of the highest recorded number of new cases per 100K prior to Sep 27 2020 (adjusted R2 is 0.431 and p < 0.001). (B) We also observe a decrease in the proportion of COVID-19 related hospitalization after 8 days, reaching −2.47 [−3.39,-1.54] percentage points after 50 days, compared with a highest recorded of 34% during our observation period (adjusted R2 is 0.511 and p < 0.001). (C) It takes 19 days for the number of deaths to start decreasing, reaching −0.80 [−1.02,-0.59] standard deviations 50 days later (adjusted R2 is 0.304 and p < 0.001). This corresponds to a decrease of 0.19 deaths per 100K, or 20% of the highest recorded number of new deaths during our observational period.
For daily new cases, we observe a delay of about 11 days after the mandates are introduced for the treatment effect to start increasing (i.e. going negative). The treatment effect continues to increase (i.e. its magnitude becomes more negative) reaching −0.45 standard deviations (95% confidence interval [−0.68,-0.26]) 50 days after the introduction of mask mandates (adjusted R2 is 0.431 and p < 0.001). The magnitude of the associated treatment effect is large, corresponding to 3.24 cases per 100,000 people or 13% of the highest recorded number of new cases per 100,000 people during our observation period.
The delay for mask mandates to have an associated treatment effect on COVID-19 related deaths is 19 days after the start of a mandate. This 8-day time lag between the decrease in the number of cases and the number of deaths is in line with the temporal pathogenesis characteristics of the virus as the interquartile range of the time between symptom onset and death is 8-26 days [53–60]. After 19 days, the treatment effect continues to increase (adjusted R2 is 0.304 and p < 0.001) reaching −0.80 [−1.02,-0.59] standard deviations after 50 days which corresponds to 0.19 deaths per 100K, or about 20% of the highest recorded number of daily COVID-19 related deaths observed during our observational period.
Finally, we also estimate the effect of mask mandates on the only type of hospitalization data available at the state level for all states since the beginning of the pandemic (here, as early as February 1, 2020): the proportion of daily hospitalization admissions‡ [29] due to COVID-19. It is important to note that we only observed the proportion of hospitalizations, which is the the number of new hospitalizations due to COVID-19 relative to the number of total admissions. Unfortunately, this proportion is a noisy proxy for the number of hospitalizations per 100K due to COVID-19 (that we would like to observe) as it is well documented that hospital capacity was deliberately increased to accommodate surges in the number of hospitalizations and that fewer people went to the hospital as regular admissions were discouraged and elective treatments postponed during the pandemic [61–64] which affects the denominator of the hospitalization proportion. However, even though noisy, we investigate the effect of mandates on the proportion of hospitalization as it still provides a useful perspective of the effect of mask mandates on COVID-19 outcomes.
Similarly to cases and deaths, we find that the introduction of mask mandates leads to a statistically significant decrease in the proportion of COVID-19 related hospitalization admissions after 8 days and it continues to decrease thereafter (adjusted R2 is 0.511 and p < 0.001). 50 days later, the proportion of COVID-19 related hospitalization admissions decreased by −2.47 [−3.39,-1.54] percentage points (not standard deviations, since this outcome is an already normalized percentage). For reference, this corresponds to a 7% decrease compared to the highest recorded proportion (34%) of COVID-19 related hospitalization admissions during our observation period.
Overall, these results suggest that the introduction of mask mandates have led to a decrease of 3.38 daily new cases per 100K, 0.19 daily deaths per 100K and −2.47 percentage points of hospitalization proportions nationally (since we include all states and D.C.) 50 days later. Full regression results are available in appendix table 4 in appendix C.
All associated treatment effect estimates include state and time fixed effects, and control for a number of confounding factors: (a) indicators of human mobility [51] (number of visits to recreational areas, grocery stores, pharmacies, parks, transit stations, workplace and residential areas) to control for variations in the spread of the virus due to variations in mobility and as a proxy for other NPIs such as stay-at-homes (as they impact mobility), (b) daily new test rate [30] to control for variations in the number of tests being administered, and (c) temperature and precipitation [52] to control for weather-induced behaviors. We ensure that all our variables have been smoothed and de-trended (for day of the week variations), and we report standard errors clustered at the state-level. We also include squared values of each mobility indicator to proxy for social contacts ( [65, 66]). See appendix table 2 for descriptive statistics on the control variables being used.
1.1.1 Robustness Checks
We conduct a number of checks to test the robustness of our results. First, we experiment with changing how mask mandates are defined. As noted earlier, so far, we have defined the start of a mask mandate as the earlier date between mandates for all employees or mandates for all members of the public (more on the different types of mandates in the State-level Mandate Introduction Dates and Types section). Here, we consider two alternative specifications in which we only consider mask mandates requiring everyone to wear a mask outside, and another specification where we only consider mandates requiring only business employees to wear masks. The former specification is more stringent than our original specification (the earlier of the two types of mandates) as these public mandates generally came later in the year than those focusing on business employees only, and more states still do not have the public mandates. As shown in appendix fig. 5 in appendix C for the public mandates and in appendix fig. 4 for the business employee mandates, the trajectory and magnitude of the treatment effect associated with mask mandates is consistent with our earlier estimates (shown in fig. 1). Specifically we observe that the business employee mandates specification treatment effect is very similar to our earlier specification results which is because almost all states had business employee mandates before public mandates, and our previous specification took the earlier start date between business employee mandates and public mandates. We also observe that the treatment effect associated with mandates that require all members of the public to wear masks takes longer to have an effect, but eventually reaches the same magnitude of effect size. We hypothesize that this delay is due to the fact that public mask adherence takes some time to increase following the introduction of a mask mandate, which we investigate in the next section. Overall, because the trajectory and magnitude of the treatment effect is consistent across all three specifications, this suggests that our treatment effects are not overly sensitive to our definition of mask mandates.
As a second robustness check, we recompute our treatment effect estimates on the number of deaths and the proportion of hospitalizations but this time controlling for the number of cases. As shown in appendix fig. 6 in appendix C, controlling for the number of cases does not significantly change the trajectory of the effect of mask mandates on deaths and hospitalization. As expected because cases on the causal pathway to deaths and hospitalizations, the more severe outcomes, the treatment effect size is decreased. Because our effect size is still strong and the trajectory still consistent even after controlling for cases, this provides additional evidence of the positive effect associated with mask mandates on COVID-19 outcomes.
So far, we have always looked at all 50 states and the District of Columbia together. Even though we control for state-specific trends, here we investigate the heterogeneity of treatment effect between earlier and later-wave states as it is possible that the effect of mask mandates on states that were part of the COVID-19 pandemic’s early wave might be different from states that were affected later. For example, mask shortages were present during the early stages of the pandemic [67–69] which might affect the proportion of people who wear masks (mask adherence) following a mask mandate for early states compared to later states. Therefore, as a robustness check, we recompute our treatment effect estimates but this time looking separately at the first 15 states to have the highest number of cases per 100K during the month of April (namely, NY, NJ, MA, CT, MI, DC, RI, IL, WA, PA, GA, VT, MD, FL, LA) compared to the rest of the states (i.e. later wave states). As shown in appendix fig. 8 in appendix C, masks mandates took more time to have an effect on earlier states (around 20 days) whereas they had a faster effect on later states but the trajectory and magnitude of the treatment effect overall is consistent with our previous main result, suggesting that our specification is not sensitive to subsets of earlier wave, later wave or all states.
Recent work has shown that declines COVID-19 outcomes are not only due to the introduction of policies such as mask mandates but are also a result of private behavioral responses to observations of increased outcome numbers [70]. To investigate if our treatment effect is confounded by behavioral responses, we control for past outcome values and growth rates (conditioning on the delayed outcome and growth rate will result in the confounding path to be blocked - see [70] for more details about the causal graphical model). As shown in supplementary figure 7, including a 14-day delayed outcome and growth rate as control does not significantly change the magnitude and trajectory of our associated treatment effect, suggesting that confounding due to behavioral responses does not play a strong role.
As a last robustness check, we investigate if we are seeing decreases in COVID-19 outcomes because the number of new COVID-19 tests done might be decreasing leading us to observe, erroneously, a lower rate of COVID-19 outcomes. Although we already control for the testing rate in our event study specification, here, we also investigate if testing rates were themselves decreasing during the period of observation, in a way that is associated with the introduction of mask mandates. We do so in two ways: first, we look at the per-state and mean (over all states) testing rate (daily new tests per 100K) every week, and as shown in appendix fig. 9 in appendix D, it does not decrease but instead increases over our period of investigation. As a further test, we conduct an event study estimation to investigate a confounding treatment effect associated with mask mandates on testing rate. As shown in fig. 10, there is no associated treatment effect of mask mandatse on testing rate. Therefore, it is unlikely that our estimates of decreasing COVID-19 outcomes after the introduction of a mask mandate are due to decreased detection of COVID-19 outcomes.
Overall, our analysis shows that the introduction of mask mandates at the state-level is associated with a statistically significant and large decrease of cases, hospitalizations, and deaths.
1.2 Mask Mandate and Mask Adherence
Here, we investigate whether the introduction of mask mandates is associated with a significant increase in mask adherence i.e., do mask mandates lead to more people wearing masks? Although there are significant on-going efforts to estimate mask adherence at a population level such as by leveraging computer vision on social media data [34–37], data that tracks mask usage at a county or state level over time has is only now starting to be available: as of September 8, 2020, the Delphi project [29] has started publicly releasing mask adherence data that measures the estimated percentage of people who wore a mask most or all of the time while in public in the past 5 days by asking the question “In the past 5 days, how often did you wear a mask when in public?” in an on-going Facebook online survey ran on millions of people in the US. We therefore use their weighted state-level estimates of mask adherence as our dependent variable. Their weighting strategy to obtain representative samples is described briefly in the State-level COVID-19 Outcome and Mask Adherence Data section, and comprehensively described in their technical report [33].
During the period between September 8 (when adherence data started being available) and November 30, 2020 (present), only 4 states have enacted new state level requirements: Hawaii and Iowa on November 16, North Dakota on November 14 and New Hampshire on November 20. Although some of these states had county-level mask mandates, had memos encouraging mandates at county or state-level, or had mask requirements in certain businesses, they did not have a state-wide requirement with implementation until the dates we outline. We run a similar treatment effect estimation as earlier but this time using mask adherence as the outcome variable and for at most 13 days after mandate introduction because this is the longest time horizon we have simultaneously for all four states.
As shown in fig. 2, there is a flat pre-treatment trend up to 8 days before the introduction of a mask mandate followed by an uptick in mask adherence after the introduction of mask mandates with a maximum increase in adherence of 23.4 [12.0,34.8] percentage points (adjusted R2 is 0.925 and p < 0.001) 13 days after mask mandate introduction. It is important to note that we observe a delay of about 6 days before we observe a statistically significant increase in mask adherence. We hypothesize that if we had adherence data for earlier states, we would observe an even larger delay because of the mask shortages that were present during the early stages of the pandemic [67–69].
Event study estimates of the treatment effect of mask mandates on mask adherence in the only four states (Hawaii, Iowa, North Dakota and New Hampshire) which had late mask mandates (Nov 14, 16, and 20) during the period we have daily state-level mask adherence data. We find that there is a flat pre-treatment trend up to 13 days before the introduction of a mask mandate followed by a 23.4 [12.0,34.8] percentage points increase in mask adherence following introduction (adjusted R2 is 0.925 and p < 0.001). This result suggests the strong effect of mask mandates on mask adherence. We expect our results to be a fair estimate of treatment effect in the US due to the fact that these four states are very geographically, culturally and politically diverse.
Again, we include state and time fixed effects, and control for a number of factors: testing rate due to its potential effect on mask usage as more people might wear masks if they are more regularly tested, and mobility as we want to control for the fact that perhaps the more people leave their house, the more they might wear masks. Additionally, we control for the number of deaths and confirmed cases because we want to control for the fact that the more cases and deaths people know about, the more they might be likely to wear masks (although in practice, omitting these two controls leads to very little difference.
As a robustness test, we investigate if our estimation changes significantly if we remove testing rate, cases and deaths as controls. As seen in appendix fig. 11 in appendix E, our estimation is robust to these controls: the treatment effect is noisier but consistent with our current result (when controlling for cases, deaths and testing rate). Full regression results are shown in table 12.
As an additional result, because we have a limited window of time where we have both COVID-19 out-comes (only cases and deaths, no hospitalization data as access ended in September) and mask adherence data, we can use it to investigate if mask adherence is more directly associated with a positive effect on COVID-19 outcomes. To do so, we implement a multi-linear regression of COVID-19 outcomes on compliance, controlled, as before, by a number of factors as specified in equation 3.2.2. Note that we use all states (including D.C.) here, not just the four states that had mandates after Sep. 8th. As shown in appendix table 13 in appendix F, an increase of 1% in mask adherence leads to a decrease of −1.63 [−1.76, −1.50] new confirmed cases per 100K (adjusted R2 is 0.502 and p < 0.001), and −0.016 [−0.015, −0.018] new deaths per 100K (adjusted R2 is 0.343 and p < 0.001), controlling for test rates, weather and mobility. For reference, the rate of COVID-19 outcomes was 25.4 new daily cases per 100K and 0.95 daily new deaths per 100K, suggesting that even a 1% increase in masking adherence can have a significant positive effect.
Although we only estimate the effect of mandates on adherence in these 4 states due to the fact that they are the only ones that implemented mask mandates late enough that we have adherence data for them, we expect this result to support our main result that mask mandates are associated with a significant decrease in COVID-19 outcomes, especially due to the fact that these four states (Hawaii, Iowa, North Dakota and New Hampshire) are very geographically, culturally and politically diverse.
1.3 International Mask Adherence and Attitudes
Here, as a supporting result of the effect of mask mandates on COVID-19 outcomes, we expand the scope of our analysis and look at the effect of mask adherence internationally. Additionally, we also look at attitudes towards mask wearing. We do so by using the novel survey-based ‘COVID-19 Beliefs, Behaviors & Norms Survey’ dataset which asks more than 150 questions (see [32] for the complete survey instrument) about COVID-19 to more than half a million online Facebook survey respondents in 51 languages from 68 countries. The survey has currently been deployed in 8 waves starting on July 7, 2020 and is still on-going. Because the 68 countries (full list in the Data section) we include in our analysis are from very different parts of the world and were in various stages of the pandemic (first wave, second waves, before mandates, after mandates, etc.) during the 4 months of our observation period, we expect that our estimates are fairly representative.
We focus on the two questions most relevant to the effect of masks on COVID-19 outcomes: a question about mask adherence: “Out of 100 people in your community, how many do you think wear a face mask or covering when they go out in public?”; and a question about mask norms: “Out of 100 people in your community, how many do you think believe the following because of COVID-19: People should wear a face mask or covering when out in public?”. The first question is a self-reported sample estimate of the percentage of a respondent’s community that wears masks in public, and the second question provides a sample estimate of the percentage of the community that believes masks to be important to wear in public.
We use a weighted survey regression approach to regress each country’s new daily deaths per 100K and cases per 100K (both smoothed) against the self-reported mask adherence and mask attitude weighted response sample with countries as survey strata and individual anonymized survey responses as survey clusters. Samples are expected to be representative because we use a unique weight for each sample which correct for a variety of biases including demographics (age bracket and sex) of the respondents compared to the census data in each country and compared to Facebook’s online population (through post-stratification), and for response and non-response drivers (the estimated design effects was below four as detailed in [32]). The dataset’s weighting strategy to obtain representative samples is described more in the International Mask Norm and Adherence Data section, and comprehensively described in their technical report [32]. We also control for mobility, new test rate, population density and the country’s human development index. The regression specification is described in equation 3.2.3.
As shown in fig. 3, the associated effect coefficients suggest that a 1% increase in mask adherence is associated with a decrease of −0.45 [−0.70,-0.29] cases per 100K and a decrease of −0.042 [−0.046,-0.037] deaths per 100K. Similarly, we find that a 1% increase in attitude about the importance of wearing masks leads to a decrease of −0.53 [−0.64,-0.42] cases per 100K and a decrease of −0.035 [−0.038,-0.032] deaths per 100K. For reference, the current statistics in the world as of December 14, 2020 is 10 daily new cases per 100K and 0.17 daily deaths per 100K, which suggest that even a 1% increase in mask adherence and attitudes is associated with a strong positive impact on COVID-19 outcomes worldwide. Full regression results are shown in appendix fig. 14 in appendix G.
Using a novel survey-based dataset of more than half a million online Facebook survey respondents in 51 languages from 68 countries, we estimate that a 1% increase in mask adherence is associated with a decrease of −0.45 [−0.70,-0.29] cases per 100K and a decrease of −0.042 [−0.046,-0.037] deaths per 100K. Similarly, we find that a 1% increase in attitude about the importance of wearing masks leads to a decrease of −0.53 [−0.64,-0.42] cases per 100K and a decrease of −0.035 [−0.038,-0.032] deaths per 100K. For reference, the current COVID-19 outcomes in the world as of December 14, 2020 is 10 daily new cases per 100K and 0.17 daily deaths per 100K, which suggest that mask adherence and attitudes are associated with a strong positive impact on COVID-19 outcomes worldwide. Samples are expected to be representative because we use a unique weight for each sample which corrects for a variety of biases.
As a robustness check, we perform the same weighted survey regression but this time dis-aggregating by survey waves. As shown in appendix fig. 12 and 13 in appendix G, the regression coefficients of mask adherence and attitudes on deaths and cases are overwhelmingly negative, which supports our results that mask adherence and attitudes have a positive impact on COVID-19 cases and deaths across waves. Full regression results tables are shown in appendix G.
2 Discussion
Our main result is that mask mandates are associated with a significant improvement in COVID-19 outcomes (corresponding to 13% of the highest recorded number of daily new cases, 20% of deaths, and 7% of hospital admissions). In support of our main result, we also observe that mask mandates are associated with a 23.4 percentage point increase in mask adherence in four geographically, culturally and politically diverse states, and that mask adherence is itself associated with a significant decrease in COVID-19 cases and deaths. Finally, we observe that community mask adherence and attitudes towards masks are associated with fewer cases and deaths in 68 countries. Taken together, these results strongly suggest the positive effect of mask mandates, mask adherence and mask attitudes on COVID-19 cases, hospitalizations and deaths.
As a limitation, we acknowledge that these treatment effects are potentially overestimates of the effect of mandates and that our estimation is not causal – both limitations are shared by recent work investigating the role of mandates on COVID-19 outcomes [7–12]. For instance, although we use mobility data as a proxy to control for other NPI policies such as stay-at-home orders (since such policies strongly affect mobility), we expect that without a proper study design that aims to estimate the marginal effect of different NPI policies simultaneously, our estimands are potentially overestimated. Another potential source of over-estimation is that although we observe flat pre-treatment anticipatory policy effects before the mask mandates were put in place and that we control for past outcomes and growth rates to minimize confounding due to behavioral responses [70], some bias from behavioral changes might still persist. On the other hand, it is possible that our treatment effects are underestimates due to the peer effects caused by neighboring states’ mask mandates affecting other states [49]. We leave to future work the design and estimation of such causal, marginal, and peer-effect estimations, and expect that our findings will still provide new insights into the long-term effect of mask mandates, mask adherence and mask attitudes on COVID-19 cases, hospitalizations and deaths.
Another limitation is that although our survey-based sample estimates of mask adherence and mask attitudes include weights aimed to to correct for a number of biases (including demographics of the respondents compared to the census data in each country and compared to Facebook’s online population through post-stratification, and response and non-response drivers [32, 33]) in order to obtain more representative samples, it is possible that there is still some bias in our sample estimates. Ongoing efforts using computer vision to estimate mask adherence from social media images and videos [34–37] should soon provide new data on mask adherence, perhaps going back to beginning of the pandemic. Additionally, we do not account for public health messaging [71] or the effect of daily updates on cases, deaths and hospitalizations from the news [72]. We leave to future work the incorporation and modeling of such factors on COVID-19 outcomes.
Our results have potential policy implications by reinforcing the need for maintaining and encouraging mask-wearing by the public. These results are especially significant in light of some states starting to remove their mask mandates and attempts by state governments to prevent local governments from implementing mask orders [26, 28]. Removal of mask-mandates or a decrease in mask adherence could result in significant increases in COVID-19 cases, hospitalizations and deaths. Maintaining mask mandates is especially important given our observation of their positive health impact, while previous research has shown their minimal economic impact (especially compared with other measures such as stay-at-home orders or business closures) [73].
From a behavioral perspective, more research is needed to investigate the optimal incentives, community health communication, and regulations to increase mask-adherence [74–78], especially in a climate where conspiracy theories are emerging to discourage mask-adherence [79]. More research is also needed to probe potential improvements in public health communication strategies to encourage mask-adherence, including harm-reduction frameworks [80] and policies tailored to help disadvantaged communities’ lived experiences [81]. For example, lockdowns and business closures have not significantly decreased the infection rates of people from poorer communities as they cannot stop going to work [65] while unfortunately having a disproportionate impact on their financial health [73]. This is especially important as similar demographic drivers behind mask avoidance [17–19, 82] might impact successful deployment of vaccines.
3 Methods
3.1 Data
As detailed in the 3.1 section in the appendix, we obtain data from a large variety of data sources. For the main event study investigating the effect of mask mandates on COVID-19 outcomes, we start with de-trended (for day of the week variations), smoothed and normalized daily state-level values for all variables (cases, hospitalizations, deaths, mobility, test rate, weather). We careful select an observational time period where we have data for all variables, Feb 1st to September 27th (which is when access to hospital data ended). For the event study of the effect of mask mandates on adherence, and for the multi-linear regression of adherence on COVID-19 outcomes, we do the same but at a daily level. For the survey regression of mask adherence and attitudes on COVID-19 outcomes, we do a similar data fusion exercise as before, but this time merging daily at the international level.
3.2 Analysis
3.2.1 Event study of mask mandates on COVID-19 Outcomes
We obtain COVID-19, mobility, new test rate and weather data at the daily level as outlined in the Data section. We then calculate the number of weeks between each day of data (in the time interval from Feb 2, 2020 to September 27, 2020) and the start of mask mandate implementation for each state. We do not use data past the end date of when a state has lifted mask mandate.
Our specification is:
where the outcome is Co,t,s (o ∈ {new deaths due to COVID-19, new confirmed cases of COVID-19, hospitalization proportion due to COVID-19}) for each day t in state s. The goal is to estimate treatment effects γτ where τ is the number of days before or after the start of a mask mandate. 1 is the delta function such that γτ is non-zero only when the number of weeks rs,t relative to the start of mask implementation is τ, αs and αt are state and time fixed effects, α′st is a state-specific trend coefficient multiplying the time trend variable to allow for the fact that different states had different outcome trajectories (see [46] for more details), Xt,s are our controls, and ∊Et,s represent the model residuals.
Because some states such as New York had a much higher COVID-19 incidence than other states, we used population-scaled (per hundred thousand people) outcomes, and we further Z-scored the daily value of each outcome variable using state-specific means and standard deviations. Due to these transformations, our treatment effect estimates are expressed in units of standard deviations.
While political affiliation has been shown to correlate the propensity to wear a mask [83], this effect should captured in the state fixed effect since we do not expect political affiliation to change significantly during our observational period.
We normalize γτ =−1 = 0 and cluster standard errors at the state level. We use the R package’s lfe’s regression function felm to run this specification as it uses the method of alternating projections to speed up regression on our large datasets [84].
As a robustness check, we repeat employ a similar event study specification to verify that the decrease in COVID-19 outcomes we observe is not due to a decrease in test rate (per 100K of population. We use the following specification:
where T Rt,s is the daily new test rate, and all other variables are similar to above, with 
being our controls without test rate (since it is now the dependent variable).
3.2.2 Mask Mandate and Mask Adherence
Using daily mask adherence data from September 8 to November 30, 2020, we estimate a similar event study specification to the above using mask adherence As,t as the dependent variable. Additionally, we control for deaths and confirmed cases because we want to control for the fact that the more case and deaths people see, the more they might be likely to wear masks (although in practice, omitting these two controls leads to practically no difference), and we do not add state and time fixed effect due to the very small number of states considered and the very short time interval without noticeable difference). As before, we also control for restaurant and recreation, grocery and pharmacy, parks, transit stations, workplace and residential percentage change from baseline (using the median day-of-the-week value from the first 5 weeks of January and February) mobility data.
We implement a multi-linear regression of COVID-19 outcomes on compliance, controlled by a number of factors, as specified below:
where Co,t,s are the daily t COVID-19 outcomes o ∈ {deaths, cases} and 
are the mobility, new test rate and weather controls.
3.2.3 International Mask Adherence and Norms
Here, we use multi-linear regression to estimate the effect of mask norms or mask adherence on COVID-19 case and death outcomes for each wave w:
where Co,t,k are the daily t COVID-19 outcomes o ∈ {deaths, cases} for each country k, for question q, controlling for mobility and new test rate 
One separate model estimation is run for each wave w, each outcome o and each question q ∈ {mask norms, mask adherence}.
More details about survey weights and the dataset can be found in the 3.1 section. We also verify that the weighted average mask adherence, weighted average mask attitude and total number of responses did not change significantly over time during the survey as shown in appendix figures 15, 16 and 14 in appendix G.
We use a complex survey regression (using R’s survey package) with countries as survey strata and individual anonymized survey responses as clusters. We only select responses that finished the whole survey (including several attention check questions), and we remove responses with missing values for the questions under consideration, and also those with missing weights (<1% of responses).
The 68 countries in our dataset are: United Kingdom, Poland, Italy, Germany, Japan, Argentina, France, Turkey, Mexico, Colombia, United States, Pakistan, Romania, Indonesia, Philippines, Egypt, Malaysia, Bangladesh, Nigeria, Brazil, Thailand, Vietnam, India, Netherlands, Azerbaijan, Australia, South Africa, Canada, Estonia, Senegal, Afghanistan, Tanzania, Angola, Ecuador, Georgia, Mongolia, Peru, Algeria, Mozambique, Bolivia, Portugal, Iraq, Cameroon, Morocco, South Korea, Uruguay, Honduras, Nepal, Sri Lanka, United Arab Emirates, Spain, Cote d’Ivoire, Myanmar, Chile, Venezuela, Guatemala, Trinidad & Tobago, Sudan, Kenya, Jamaica, Ghana, Uganda, Ukraine, Taiwan, Singapore, Cambodia, and Kazakhstan.
Data Availability
All data and code are available at https://github.com/d-val/Mask_Mandates_Adherence_Attitudes
Appendix
A Data
A.1 State-level Mandate Introduction Dates and Types
To code the start (and, if any, the end) of a state’s mask mandate, we start with publicly available mask mandate datasets [26–28] and use their references to official state government memos. We use these memos to define the starting date of a state mandate to be the one that requires the wearing of masks either in public or in businesses – as opposed to using county-level mask mandates, or using mandates that merely encouraging people to wear masks. Some states had several memos within a couple of days requiring mask mandates (e.g., when mask mandates are bundled with other NPI’s) in which case we choose the earliest memo’s dates. Some states also had memos announcing the future start of a mask mandate and we use the start of the implementation of the mask mandates as the start date. Finally, some states had memos requiring all employees of all businesses to wear masks, and, typically later, memos requiring all members of the public to wear masks. We investigate all three possibilities of which type of mandates count as the start of a treatment: just mandates requiring business employees, just memos asking the public, or using whichever came first as the start date of a mandate.
Our table of mask mandate start and end dates is shown in supplementary table B in appendix B.
A.2 State-level COVID-19 Outcome and Mask Adherence Data
State-level COVID-19 outcome data was obtained from the Delphi project [29]: we use the daily new confirmed cases per 100K population, new confirmed deaths per 100K population, percentage of new hospital admissions with COVID-associated diagnoses, and the percentage of outpatient doctor visits primarily about COVID-related symptoms. In all cases, we use the smoothed outcomes measures using 7-day averages and with systematic day-of-week effects removed. Our outcome data covers all 50 states in addition to the District of Columbia, and runs from Feb 2, 2020 to September 27, 2020 (we cannot obtain data past September for some some outcomes such as hospitalization). 43 states and the District of Columbia had a required state-level mask mandates during this period of data. We make sure not to use any outcome data that was after a state’s mask mandate was officially lifted to prevent end-of-treatment effects from biasing our treatment estimates. Overall, this dataset contains 9,483 data points used for regression.
State-level mask adherance data (how much people are wearing face masks) was not available until the Delphi project [29] made it accessible on September 8, 2020. This data was obtained by the Delphi Project via online surveys of more than 1 million U.S. residents and we use their weighted state-level estimates as our indicator of state-level mask adherance. Their survey weighing [33] accounts for various factors including adjusting estimates so that they are representative of the US population according to a state-by-age-gender stratification, adjusting for the US Facebook user population, and adjusting for the propensity of a Facebook user to take the survey. For the regression of the effect of mask mandates on adherence, we have 323 data points for the four states, and for the regression of adherence on COVID-19 cases and deaths, we have 4,111 data points for all states (including D.C.).
A.3 International Mask Norm and Adherence Data
International mask norm and adherence data was obtained from the COVID-19 Beliefs, Behaviors and Norms Survey* [32] which collected more half a million online Facebook survey responses in 51 languages, from 68 countries. The survey was deployed in 8 waves starting on July 7, 2020 and is still on-going. Each survey response comes with a weight that allows for reprensentativeness of estimates at the country and wave-level. Weights are calculated to account for a variety of biases including demographics (age bracket and gender) of the respondents compared to the census data in each country (through post-stratification), and response or non-response behaviors. The reported design effects due to both the non-response and post-stratification weighting was below four. In our analysis, we only use survey responses where the full survey was completed (including the demographic part) and where the mask-related questions we use in our analysis were completed. Overall, we have 479,917 data points covering 8 waves of analysis.
A.4 Controls
We control for a number of variables in our analyses: we use restaurant and recreation, grocery and pharmacy, parks, transit stations, workplace and residential percentage change from baseline (using the median day-of-the-week value from the first 5 weeks of January and February) mobility data from Google’s COVID-19 Community Mobility Reports [51]; new tests per 100K population from the COVID Tracking Project [30], and temperature and precipitation data from NOAA’s Weather and Climate Toolkit [52]. For international mask norm and adherance, we obtain COVID-19 country-level confirmed cases and confirmed deaths from the Our World in Data initiative [31].
Because we also include the squared values of each mobility indicator as a proxy for social contacts ( [65, 66]), we first re-scale the Google mobility index [51] as such: re-scaled mobility = (mobility + 100)/100. This is done to prevent the issue of Google’s the square of negative mobility indicator values to be the same as the positive values.
B State-level Mask Mandates Dates
C Event Study Results
Robustness check of main event study results (Section 1.1 fig. 1) where we only consider mask mandates that require business employees to wear masks. Because the trajectory and magnitude of the treatment effect is consistent with our previous main result under this new specification, this demonstrates that our main result is not sensitive to our definition of mask mandate start dates. Full regression results are shown in table 6.
Robustness check of main event study results (Section 1.1 fig. 1) where we only consider mask mandates that require all members of the public to wear a mask outside. Because the trajectory and magnitude of the treatment effect is consistent with our previous main result under this new specification, this demonstrates that our main result is not sensitive to our definition of mask mandate start dates. Full regression results are shown in table 5.
Robustness check of main event study results (Section 1.1 fig. 1) where we compare the effect of mask mandates on deaths and hospitalization with and without controlling for cases. Because the trajectory and magnitude of the treatment effect is consistent with our previous main result under either specification (with and without controlling for cases), this demonstrates that our main result is not sensitive to whether we account for the cases underlying the more severe outcomes. Full regression results are shown in table 7.
Robustness check of main event study results (Section 1.1 fig. 1) where we control for past outcome values (with a delay of 14 days) and growth rates as a way to minimize confounding due to peoples private behavioral changes to COVID-19 [70]. As can be see in the figure, the overall trajectory and magnitude of the treatment effect is consistent with our previous main result. Full regression results are shown in table 8.
Robustness check of main event study results (Section 1.1 fig. 1) where we compare the effect of mask mandates on cases, deaths and hospitalization for earlier wave states only (i.e. states who had early spikes of COVID-19 peaks in the pandemic, namely, NY, NJ, MA, CT, MI, DC, RI, IL, WA, PA, GA, VT, MD, FL and LA) vs later wave states. Masks mandates took more time to have an effect on earlier states whereas they had a faster effect on later states. However, because the trajectory and magnitude of the treatment effect overall is consistent with our previous main result, this demonstrates that our main result is applicable to both earlier and later waves. Full regression results are shown in tables 9 and 10.
D Robustness Check: Test Rate
Per state (each state a different color) and average (over all states, thicker black line) testing rate increased during our period of investigation.
Event study of testing rate over all states controlling for a number of factors showing that mask mandates are not associated with changes in testing rate during our period of investigation. This suggests that testing rate decreases are not behind the decrease in COVID-19 outcomes we observe after the introduction of mask mandates. Full regression results are shown in table 11.
E Mask Adherence following mask enforcement
Event study of mask adherence when not controlling for cases, deaths and testing rate. As expected, the treatment effect becomes noisier but reamins consistent to when controlling for cases, deaths and testing rate. Full regression results are shown in table 12.
F Daily Mask Adherence and COVID-19 Outcomes
G International Community Mask Adherence and Attitudes
Number of responses per wave of survey.
Average (weighted) community mask adherence response per wave.
Average (weighted) community mask attitude response per wave.
Footnotes
↵* Data and code are available on Github.
↵§ Hospitalization admissions proportions due to COVID-19 are only available until September 27, 2020
↵‡ See the Facebook Data Use Agreement for more details. Our use of this survey data (not collected by us) was covered under MIT’s IRB protocols E-2578 and E-2859. Respondents were 18 years or older, and consented to their responses being used by researchers.
↵‡ Admissions are coded as COVID-19 related if the admission code U071, U072, B9729, or if primary ICD-10 code is R05, R060, R509, Z9911, R0902, R0603, R0609, R062, R069, R0602, R05, R0600, J9691, J9692, J9621, J9690, J9601, J9600, J189, J22, J1289, J129, J1281, B9721, B9732, B342, B349, A419, R531 or R6889 and there is a secondary ICD-10 code of Z20828, or if the primary ICD-10 code is Z20828.
↵* See https://covidsurvey.mit.edu for more details.
References
- [1].↵
- [2].↵
- [3].
- [4].
- [5].
- [6].↵
- [7].↵
- [8].↵
- [9].
- [10].↵
- [11].↵
- [12].↵
- [13].
- [14].
- [15].
- [16].↵
- [17].↵
- [18].
- [19].↵
- [20].↵
- [21].
- [22].
- [23].
- [24].
- [25].↵
- [26].↵
- [27].
- [28].↵
- [29].↵
- [30].↵
- [31].↵
- [32].↵
- [33].↵
- [34].↵
- [35].
- [36].
- [37].↵
- [38].↵
- [39].↵
- [40].↵
- [41].↵
- [42].
- [43].
- [44].↵
- [45].↵
- [46].↵
- [47].↵
- [48].↵
- [49].↵
- [50].↵
- [51].↵
- [52].↵
- [53].↵
- [54].
- [55].
- [56].
- [57].
- [58].
- [59].
- [60].↵
- [61].↵
- [62].
- [63].
- [64].↵
- [65].↵
- [66].↵
- [67].↵
- [68].
- [69].↵
- [70].↵
- [71].↵
- [72].↵
- [73].↵
- [74].↵
- [75].
- [76].
- [77].
- [78].↵
- [79].↵
- [80].↵
- [81].↵
- [82].↵
- [83].↵
- [84].↵
