Introduction

Since the COVID-19 outbreak began in early 2020, the governments of countries, states, and cities around the world have implemented various combinations and timings of strategies for balancing the health and safety of their citizens with the economic costs of those strategies and the disease itself. In doing so, these governments have effectively done hundreds of experiments, which are being studied to provide insights in determining the strategies that have been effective in reducing the number of COVID-19-related deaths and the strategies that have not. A vast number of such studies have been and are taking place, attempting to study either the effectiveness of individual policies or the overall aggregate effectiveness of the stringency of policies and not the portfolio of policies each considered individually. The goal of this study is to examine the effectiveness of a wide range of strategies employed by the OECD countries from mid-March through the end of October 2020 to determine which strategies are most effective in reducing COVID-19 related deaths so policymakers might make informed policy choices.

Key elements in modeling epidemics and guiding policymakers include infection rates, mortality rates associated with infections, the ability and effectiveness of the policies, the medical system, and how society adapts to the changing dynamics of a pandemic, as well as other structural factors (Verity et al. 2020). When vaccines are absent—such as in the case of COVID-19 up to December 2020—governments try to limit social contact in order to flatten the curves of infection and mortality. This strategy has been adopted by most countries in the world. Additionally, institutional and demographic characteristics (average population age, percent of population with diabetes, gross domestic product per capita, etc.) can influence mortality dynamics both directly through the size of vulnerable populations, and indirectly through citizens’ perceptions and behavioral responses to stringent policies (Van Bavel et al. 2020).

Age seems to consistently be the largest factor in mortality—people 65 and older account for most of COVID-19 related deaths in the US, though the Our World in Data (OWD-Oxford) site suggestsFootnote 1 that this might in part be due to the higher incidence of pre-existing health conditions that come with higher risk. The CDC has concluded that pre-existing conditions, such as heart disease, obesity, diabetes, kidney disease, and immune deficiency put people at higher risk of mortality and other factors, such as asthma, liver disease, hypertension, and pregnancy might put people at even greater risk; the OWD-Oxford site reports similar conclusions.1 The mortality rate for those with no pre-existing health conditions (0.9%)Footnote 2 is less than a tenth the rate for those with such conditions.

Our paper focuses on the mortality rate instead of the case rate for two reasons. First, and most importantly, it is the mortality rate of COVID-19, as compared to other diseases, that distinguishes it and is the cause for concern. Although the number of US cases of influenza in recent yearsFootnote 3 is about the same as the number of confirmed US cases of COVID-19 over the past year,Footnote 4 the number of COVID-19 related deaths in the US over the past year is more than ten times the typical number of US deaths from influenza in recent years.Footnote 5 A second reason is that the data seems to be more reliable. While death is definitive and the cause of death more carefully determined, there seems to be a much broader range of practice in defining COVID-19 cases (Jinjarak et al. 2020).

Governments have used a wide range of strategies executed with various timings to manage the number of COVID-19 cases and deaths. Government interventions may affect people’s mental health and can cause substantial economic and social costs. Therefore, it is crucial to investigate which government policies (or NPIs) are the most effective to combat COVID-19 or any other future respiratory epidemic. The challenging part in studying NPIs is that many countries rolled out several policies simultaneously; therefore, it is much more difficult to decouple the impact of each individual intervention.

In terms of representative studies on NPIs, COVID-19 transmission and social mobility, Brauner et al. (2021), using data between January and May 2020 in 41 countries (34 European and 7 non-European), find that closing schools was highly effective, banning gatherings and high-risk businesses was effective but closing most other businesses had limited further benefit. Flaxman et al. (2020) study the effect of government policies in 11 European countries using data from February to early May 2020 and find that these interventions have been effective in reducing the transmission rate of COVID-19. Haug et al. (2020) rank government interventions for 226 countries worldwide, for the purpose of mitigating the spread of COVID-19. Aravindakshan et al. (2020) measure the impact of NPIs on social mobility and the resulting disease mitigation by exploiting the spatio-temporal variations in policy measures across Germany’s states.

Hsiang et al. (2020), with data on local, regional, and national NPIs in six countries, find that anti-contagion policies are effective in reducing the growth of COVID-19 infections. Liu et al. (2021) find mixed evidence for the association between NPIs and reduced infection rates, with only school closure and internal movement restrictions showing unequivocal evidence of being associated with a decrease in the effective reproduction number of COVID-19. On the other hand, Courtemanche et al. (2020) find that shelter-in-place orders and closures of restaurant and entertainment-related businesses were effective in reducing the growth of COVID-19 cases, while bans on large social gatherings and school closures had no effect. Berry et al. (2021) explore the effects of shelter-in-place policies during an early period of the COVID-19 pandemic. Their study focuses on how these policies affect not only new cases and deaths but also mobility behavior and unemployment rates. According to their findings, the effects on infections, deaths, and unemployment appear moderate and insignificant. Amuedo-Dorantes et al. (2021) examine how the speed of adoption of NPIs affects mortality in the US and conclude that advancing the date of NPI adoption by one day lowers the COVID-19 mortality rate by 2.4%.

Regarding the effectiveness of mask mandates, Bundgaard et al. (2021) discover through a randomized controlled trial in Denmark, that the mask mandate as a supplement to other public health measures does not reduce the COVID-19 infection rate by more than 50% in a community with relatively moderate social distancing. On the other hand, Karaivanov et al. (2021) investigate the impact of mask mandates and other NPIs on COVID-19 case growth in Canada and find that in the first few weeks after implementation, mask mandates were associated with a 25% decrease in the weekly number of COVID-19 cases.

We contribute to the continuously growing literature on COVID-19, that deals with non-pharmaceutical interventions, by investigating the impact of seven government policies on new mortality growth rates across the 37 OECD countries. The application of policies and reporting of data varies widely across countries. We focus on policies implemented between March 15th and October 31st, studying the impact of those policies up to 70 days after policy implementation,Footnote 6 controlling for policy and structural factors subject to data availability and quality.

For our analysis, we examine how the growth in the new deaths attributed to COVID-19 (7-day smoothed) per million for OECD member countries, reported by OWD-Oxford, is influenced by changes in seven policy variables (ranging from school and business closures to restricting internal travel), reported by CGRT-Oxford, controlling for population, economic, and health data (such as age, GDP per capita, and diabetes prevalence in the population). The lag time between a change in policy to possible change in the rate of deaths is accounted for by the policy variables 7 to 70 days. This range of lags was decided based on the CDC’s estimates of the time from exposure to symptom onset to death. Related research (Jinjarak et al. 2020) finds also effects from lags of 2 to 4 weeks.

In addition to the ordinal values associated with each of the policies to indicate stringency of implementation (discussed in more detail in the “Data” section), each also has an associated binary “flag” variable to indicate whether the policy implementation was geographically targeted or general. We use the different levels of stringency of policy implementation in our study but do not use the flag variable data. A reasonable argument is that selective geographic implementation implied implementation only where it was needed, and therefore it was tantamount to general application in effect. Given the restriction of the dataset, it would not be possible to measure mortality rates by region, and therefore, we do not include the flag variables in our model.

Furthermore, compliance, which may vary between and even within countries over space and time, may be a critical factor in determining the efficacy of many of these policies. A paper by Singh et al. (2021) analyzes the effect of introducing and lifting NPIs in the US on COVID-19 cases and shows that part of the variation in health outcomes is driven by variation in compliance (that is, reduction in mobility due to NPIs). In their paper, compliance is measured as mobility, whereas mobility is proxied as the percentage of mobile devices staying home out of total mobile devices on the county level.

In addition to any direct effects the implementation of NPIs might have on COVID-19 cases and deaths, there is evidence that their implementation has indirect effects, notably through information channels that may alter perceived risk of contracting the disease. For instance, Cronin and Evans (2020) use structural break and difference in difference models and show that mobility, foot traffic, and social distancing were all reduced after the declaration of a state of emergency and before restrictions were in place. This implies that the decline can be explained by the private decisions of people to reduce the risk of contraction or transmission of the virus, because of the information received at the national level. Another study by Dave et al. (2020) reports that termination of Wisconsin’s social distancing order by the Wisconsin Supreme Court had little impact on social distancing practices or COVID-19 case growth, reinforcing the role of information channel in motivating people’s behavior.

We focus on the OECD countries because there seems to be more consistency in application and compliance within and between those countries. Further study of the issue of compliance, as well as the impact of the information value of policy implementation in changing the risk perceptions, and therefore behavior of individuals, is beyond the scope of this paper.

Our paper is structured as follows: “Data” section presents the data utilized; “Model Specification” section describes the model and methodology; “EstimationandResults” section provides discussion of results and analysis, and “Conclusion” section concludes.

Data

We analyze the effects of government policy responses in the 37 OECD countries.Footnote 7 We accessed the data on January 15th, 2021. For each country, we consider the impact of policy variable changes beginning on the 15th of March and ending on the 31st of October. Our analysis starts on March 15th because from this date and after all the sample countries of our study were reporting complete data and there was more consistency in policy implementation. Our model requires 70 days of complete mortality data after the start of a government policy, implying that mortality rates extend through January 10th for policies in effect up to October 31st. Our data is drawn from two sources, OWD-Oxford and the Oxford Coronavirus Government Response Tracker (CGRT-Oxford). The database at OWD-Oxford reports the dependent variable our paper uses as the target to be reduced by policy, namely the growth in the new deaths attributed to COVID-19 (7-day smoothed) per million for OECD countries.

The database at CGRT-Oxford reports the policy variables upon which our treatment variables are based. Specifically, CGRT-Oxford reports ordinal statistics for several policy variables categorized as Containment and Control, Economic, and Health System policies. For each policy, one or more levels of rigor/stringiness of application are reported, with an ordinal number statistic (tables reporting the relationship between the ordinal numbers and rigor of application of policies can be found in an associated Codebook).Footnote 8 The study focuses on the first seven of the Containment and Control policies in the CGRT-Oxford database, seeking to shed light on the domestic containment policies implemented by governments. The policies studied are listed below and each was represented as a value of 0 (zero) if that policy was not implemented at all or a value of 1 or higher if the policy was implemented, with increasing ordinal values representing increasing levels of rigor with which the policy was applied (for example, recommendation vs. requirement).

  1. (a)

    C1 School Closings

  2. (b)

    C2 Workplace Closing

  3. (c)

    C3 Cancel Public Events

  4. (d)

    C4 Restrictions on Gatherings

  5. (e)

    C5 Close Public Transport

  6. (f)

    C6 Stay at Home

  7. (g)

    C7 Restrictions on Internal movement within country

While our analysis focuses on these ordinal variables, we also discuss scatter plots to determine differences in impact on the growth in the new deaths (7-day smoothed) per million of these various levels of rigor. Results of this analysis are discussed in the “EstimationandResults” section of the paper.

Model Specification

We focus on the pandemic period between March 15th, 2020 and October 31st, 2020. The reason our sample starts from March 15th is because most of the countries considered had imposed containment and policy closure interventions around this date and all were reporting data (see Oxford’s COVID-19 Government Response Tracker link). We examine the growth of COVID-19 new mortality rate, and how it is affected by government policy responses. To filter out noise in the daily mortality data, we use for our estimations a variable that represents the new deaths (7-day smoothed) per million. For simplicity, this 7-day smoothed new mortality rate will be henceforth referred to as mortality rate.

Our dependent variable, the growth of mortality rate, \({Y}_{i,t+h}\) in country \(i\) on date \(t\), \(h=\{7, 14, 21, 28, 35, 42, 49, 56, 63, 70\}\) is defined as

$${Y}_{i,t+h}=log \left({MortalityRate}_{i,t+h}\right) -log({MortalityRate}_{i,t}),$$
(1)

where \({MortalityRate}_{i,t}\) represents new deaths attributed to COVID-19 (7-day smoothed) per 1,000,000 people in country \(i\) on date t.

Our model’s specification is as follows:

$${Y}_{i,t+h}={\alpha }_{i}\left(h\right)+{ \beta }_{t}\left(h\right)+\gamma \left(h\right){G}_{i,t}+{\varepsilon }_{i,t}(h),$$
(2)

where \({Y}_{i,t+h}\) is the growth in the mortality rate in country \(i\) on date \(t+h\).

\({G}_{i,t}\) represents the government policy responses constructed as described in the “Data” section of the paper. Fixed effects are denoted as \({\alpha }_{i}\) and \({\beta }_{t}\), representing the country and time fixed effects, respectively.Footnote 9 We examine the response of growth of mortality rate with respect to policy responses \({G}_{i,t}\) at horizons over one week. The collection of estimates \(\widehat{\gamma }\left(h\right)\) for lag \(h\) trace out the dynamic impact of government responses on mortality growth at the daily frequency.Footnote 10 For high lag levels (49 or higher), there might be policy endogeneity. Although government responses are probably endogenous to prior mortality rates, and this might bias the related estimates, we do not restrict our analysis to lower lag levels. Higher lags can still provide insight into the effectiveness of the examined NPIs. In order to overcome the endogeneity challenge, additional and more detailed data could be collected, and future research will be able to shed more light on addressing this issue.

Estimation and Results

Table 1 shows how the countries were affected by COVID-19 over the period studied by reporting the average total deaths and average new deaths per million. Table 2 provides an assessment of the relative effect sizes of the outcome variables, given the different lags used in the study. Tables 3 and 4 report the baseline results of our regression analysis. Of the seven Containment and Control policies studied, only school closings and public transportation closings have a persistently significant impact, both with negative coefficients, as it would be expected. Stricter School Closing policies can decrease the growth in the mortality rate by at least 17.43% on average after 5 weeks while stricter public transportation closing policies reduce the growth in the mortality rate by at least 26.68% on average after 5 weeks. The coefficients of both of these variables grow larger and more significant the longer the lag.Footnote 11

Table 1 Descriptive statistics on mortality per country
Full size table
Table 2: Descriptive statistics on model specifications’ outcomes
Full size table
Table 3 Baseline panel analysis on new mortality growth rate
Full size table
Table 4 Baseline panel analysis on new mortality growth rate
Full size table

By examining the plot of variable C1 (School Closing) in Fig. 1, we observe that there are several cases in countries without any measures on school closing where the growth of mortality rate increases. The distribution of mortality rate growth for each level of stringency over the country panel is presented as a box-plot. These plots help us to get a better insight not only about the spread but for the median and the center of the distribution as well. For example, it is clear and remarkable that, especially for the panels of 49, 63, and 70 lags of Fig. 1, the medians are almost of the same level. For countries with very strict school closing measures, there is a skewed spread that increases with higher lags, showing an increased number of negative growth rates for those countries.Footnote 12 One possible explanation is that closing schools precipitated other behaviors that might have reduced transmission. For instance, school closings meant that parents had to stay home, which may have contributed to the reduction. School closings probably also precipitated reduction of other activities kids’ activities (sporting events etc.), where transmission may have been more likely to occur.

Fig. 1
figure 1

Growth of mortality rate (%) for different levels of measures on School/University closing. Source Codebook: 0—no measures, 1—recommend closing or all schools open with alterations resulting in significant differences compared to non-Covid-19 operations, 2—require closing (only some levels or categories, for example, just high school, or just public schools), 3—require closing all levels

Full size image

The same pattern is observed in Fig. 2 for the variable C5 (public transportation closing) at a more intensive level. Stringent levels on this variable indicate a higher level of strictness on other variables such as C2 (Workplace closing) and C6 (Stay at home), especially on big metropolitan areas that asymmetrically affect the entire country's mortality rates. Stay-at-home policies showed significant impact after 70 days, and only at the 90% level even then. As we observe in Fig. 5, many countries with required Stay-at-home restrictions exhibit an increased number of negative mortality rates after several weeks of adopting the policy. Similar limited effects of stay-at-home policies have been identified and discussed in the paper by Berry et al. (2021). Our study extends their work, using data beyond the first wave, as suggested by the authors.

Fig. 2
figure 2

Growth of mortality rate (%) for different level of measures on closing public transportation. Source Codebook: Close public transportation: 0—no measures, 1—recommend closing (or significantly reduce volume/route/means of transport available), 2—require closing (or prohibit most citizens from using it)

Full size image

On the other hand, it is remarkable that especially for the variable C7 (domestic travel restrictions), the effect appears insignificant in our model. Figure 4 indicates that the growth rate for countries with strict measures on internal movements is spreading symmetrically, an indicator of a possible insignificant effect regarding this policy for the period we study. It is likely that even without internal traveling measures, especially for the first wave of the pandemic, most people voluntarily cancel or delay unnecessary travel plans in response to increased public fear and anxiety for the COVID-19 outbreak. On the other hand, low infection rates combined with many asymptomatic patients can severely intensify the transmission within a city because people feel safer and do not alter their commuting behaviors. Consequently, restrictions on public transportation could be proved crucial at an early phase of the pandemic. A similar symmetric spread of mortality rates over the different lags is observed for the variable C4 about the restrictions on gatherings, as shown in Fig. 3.

Fig. 3
figure 3

Growth of mortality rate (%) for different level of measures on restrictions on gatherings. Source Codebook: Restrictions on gatherings: 0—no restrictions, 1—restrictions on very large gatherings (the limit is above 1000 people), 2—restrictions on gatherings between 101-1000 people, 3—restrictions on gatherings between 11-100 people, 4—restrictions on gatherings of 10 people or less, Blank

Full size image

Figure 6 presents the 7-day moving average of new deaths per million between March 15th, 2020, and October 31st, 2020. While Figs. 1, 2, 3, 4 and 5 show the growth of mortality rate for all countries jointly and from different time points within our period when a specific measure was in effect, Fig. 6 illustrates mortality dynamics for every country separately. The variable days shows the days after March 15th; the dashed line indicates the start of measures on schooling closing while the dash-dotted line indicates the start of measures on public transportation closing. Although these two policies are in effect for most countries before March 15th, or a few days after that date, the policies are repealed over the studied period for several countries. Although Fig. 6 displays the mortality trends over the sample period and when two crucial government policies were initially implemented, Table 5 provides more details on how the governments of OECD countries responded differently to the epidemic.

Fig. 4
figure 4

Growth of mortality rate (%) for different level of measures on stay-at-home requirements. Source Codebook Stay at home requirements: 0—no measures 1—recommend not leaving house, 2—require not leaving house with exceptions for daily exercise, grocery shopping, and 'essential' trips, 3—require not leaving house with minimal exceptions (for example, allowed to leave once a week, or only one person can leave at a time, etc.)

Full size image
Fig. 5
figure 5

Growth of mortality rate (%) for different level of measures on restrictions on internal movement. Source Codebook Restrictions on internal movement: 0—no measures, 1—recommend not to travel between regions/cities, 2—internal movement restrictions in place

Full size image
Fig. 6
figure 6

Mortality trends. Notes: Mortality by country over time. Mortality is the 7-day moving average of new deaths per million. Days shows the days after March 15th; the dashed line indicates the start of measures on school closing. The dash-dotted line indicates the start measures on public transportation closing.

Full size image
Table 5 Date implementation of C1–C7 government policies
Full size table

Conclusion

Our paper focuses on estimating the impact of government containment and policy closure interventions against the COVID-19 pandemic, contributing to a growing and diverse body of evidence, simulation studies and clinical experience based on epidemics. Using a large dataset, based on the 37 OECD countries, spanning from March 15th, 2020 through January 10th, 2021, we examine the response of new mortality growth rate over 70 days after implementation with respect to seven government policies targeting domestic containment, that were implemented from March 15th, 2020 through October 31st, 2020.

Our results suggest that, of the variables considered, only school closings and closing of public transportation have a persistently significant impact on growth of mortality rate, with the impact of each being negative, as expected. Although stay-at-home policies show a significant and negative effect, they do so only at the 90% significance level and at 70 days of lag.

Ultimately, we hope that this study can help policymakers in the reduction of incidence and excess mortality of COVID-19 associated with potential further waves of infections, as well as informing regarding the areas of public life that can continue operating during a pandemic with limited restrictions.