The estimated impact of non-pharmaceutical interventions on documented cases of COVID-19: A cross-country analysis

2.1. Data on documented COVID-19 cases

We perform a cross-country analysis for n = 20 countries. The sample comprises the United States, Canada, Australia, the EU-15 countries (Austria, Belgium, Denmark, Finland, France, Germany, Greece, Ireland, Italy, Luxembourg, the Netherlands, Portugal, Spain, Sweden, and the United Kingdom), Norway, and Switzerland. Our selection of countries defines a sample that follows a similar and comparable overall strategy in controlling the COVID-19 outbreak and shares a common cultural background. In particular, we exclude Asian countries despite the availability of excellent data, as these countries have often responded quite differently based on their experience with previous pandemics such as the 2013 SARS-CoV-1 outbreak. Nevertheless, the sample entails considerable variation across countries, as some countries have been affected severely, while others responded early as part of their mitigation strategy. Overall, the sample covers a population of ∼0.8 billion people.

SARS-CoV-2 infection figures for each country have been obtained from the Johns Hopkins Coronavirus Resource Center, which was developed for real-time tracking of reported cases of the coronavirus disease 2019 (COVID-19) and directly aggregates cases recorded by local authorities in order to overcome time delays from alternative reporting bodies ²¹. Hence, these numbers are supposed to account for all COVID-19 cases identified on a specific day. Case numbers are collected through April 15, 2020. In total, our sample comprises ∼1.6 million cases.

2.2. Data on non-pharmaceutical interventions

NPIs are systematically obtained from government resources and news outlets before being classified into seven categories (see Tbl. 1): (1) school closures, (2) border closures, (3) public event bans, (4) gathering bans, (5) venue closures (e.g., shops, bars, restaurants, and venues for other recreational activities), (6) lockdowns prohibiting public movements without valid reason, and (7) work bans on non-essential business activities. Note that “border closures” represents a measure that is fairly severe in restricting international travel. As lockdowns put implicitly other NPIs into effect, this variable measures the additional effect (i.e., the additional effect over event bans, gathering bans, and venue closures). The timing is encoded such that it refers to the first day an NPI goes into effect. We only consider NPIs that have been implemented throughout a country or in at least two thirds of its regions. See Appendix G for sources and details on data collection for non-pharmaceutical interventions.

Overall, the number and timing of implemented NPIs vary across countries (Fig. 1). Within the period of our analysis, school closures were implemented by 18 out of the n = 20, lockdowns by 11, and work bans by 6. Rising numbers of documented cases prompt more severe NPIs.

• Download figure
• Open in new tab

Fig. 1.

Heatmap showing the cumulative number of documented COVID-19 cases for each country until a non-pharmaceutical intervention (NPI) is implemented. By reporting the cumulative number of cases, the heatmap highlights the ordering of NPIs within countries. If an NPI is not implemented by a country, it is colored gray. A heatmap showing the cumulative number of cases relative to the country’s population is shown in Appendix Fig. 1.

We aim at investigating the effectiveness of different NPIs. The outcome of interest is the number of new documented COVID-19 cases per day. This choice reflects that the true number of new cases is unknown and, therefore, the documented numbers serve as the basis on which NPIs are monitored and evaluated by decision-makers in health-policy. In particular, case numbers must be controlled to an extent, so that surge capacities in critical care are not exceeded at the peak of the epidemic ²². Fig. 2 shows the number of new cases over time with the implemented NPIs indicated. Further descriptive statistics of our data are shown in Appendix A.

• Download figure
• Open in new tab

Fig. 2.

Temporal development of the observed (documented) number of new cases by country (n = 20). The points at which non-pharmaceutical interventions (NPIs) have been implemented are annotated. The solid line refers to the study period, starting from a cumulative case count of 100. To show the timing of NPIs potentially implemented before that period, the dashed line refers to the early period, starting from a cumulative case count of 10.

2.3. Overview of statistical analysis

We assume the daily number of new cases of COVID-19 to be affected by the NPIs that are already implemented. We assume that NPIs can only influence the number of new cases with a delay of t₀ = 7 days. This choice of the delay considers the following factors. First, prior research on SARS-CoV-2 transmissions indicates an incubation period of around 5 days ³. Second, as testing is often performed only in response to symptoms and positive results may be reported with some delay, a further overall delay about two days is not unlikely. Third, behavioral responses can vary with people adjusting their behavior with a certain delay or even before the NPI is issued. Given the overall uncertainty in the delay with which NPIs can take effect, the sensitivity of our results to the chosen time delay of t₀ = 7 days is studied as part of a sensitivity analysis.

The introduced model relates the daily new cases to the actual number of cumulative cases, such that the estimated effect for each NPI quantifies the relative reduction in new cases. We report posterior means and 80% and 95% credible intervals (CrI). The model considers day-of-the-week effects as the frequency of testing may depend on the day of the week and reporting may be delayed to a higher degree during weekends. The model allows the background rate of new cases to vary across countries as this rate depends on the (unknown) age composition of the true cases, the population density, and other country-specific factors.

We exclude the very early phase up to 100 documented cases as we have to expect that each country had to establish its documentation practice in the early phase. However, we vary the number of cases at the start as part of the sensitivity analysis.

The model assumes the effect of each NPI to be equal in each country. We further assume implicitly that any change in the rate of new cases is due to one of the NPIs considered. The latter assumption is later discussed as a limitation of our study.

2.4. Model specification

Let C_r,d denote the number of documented cases in country r at day d. Then N_r,d = C_r,d+1 − C_r,d refers to the number of new cases in country r from day d to the next day d + 1. We model the number of new cases to follow a negative binomial distribution (NB), i.e., with mean μ_r,d, variance , and an overdispersion parameter φ. The mean is related to the log of the expected rate of new cases, , via

The log-rate is related to the NPIs, the day of the week, and the country via

Here α_r is a country-specific effect reflecting the growth in the number of cases for country r in the absence of any NPI, γ_w(d) are dummies for each day of the week, is a time-delayed response function, and is the number of days since measure (NPI) m took effect in country r.

We model the time-delayed response function via a step function so that NPIs can only influence new cases with a delay of t₀. We set t₀ = 7 days, as detailed above. Both the choice of the influence function and the time delay are varied as part of the sensitivity analysis.

The model parameters are estimated by a fully Bayesian approach with weakly informative priors, except for restricting the effect parameters θ_m a priori to non-positive values (i.e., NPIs can only reduce the number of new cases). This assumption is relaxed as part of a sensitivity analysis. Estimation details are given in Appendix B. For each NPI m, we report the posterior distribution of 1 − exp(θ_m), that is, the relative reduction in the number of new cases.

3.1. Effects of non-pharmaceutical interventions

Using the data from the early stages of the outbreak until April 15, 2020, we compare the estimated reduction in the number of new cases by different NPIs (Fig. 3, detailed estimation results are to be found in Appendix C). Our model estimates that the closure of venues is associated with the highest reduction in the number of new cases (36 %; 95% CrI 20–48 %). The mean reduction is slightly lower for gathering bans (34 %; 95% CrI 21–45 %), border closures (31 %; 95% CrI 19–42 %), and work bans on non-essential business activities (31 %; 95% CrI 16–44 %). Event bans lead to a slightly less pronounced reduction (23 %; 95% CrI 8–35 %), whereas school closures (8 %; 95% CrI 0–23 %) and lockdowns (5 %; 95% CrI 0–14 %) appear to be the least effective among the NPIs considered in this analysis.

• Download figure
• Open in new tab

Fig. 3.

Estimated reduction (posterior mean as dots with 80% and 95% credible interval as thick and thin lines, respectively) in the number of new cases (in %) for each non-pharmaceutical intervention (NPI). The variable for lockdowns is encoded in our analysis such that it quantifies the additional effect over other NPIs limiting personal movement (i.e., event bans, gathering bans, venue closures).

For a precise ranking of the effectiveness of NPIs, one has to consider the range of credible effects. Thereby, venue closures, gathering bans, border closures, and work bans show similar effect sizes with a magnitude of the relative reduction above 15 %. With respect to event bans we can exclude an effect close to 0 % but the range of credible effects includes a moderate relative reduction by 10 % up to much larger reductions. The posterior distributions of school closures and lockdowns focus on small effects with a the magnitude of 0 % to 10 %, but for school closures a 20 % reduction is still credible. Also, recall that our encoding of lockdowns only attributes the additional effect over other NPIs, which is reported as part of our estimations. When considering that lockdowns put implicitly other NPIs into effect, the combined effect size is considerably larger.

Fig. 4 directly depicts the posterior distribution of the ranking of the effects. For each of the four NPIs with the highest effect estimates, we can be at least 85%sure that they are among the four most effective NPIs. For school closures and lockdowns, we can be at least 85%sure that they are among the two least effective NPIs

• Download figure
• Open in new tab

Fig. 4.

Posterior distribution of ranking non-pharmaceutical interventions (NPIs) by their effect. Shown is the proportion of posterior samples (4,000 in total) for each rank of the NPI parameter (θ_m). Highest reductions obtained rank 1 and lowest reductions rank 7.

3.2. Sensitivity analysis

Results from the sensitivity analysis are reported in Appendix D. They indicate a small influence on the estimated effect of NPIs. A model comparison (Appendix E) suggests that an influence function with a longer delay or a first-order spline is beneficial, but the estimated reduction of each NPI remains qualitatively similar.

3.3. Illustrating the effectiveness of non-pharmaceutical interventions

Fig. 5 illustrates the estimated impact of different interventions on new COVID-19 cases in single countries. For this purpose, we use the posterior mean estimate of the corresponding NPI to predict the number of new cases as if the NPI was not implemented. In this hypothetical setting, the number of new cases is subject to considerable growth. In contrast, the observed (documented) count remains well below the hypothetical prediction. Taking the United Kingdom as an example, the case count predicted under no interventions quickly exceeds 10,000 new cases per day, while the observed number ranges below 10,000 per day. Fig. 5 also allows to compare the daily estimated number of new cases from our full model (based on the NPIs that the country actually implemented) with the observed (documented) number of new cases. Here we observe an acceptable degree of agreement for the UK and US. Figures for all other countries are listed in Appendix F. Note that these figures solely serve as a visualization of the mean estimated effect of NPIs on new cases (and credible intervals are thus omitted in these figures).

• Download figure
• Open in new tab

Fig. 5.

Hypothetical number of daily new cases if non-pharmaceutical interventions (NPIs) would not have been implemented. The stacked bars illustrate how the mean estimated number of new cases would have changed if NPIs would not have been enforced. The solid line refers to the observed (documented) and the dashed line to the estimated number of new cases. The NPIs are shown for the time period in which they have been implemented while considering that their effect is subject to a time delay of 7 days.

4.1. Relevance for health-policy

NPIs have a substantial impact on society and entail immense economic costs. Hence, there is an urgent need to better understand the relative effectiveness of NPIs. A remedy is provided by our analysis. It attributes to what extent the rate of new cases is reduced due to the implementation of specific NPIs (or potentially a combination thereof). This is an attempt to obtain early insights, while further effort is necessary in order to arrive at definite answers.

Our analysis is designed to be of direct relevance for health-policy. To this end, a focus is placed on documented COVID-19 cases. Documented case figures are readily available and serve as the basis for decision-making in health-policy, specifically in order to control the epidemic so that the surge capacity in critical care is not exceeded. Our estimates aid health-policy design and evaluation by providing empirical evidence regarding how NPIs influence the number of new cases. Other metrics (e.g., the effective reproduction rate or the serial interval) serve a different purpose, as they are primarily used to study epidemiological characteristics of how infectious diseases spread. Nevertheless, NPIs aim at reducing the transmission rates and, hence, corresponding models can be an alternative means of understanding the impact of NPIs. Such models draw upon the true number of cases, yet which is unknown. Hence, following such approaches requires one to link these unknown numbers to observable measures by additional assumptions on the model specification, as done, for example, by Flaxman et al. ²⁰. By directly modeling the impact on the observed number of new cases, we avoid this step. Still, domain knowledge from the realm of infection epidemiology informs our modeling in the form of prior knowledge concerning the incubation time of COVID-19.

4.2. Summary of results

We performed a cross-country analysis based on n = 20 countries in order to assess the relative effectiveness of NPIs. For four of our seven NPIs, we could assign an estimated relative reduction in new cases of 30 % or above and a range of credible effects above 15 %, indicating a highly relevant role. Both venue closures and gathering bans were among these four NPIs, and each was implemented by 19 out of n = 20 countries in our sample. This is an interesting addition to prior literature, in which these NPIs have received rather little attention and thus deserve further investigation. Work bans also belongs to the group of effective measures, but this NPI was implemented by only six countries, which implies greater uncertainty about the estimate. The NPI border closures is the last member of this group. Simulations confirmed that a rapid dissemination of SARS-CoV-2 was associated with large numbers of undocumented infections ⁶, especially if not prevented by travel restrictions ⁸. However, this measure should be interpreted with caution. It primarily targets international transmissions, which might be more pronounced in early stages of an outbreak as studied in this analysis. In the later stages of an epidemic, the effect might be substantially smaller, with this measure primarily steering international travelers or cross-border commuters towards increased social distancing.

School closures restrict access to education with unwanted implications for members of society who are already underprivileged. In our analysis, this NPI belonged to those with only a rather moderate estimated effect, amounting to a 10 % reduction. This finding is in line with prior literature in which the transmissibility of SARS-CoV-2 among children is regarded as comparatively small ^11,23. The preceding argument also provides a possible explanation for why our model attributes only a small reduction of cases to this NPI. Also lockdowns have only a very moderate and possibly negligible effect. Yet, lockdowns put implicitly other NPIs (bans on public events, bans on mass gatherings, and venue closures) into effect. When considering the simultaneous presence of these other NPIs, the combined effect is substantial. Event bans were located between the groups of NPIs with large and small relative reductions, with an estimated relative reduction of about 25 %, but a wide range of credible effects.

Due to the concurring introduction of NPIs in many countries, it can be difficult to distinguish between the effects of single NPIs. This is reflected by large credible intervals and a negative association between effects (Appendix C.2), suggesting that the effect of one NPI may be attributed partially to another. However, with respect to the ranking of NPIs, we were able to demonstrate that we can be fairly confident that venue closures, gathering bans, border closures, and work bans are among the four most effective NPIs and school closures and lockdowns among the two least effective when applied as a combination of NPIs.

4.3. Limitations

Our analysis is limited by the type of data utilized and by the need to make modeling assumptions. Using the number of documented cases implies that documentation practices may have an impact on the results. In particular, definitions and documentation practices may differ between countries and over time. However, as we are investigating the rate of new cases, many deviations from an optimal documentation practice will affect both the number of new cases and the number of existing cases in a similar manner and may thus cancel out. Short-term fluctuations are considered by including day-of-the-week effects and allowing for overdispersion. Moreover, we must assume that each country had to establish its documentation practice in the early phase of the outbreak, which is taken into account by excluding the very early phase up to 100 documented cases. However, there may still remain an undue influence of country-specific changes in documentation practices over time.

One fundamental issue with our modeling approach is the implicit assumption that any deviation from a constant rate of new cases is explained by the NPIs. There may be additional measures or an increasing general awareness which encourages social distancing and hence leads to less infections. If this is the case, such effects will erroneously be assigned to the NPIs and possibly overstate their overall effectiveness. A further limitation arises from the lack of reliable information on the number of recovered cases, as it would facilitate in relating the number of new cases to the number of non-recovered cases. However, within our study period, the number of recovered patients is comparatively low relative to the overall population and, hence, it should be justified to neglect this issue when analyzing the early stages of the outbreak. With respect to the true infection rate, theoretical simulations ²² suggest a stationary rate in the absence of NPIs for this early phase and, hence, independence from natural attrition phenomena.

Our modeling assumptions do not account for any interaction between countries and the effect of NPIs. The simplifying assumption of a common effect reflects our aim to estimate an “average” effect and the current lack of power to investigate cross-country variation.

4.4. Outlook

Our findings draw upon data from the early stages of the COVID-19 outbreak. For instance, border closures might prevent SARS-CoV-2 infections from being imported ^6,8, while their effectiveness is likely to change when domestic transmissions start to drive epidemic growth. Therefore, it is not clear to what extent NPIs are effective in the later stages of the epidemic. Several countries, which are still in the early stages of the epidemic, are faced with the prospect of implementing health-policies including transmission control measures. Other countries, especially in the developed part of the world, are on a path towards progressively relaxing existing measures. Future research could explore how transmission rates develop when NPIs are sustained for longer periods of time or when NPIs are lifted. Here the effect could point in different directions. On the one hand, people might have integrated social distancing into their daily routines and thus stick to them even after policy measures are lifted. On the other hand, people have postponed several activities with social interactions (e.g., visits to general physicians or shopping activities), which would take place after lifting. Our sample is based on Western countries as their behavioral responses to NPIs should be fairly homogeneous, yet further research is needed in order to analyze to what extent the findings can be generalized to other countries.

Our objective is to provide timely estimates that could inform health-policies. With more data becoming available, we may be able to refine our estimates. However, as many countries have implemented most measures already several weeks ago, increasing the observation period is of little informativeness. Additional information may arise from taking the federal structure of some countries into account (e.g., the United States and Germany), from more detailed case data (e.g., recovery status or age), or from rescinding NPIs. It would be of great interest to use this additional information to also study between- and within-country variation of NPI impact (i.e., modeling country- or region-specific effects of the NPIs) and to explain this variation through the lens of country characteristics. This would allow us to develop country- or even regional-specific recommendations.