The impact of pausing the Oxford-AstraZeneca COVID-19 vaccine on uptake in Europe: a difference-in-differences analysis

Data Sources

A longitudinal panel was constructed for 28 EEA countries with vaccine uptake data at three-week intervals from week 3 to 18 of the COVID-19 vaccination rollout in 2021. This did not include the United Kingdom (UK), Denmark or Norway. The UK is no longer in the EEA and had different plans and systems for vaccine rollout compared to the rest of the EEA. Denmark and Norway did not restart their use of the Oxford-AstraZeneca vaccine after the pause, unlike other countries. Media reports were used to identify countries that paused the Oxford-AstraZeneca vaccine and the timing of this(1–3). Data on overall COVID-19 vaccine uptake and only Oxford-AstraZeneca uptake, for first and both doses, were obtained directly from the European Centre for Disease Control and Prevention COVID-19 Vaccine Tracker and Open Data(12,13). Uptake was provided as a cumulative percentage of the adult population (18+) for all vaccines, and due to data limitations, calculated as a cumulative percentage of the entire population for the Oxford-AstraZeneca vaccine.

Data on numbers of vaccines supplied, for both the Oxford-AstraZeneca vaccine alone and for all vaccines, were also obtained from the same dataset and adjusted for the population size of each country, prior to analysis. Data on other potential confounders, including demographic, economic, and health system indicators, were from 2019 and collected from the World Bank Open Database(14). Two indicators (universal health coverage service index and health expenditure per capita) were from 2017 and 2018 respectively. Given the role of public confidence and trust in vaccine uptake, the state of democracy in included countries was ascertained through the Economist Intelligence Unit’s 2020 Democracy Index(15). Data on weekly COVID-19 cases and cumulative deaths at baseline (week 3 of 2021) were obtained from the open-access Our World in Data(16).

Exposures and Outcomes

Exposure to the intervention was defined as those EEA countries that paused the rollout of the Oxford-AstraZeneca vaccine. This action was taken over the week of 8^th March. Those that did not pause the vaccine at all were treated as the control group. A dummy variable was generated to indicate whether the vaccine rollout was paused in a given country at a given week of the vaccination programme. Our primary outcome was cumulative uptake for first and both doses for all vaccines, defined as the percentage of the adult (18+) population that had received 1) one dose and 2) both doses of a COVID-19 vaccine, from week 3 of 2021 to week 18. The secondary outcome was cumulative uptake for first and both doses of the Oxford-AstraZeneca vaccine from week 6 to 18. Due to data availability this was defined as the percentage of the whole population that had received 1) one dose and 2) both doses of the Oxford-AstraZeneca vaccine.

Statistical Analysis

All analyses were conducted using STATA 14(17). We first compared both groups of countries on a range of baseline characteristics, using the Wilxocon Rank Sum test to identify statistically significant differences in mean values between groups. P-values of less than 0.05 were considered statistically significant. We then fit multivariate linear regression models in which the key variable of interest was a difference-in-differences estimator, interpreted as the average effect of pausing the Oxford-AstraZeneca vaccine on overall vaccine uptake, in the countries that paused it. Our main model that investigated uptake of all vaccines as the primary outcome controlled for weekly supplies of all vaccines per population, and the model investigating the impact on uptake of the Oxford-AstraZeneca vaccine only controlled for weekly supplies per population of this specific vaccine. This was deemed to be a critical and observed time-varying confounder, which also varied across countries despite joint procurement plans, therefore being potentially correlated with differences in vaccine uptake between the countries that paused the vaccine rollout and those that did not. We included both country fixed effects to control for underlying time-invariant differences between countries, and week fixed effects to account for changes over time common to all countries.

We ran several further checks to assess whether our results were robust and to probe the assumptions that support the methodology, including the parallel trends assumption, which states that in the absence of the pause in vaccine rollout, included countries would have had the same trends in vaccine uptake as control countries. We first constructed graphs to examine the trends in average vaccine uptake before the pause. We then ran a statistical test for parallel trends using the ‘estat ptrends’ command in STATA 17(18). We ran a second model also adjusting for country and week fixed effects, to exclude countries that imposed later age restrictions on the use of the Oxford AstraZeneca vaccine from early April(19–23): Belgium, Estonia, Finland, France, Germany Iceland, Ireland, Italy, the Netherlands, Portugal, Sweden and Spain. We also ran a third multivariable model using the random-effects estimator and adding time-invariant covariates that were statistically significantly different between both groups of countries on prior hypothesis testing, including GDP per capita and EIU democracy index. In all models, similar results were obtained (Supplementary File).

Data was collected at the country-level and all countries were part of the EEA trading bloc, part of a joint vaccine procurement and distribution strategy(24). The difference-in-differences panel estimator could therefore be subject to serial autocorrelation, within-cluster correlation and cross-sectional dependence, potentially leading to standard errors that are not robust. We used clustered standard errors in all models to account for this.

Key Findings

There was no significant difference in subsequent vaccine uptake in countries that paused the Oxford-AstraZeneca vaccine in mid-March compared with those that did not. This was true for both one and two doses of the vaccine, and when looking at overall vaccine uptake (including all vaccines) and the Oxford-AstraZeneca vaccine alone. This suggests that over time, public confidence in vaccination was not significantly more damaged in those countries that paused the vaccine compared to those that did not. It appears there has been a shift away from using the Oxford-AstraZeneca vaccine in the EEA due to relatively low rates of uptake as a proportion of all vaccine uptake and a shallower increasing gradient in uptake over time compared to all vaccines, but this trend was similar across all included countries. This analysis suggests that as new COVID-19 vaccines emerge, regulators should be cautious to deviate from usual pharmacovigilance protocols on the sole basis of potential impacts on public confidence, if further investigation on clinical or epidemiological grounds is warranted.

Explaining findings

Many experts were concerned about the impact of pausing the Oxford-AstraZeneca vaccine on public confidence and consequent demand(6). Regulators that paused the rollout highlighted that VITT was serious but rare. Media and scientific communications emphasised the fact that investigating safety signals in this manner is an established phamacovigilance process done out of an abundance of caution, reiterating the well-established safety and effectiveness of vaccine(25). For instance, after pausing the vaccine Spain’s health minister Carolina Dias said “We take this decision today in the interest of caution,” referring to “infrequent cases which are very few but very significant and have prompted Spain to join the other countries that have opted for this precautionary suspension” (26). Around the same time, the UK Medicines and Healthcare products Regulatory Agency (MHRA) reported that VITT was seen in less than 1 per million vaccinated people in the UK, which had administered more Oxford-AstraZeneca doses than other European countries, and that a causal association had not been established(27). Most countries restarted using that vaccine approximately one week later, serving to further reassure the public in countries that initially paused the vaccine. It is likely that a short pause, variation in regulatory practice across countries, and reassuring public communications all helped to mitigate any potential impact on vaccine uptake in countries that did pause their rollouts.

Vaccine uptake may also have been similar across both groups of countries because of later scientific and policy developments on the Oxford-AstraZeneca vaccine. Although the majority of countries that had initially paused the vaccine in mid-March restarted it shortly after, 12 out of 28 included countries restricted its use in younger people from the second week of April(20–22,28). This was in response to concern about the balance of risks and benefits in the young. In these countries, the later impact of the pause on vaccine uptake (at weeks 15 and 18 in our study) may have been blurred by the impact of these restrictions. We did however control for vaccine supply over time across countries, which can be considered a proxy for the impact of such additional regulations since countries would be unlikely to procure vaccines they were not planning to use. A sensitivity analysis (Supplementary File) excluding the countries that imposed additional later restrictions on the use of the Oxford-AstraZeneca vaccine also showed that the results did not change. The relatively timid increase over time in uptake for both doses of the Oxford-AstraZeneca vaccine is likely a product of timing (with the second dose administered at 4-12 weeks after the first(29)) as well as these additional restrictions. Importantly, the same trend was seen in both countries that paused the vaccine and those that didn’t, suggesting that countries that paused the vaccine were not at a later disadvantage compared to others.

Strengths and Limitations

We were able to answer this critical public health policy question using a natural experiment, requiring no manipulation to allocate the intervention. All included countries were part of the EEA and therefore to some extent comparable, and our statistical analyses accounted for important differences across groups of countries where present. The intervention itself occurred at approximately the same time across countries, with a direct and immediate impact on both public information and national vaccine programme rollouts, further supporting our approach. Outcome data were available from a single and reliable source (the ECDC) meaning our dataset was largely complete and of a high quality. Through investigating the impact on uptake for all vaccines as well as the Oxford-AstraZeneca vaccine specifically, our study was well designed to understand the various mechanisms through which the pause could have affected vaccine uptake.

Our study had several limitations. First, although we were able to allocate countries to one of two groups, it is possible that there was contamination from the intervention to the control group. The European medicines regulatory system is based on a network of around 50 regulatory authorities from across EEA countries(30). Citizens may place more trust in their own national regulators compared with others. But populations in countries that did not pause the Oxford-AstraZeneca vaccine may have also been affected by knowing that similar countries (operating under a common European regulatory framework) decided to pause their rollouts of the vaccine, despite it being restarted soon after. While this could theoretically minimise any observed differences in vaccine uptake between countries, this is an important corollary of the intervention in a complex global regulatory environment during a pandemic, which should not be artificially adjusted for but considered part of the intervention itself. Second, although we were able to include data spanning 15 weeks, we were unable to assess the long-term impact of the pause on vaccine uptake, although for this particular question a short timeframe was deemed most appropriate to assess impact. Third, the difference-in-differences methodology relies on the assumption that there are no factors beyond the pausing of the Oxford-AstraZeneca vaccine that differentially affect vaccine uptake and thereby confound the observed effect. We accounted for these through adjusting for fixed country and week effects, adding time-variant fluctuations in supplies of vaccines across countries. We also controlled for other potential time-invariant confounders through a random-effects model. Nevertheless, the omission of other important, unobserved, time-varying characteristics could bias our findings if they affected the intervention and control group differentially. Finally, although difference-in-differences is an often used quasi-experimental method(31), inferences about causality can be less definitive than those obtained through a well designed randomised controlled trial. But it would not be possible to evaluate the impact of pausing the rollout of a vaccine during a pandemic, making quasi-experimental methods such as those used here the gold standard for policy evaluation.

Implications for research and policy

Although we found small non-significant decreases in overall vaccine uptake in countries that paused the Oxford-AstraZeneca vaccine compared to those that did not, we found a small non-significant increase in uptake for the first dose of the Oxford-AstraZeneca vaccine itself. Further research is required to understand whether populations feel more reassured after they know regulators have closely scrutinised the data on a particular vaccine and whether this paradoxically puts it at an advantage compared to others in terms of public perception and attitudes.

Despite arguments made against pausing the Oxford-AstraZeneca vaccine rollout, we found no evidence that it significantly impacted vaccine uptake. The findings from this study cannot however be generalized to future scenarios where vaccine risks, the stage of the pandemic, the mechanism and duration of pausing rollout, public perception of the regulators that pause and those that do not, and public communications could all be different. Future qualitative research will be needed to understand how these factors affect the impact of a nationwide pause in the rollout of a novel vaccine, and which policies and plans can be put in place to pause vaccines in as safe a way as possible, allowing further investigation whilst preserving public confidence. This analysis also highlights the general lack of alignment between European decision-making bodies, despite the fact that the public health impact of a regionally uncoordinated approach to stopping, restarting, and later restricting vaccine use, was largely unknown. Policymakers and regulatory authorities in Europe must collaborate through regional organizations like the EMA and global organizations like the WHO, to ensure that in the context of an international health threat, national policies and processes on vaccines are harmonised as far as possible. In the long-term this will be necessary to promote public trust in COVID-19 vaccines and in the institutional processes related to their selection, procurement, distribution and monitoring.