COVID-19 Vaccination Mandates and Vaccine Uptake ================================================ * Alexander Karaivanov * Dongwoo Kim * Shih En Lu * Hitoshi Shigeoka ## Abstract We evaluate the impact of government-mandated proof of vaccination requirements for access to public venues and non-essential businesses on COVID-19 vaccine uptake. We find that the *announcement* of a mandate is associated with a rapid and significant surge in new vaccinations (more than 60% increase in weekly first doses), using the variation in the timing of these measures across Canadian provinces in a difference-in-differences approach. Time-series analysis for each province and for France, Italy and Germany corroborates this finding. Counterfactual simulations using our estimates suggest the following cumulative gains in the vaccination rate among the eligible population (age 12 and over) as of October 31, 2021: up to 5 percentage points (p.p.) (90% CI 3.9–5.8) for Canadian provinces, adding up to 979,000 (425,000-1,266,000) first doses in total for Canada (5 to 13 weeks after the provincial mandate announcements), 8 p.p. (4.3–11) for France (16 weeks post-announcement), 12 p.p. (5–15) for Italy (14 weeks post-announcement) and 4.7 p.p. (4.1–5.1) for Germany (11 weeks post-announcement). Keywords * COVID-19 * vaccine mandates * proof of vaccination * vaccine uptake * vaccine hesitancy * difference-in-differences * time-series analysis * counterfactuals ## 1 Introduction Immunization with mRNA or adenoviral vaccines has proven very effective for reducing the spread and severity of COVID-19, with vaccinated people benefiting from a large reduction in the risk of severe outcomes.1 Yet, following a quick-paced uptake in early 2021, COVID-19 immunization rates in many countries slowed down significantly, at about 60% of the population, during the summer despite the vaccines’ proven benefits (see Fig. B1). In addition, most countries or regions, even those with high vaccination rates, experienced increased viral transmission or had to maintain or re-introduce non-pharmaceutical interventions such as mask-wearing mandates or indoor capacity limits in fall 2021 because of the elevated reproduction rate of the Delta variant.2 A further increase in vaccination rates therefore remains essential for reducing the health and economic impacts of COVID-19 and allowing the eventual lifting of restrictions. Moreover, the administration of booster doses or vaccines with updated formulations may be necessary in the face of new variants. Public health authorities all over the world have actively sought the most effective strategies to increase vaccine uptake and provide incentives for hesitant, procrastinating or other people who have had the opportunity to be immunized for a prolonged time but have not taken it up. In response to this challenge, many local or national governments have recently introduced *proof of vaccination* mandates, which allow only vaccinated persons to attend non-essential sports or social activities, events and settings such as concerts, stadiums, museums, restaurants, bars, etc.3 Some mandates accept a recent negative COVID test or past positive test as a substitute of vaccination or allow businesses to opt out if they abide by additional restrictions.4 The goal of these policies is twofold: to provide incentives for immunization and to reduce viral transmission in risky indoor or crowded settings.5 We evaluate and quantify the effect of proof of vaccination mandates on first-dose vaccine uptake in the ten Canadian provinces and three European countries (France, Italy and Germany) that announced and implemented such mandates in the period July–October, 2021. This is a period without binding vaccine supply or access constraints and with a high base first-dose vaccination rate (above 60% of those eligible in the three countries and above 80% in Canada, at the time of the mandate announcements). Hence, we evaluate the mandates’ impact on people, such as the vaccine hesitant, that have remained unvaccinated for weeks or months after immunization became available to them. While requiring proof of vaccination is expected to raise vaccine uptake, the magnitude and speed of the increase are hard to predict: they depend on the relative importance of the factors leading to delay or hesitancy, e.g., lack of social or economic incentives, misinformation, or entrenched political or religious beliefs. We use first doses as the main outcome in our statistical analysis because they most directly reflect the decision to be immunized.6 The Canadian provincial data is key for our identification strategy as it allows us to use the time variation in mandate announcement dates (from August 5, 2021 in Quebec to September 21, 2021 in Prince Edward Island) across different geographic units in the same country, via a difference-in-differences (DID) approach.7 In contrast, the French, Italian or German mandates, or the announced U.S. vaccine mandate for employees, apply at the national level, which makes it more challenging to separate the effect of the mandate from that of time trends or other concurrent events or policies. Fig. 1 plots the weekly vaccine first doses administered in the four most populous Canadian provinces and four European countries. All except Spain had introduced a province-wide or country-wide proof of vaccination mandate in the studied period. We observe a sizable boost in vaccine uptake within two weeks after the mandate announcement date (the dashed vertical line) in all four provinces and in France, Italy and Germany, often following a sharp decline in the pre-announcement weeks. In France, daily first-dose appointments data also shows a striking surge on the day after the mandate announcement (see Fig. B2). In contrast, Spain exhibits a steady decrease in weekly first doses over the displayed period. ![Figure 1:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/01/13/2021.10.21.21265355/F1.medium.gif) [Figure 1:](http://medrxiv.org/content/early/2022/01/13/2021.10.21.21265355/F1) Figure 1: Vaccination mandates and first dose uptake Notes: The figure plots the weekly first doses of COVID-19 vaccines administered for dates *t−* 6 to *t*, where *t* is the date on the horizontal axis. The dashed red lines denote the proof of vaccination mandate announcement dates (for countries, this is the date of a *national* mandate). Spain had not announced a national proof of vaccination mandate as of October 31, 2021. We show the four most populated Canadian provinces totalling about 87% of Canada’s population (see Fig. B4 for all ten provinces). Alberta also had a $100 debit card incentive for doses received between Sep. 3 and Oct. 14, 2021. Motivated by this evidence, we aim to answer two important policy-relevant questions. First, what is the magnitude of the increase in the pace of first dose vaccinations after a proof of vaccination mandate announcement? Second, how long do these gains persist and what is the cumulative effect of the mandates on vaccine uptake? Using Canadian province-level data in a difference-in-differences (DID) statistical analysis, we find a quick and significant increase in vaccine uptake, measured by new weekly first doses, following the announcement of a proof of vaccination mandate. In our baseline DID specification, using the period June 15 to Sep. 14, 2021 and splitting the data into five ‘treated’ provinces (those with the earliest announcements) and five ‘control’ provinces (not yet announced), we estimate a 66% (50.6 log points, 95% CI 24.5–76.7)8 average increase in weekly first doses following the mandate announcement, relative to in absence of mandate. The increased uptake is sustained over the post-announcement period with available data. We obtain similar results, more than 60% increase in weekly first doses on average over the post-announcement weeks, also when using alternative dates and treatment groups varying from the first three to the first nine provinces to announce a mandate. We complement and extend the DID panel-data results with structural-break and time-series analysis, which allows us to study the mandates’ longer-term impact on vaccine uptake using our full dataset up to October 31, 2021 and to obtain individual policy effect estimates for each province and country. We test for the presence of a structural break at the announcement date and find that we can reject the null hypothesis of no break for each of the ten provinces and three countries. The time-series results confirm the rapid and large increase in first-dose uptake after a mandate announcement relative to the no-mandate trend found in the DID analysis. The estimated effect varies across the provinces, e.g., a 34% (29 log points, 8.2–50.5) initial increase in Ontario vs. 326% (145 log points, 110–181) in Alberta. We note that several factors may contribute to the heterogeneity of the estimates, including the time between the mandate announcement and implementation and the fraction of the population that is already vaccinated at the time of announcement (both are negatively correlated with the size of the estimated initial effect). We find similar results for the countries – an average estimated initial increase in weekly first doses of 17% (15.4 log points, 5.0–25.8) for France, 179% (103 log points, 53–152) for Italy and 55% for Germany (43.8 log points, 31.4–56.1), relative to the respective pre-mandate trends.9 Using our time-series estimates for each province in counterfactual simulations, we find large cumulative increases in the first-dose vaccination rate for all provinces, relative to in the absence of mandate: from 1.9 p.p. (90% CI: -0.3–3.0) in Ontario to 5 p.p. (4.0–5.6) in Saskatchewan and 5 p.p. (3.9–5.8) in New Brunswick, with all other provinces in between, as of October 31, 2021. These estimated total gains in uptake add up to 2.9 p.p. (1.3–3.8) of the eligible population, or 979,000 new first doses (425,000–1,266,000), for Canada as a whole. This is a significant increase in vaccine uptake considering the relatively short time in which it was achieved and the very high pre-mandate first dose vaccination rate in Canada (over 80% on average at the time of the mandate announcements). We also estimate large increases in first-dose vaccinations relative to the no-mandate counterfactual in the three countries: 8 p.p. (4.3–10.8) or 4.59 mln (2.47–6.25) doses in France, 12 p.p. (5.0–15.1) or 6.48 mln (2.67–8.14) additional first doses in Italy, and 4.7 p.p. (4.1–5.1) or 3.47 mln (3.06–3.81) doses in Germany as of October 31, 2021, in our baseline specification. These estimated gains are larger than that for Canada, possibly because of the earlier mandates in these countries or the lower starting vaccination rates, and they may partly reflect expanding the scope of the initial mandates, e.g., in Italy. Vaccination mandates have been controversial, as some people perceive them as restrictions on personal freedom. This can affect compliance and increase both the direct implementation and enforcement costs, as well as the political costs of introducing a mandate. We do not address ethical considerations in this paper. Our goal is to assess the mandates’ effectiveness purely in terms of raising vaccine uptake, which can then be weighed against various costs and compared to other policies including financial incentives (cash, gift cards, lotteries) or behavioral nudges (e.g., messages from experts, appointment reminders). In related work, Mills and Rüttenauer (2021) use a synthetic control approach to evaluate the impact of COVID-19 certification mandates on vaccine uptake, overall and by age group, in six countries. The authors find significant increases in daily total vaccinations in France, Israel and Italy between 20 days prior to 40 days after the mandate implementation.10 While the main takeaway is similar, our paper differs in several important ways. First, we provide robust difference-in-differences evidence using the variation in mandate timing within the *same country*, Canada, in addition to evidence using time-series methods for France, Italy and Germany. Second, instead of total vaccinations, we focus on *first doses*, as most directly reflecting the decision to become immunized and avoiding potential issues related to second dose scheduling or availability in spring/summer 2021. This is one possible reason why, unlike Mills and Rüttenauer, we find a statistically significant increase in vaccine uptake in Germany around the mandate.11 Third, instead of using a fixed 20-day cutoff before implementation, we use the actual mandate *announcement* date12 as the intervention indicator and show a strong impact on vaccine uptake thereafter. There is little evidence that financial or behavioural nudges increase the vaccination rate among hesitant people. Chang et al. (2021) report results from a mid-2021 randomized controlled trial (RCT) with unvaccinated members of a large Medicare health plan in a California county with 77% vaccination rate at the time of the study. The authors examine the effect of $10 or $50 financial incentives, different public health messages and an appointment scheduler, on vaccination intentions and uptake within 30 days of the intervention. They find that none of the financial or behavioural treatments increased the vaccination rate among the treated. The proof-of-vaccination mandates we analyze target a similar group of unvaccinated people that have had the opportunity to be vaccinated for a long time. The population-weighted average first-dose vaccination rate on the dates of the provincial mandate announcements is 83.3%, even higher than in Chang et al. (2021), suggesting that financial or behavioural incentives are even less likely to be effective in our setting.13 On behavioral nudges, see also Dai et al. (2021), Alsan and Eichmeyer (2021) and Kluver et al. (2021). On financial incentives, see Barber and West (2021), Brehm et al. (2021), Sehgal (2021), and Walkey et al. (2021) on Ohio’s vaccine lottery, with mixed results. ## 2 Data ### 2.1 Data sources and definitions We use data on COVID-19 vaccination numbers, cases and deaths for all ten Canadian provinces, as well as for France, Italy, Germany and Spain.14 Announcement and implementation dates of the proof of vaccination mandates were collected from the respective government websites and major newspapers (see Table C1). The main variables in our statistical analysis are defined below. Everywhere, *i* denotes province or country and *t* denotes time measured in days (date). We aggregate the data on vaccinations, cases and deaths on a weekly basis (totals for the week ending on date *t*, i.e., dates *t −* 6 to *t*) to reduce the influence of day-of-the-week effects and reporting artifacts.15 #### Outcome, *V**it* Our main outcome variable is the logarithm of administered vaccine *first doses* per 100,000 people, *V**it*, for the week ending at *t* (dates *t −* 6 to *t*). We use first doses as they most directly reflect the impact of proof of vaccination mandates on the intent to be immunized.16 Using the logarithm of weekly first doses allows us to interpret the regression coefficients as percentage changes in the outcome; moreover, the estimates are invariant to normalization, e.g., by population, as its effect is subsumed in the regression constant or fixed effects. We thus use “log weekly first doses” for simplicity throughout the paper, except where the actual scale is important. #### Policy, *P**it* Let ![Graphic][1] be the announcement date of the proof of vaccination mandate in jurisdiction *i*. We construct a binary policy variable *P**it* equal to 1 for all post-announcement dates ![Graphic][2] and equal to 0 for all ![Graphic][3]. Proof of vaccination mandates were announced in all ten Canadian provinces over the period August 5, 2021 to September 21, 2021 (see Table C1 and Fig. B3).17 The last six provinces to act were the four Atlantic provinces, which had the lowest per capita case rates in August 2021, and Alberta and Saskatchewan, which had the highest per capita case rates in August 2021. Several European countries, including France, Italy and Germany, also introduced proof of vaccination mandates in summer 2021 (see Table C1). #### Information, *I**it* We construct variables related to the underlying COVID-19 epidemiological situation, specifically log of weekly cases, *C**it*, and log of weekly deaths, *D**it*, for the week ending at date *t* (dates *t −* 6 to *t*).18 We refer to these variables jointly as *information, I**it*, (see Chernozhukov et al., 2021 or Karaivanov et al., 2021) since they can inform a person’s COVID-19 exposure risk assessment and/or decision to be vaccinated.19 #### Controls, *W**it* We include province fixed effects and date fixed effects in our panel data analysis. The province fixed effects account for any time-invariant province characteristics such as sentiment towards vaccination, age distribution, etc. The date fixed effects control for nation-wide trends or events, e.g., public messaging, vaccine-related international travel regulations, or political campaigning for the September 2021 federal election. In the time-series analysis, we control for time trends. #### Time period We use the period June 15, 2021 to October 31, 2021. The start date is chosen to ensure that possible constraints to obtain a first dose related to eligibility or vaccine supply are minimal or non-existent.20 We explore different sample start dates in robustness checks. The sample end date is based on data availability at the time of statistical analysis and writing. First doses for the 5–11 age group were not approved in this period. ### 2.2 Descriptive analysis Fig. 2 plots the time profile of weekly first doses after a mandate announcement, with first doses in the week ending on the mandate announcement date normalized to 100. The figure only provides a raw-data illustration of the magnitude of the increase in weekly first doses following a mandate announcement. We follow up with formal analysis, controlling for information, fixed effects, pre-trends and other factors, in Sections 3–5. ![Figure 2:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/01/13/2021.10.21.21265355/F2.medium.gif) [Figure 2:](http://medrxiv.org/content/early/2022/01/13/2021.10.21.21265355/F2) Figure 2: First doses after mandate announcement Notes: The figure plots the weekly administered first doses of COVID-19 vaccine for all dates after the mandate announcement against the number of weeks since the respective announcement date (denoted by 0 on the horizontal axis), as of Oct. 31, 2021. The weekly first doses for the week just prior to the mandate announcement are normalized to 100 for each respective province (on the left) or country (on the right). Fig. 2 shows that weekly first doses in the Canadian provinces and in France, Italy and Germany grow quickly after the proof of vaccination mandate announcement, reach a peak at about 1 to 3 weeks after the announcement date and then decrease, as in the pre- announcement trend in most locations (see Fig. B4).21 Several provinces, notably New Brunswick, Newfoundland, Alberta, Nova Scotia and Saskatchewan, as well as France, registered large increases in first-dose vaccine uptake of over 100% relative to the pre- announcement week. In other locations, e.g., Quebec, Ontario, Manitoba and Germany the observed increase is more moderate (less than 50%). ## 3 Panel data results – Canadian provinces ### 3.1 Estimation We estimate a behavioral model in which the decision to receive a COVID-19 vaccine (measured by new weekly first doses) is affected by the policy setting, *P**it* (whether a proof of vaccination mandate has been announced) and the current COVID-19 epidemiological and public health conditions, *I**it*, proxied by weekly cases and deaths. Based on the raw data patterns in Fig. 1 and Fig. B4 and the absence of vaccine supply constraints in the studied period, we assume no lag between the mandate announcement and a person’s ability to receive a first vaccine dose.22 Finally, we assume that individuals take into account the weekly public health indicators, *I**it*, for the week ending at date *t*, with no lag. We use a difference-in-differences (DID) method to study the average effect of proof of vaccination mandates in Canada. In addition to information *I**it*, we control for other potential confounding variables or unobserved heterogeneity by including time and location fixed effects. We estimate the following equation: ![Formula][4] where *V**it* is log weekly first doses for location *i* and date *t, I**it* is information, *γ**t* are date fixed effects, *δ**i* are province fixed effects, and *ε**it* is the error term. The coefficient *α* on the policy variable *P**it* captures the average effect of the mandate announcement on weekly first doses over all post-announcement dates. To capture dynamic effects of the mandates, we also estimate a version of equation (1) with *P**it* broken into separate indicator variables, one for each week after the mandate announcement date. To correct for the small number of clusters in the estimation due to there being only ten provinces, we compute and report “wild bootstrap” p-values (Cameron et al., 2008).23 The recent methodological literature has argued that the standard OLS two-way fixed effects (TWFE) estimator can be invalid in panel-data settings with staggered adoption like ours if there is heterogeneity in the treatment effect across groups/cohorts (provinces in this paper) and/or over time. The reason is that the TWFE estimate is a weighted average of many 2-by-2 DID treatment effects, where some of the weights can be negative or incorrect because of contamination from other periods (Goodman-Bacon, 2021).24 In particular, Sun and Abraham (2021) develop an ‘interaction-weighted’ difference-in-differences estimator that is valid under these conditions and which we use to estimate (1). The Sun and Abraham estimator uses never-treated or last-treated units as the control group and is constructed as an appropriately weighted average of treatment effects for each cohort (by date of mandate announcement) and each relative time after or before the announcement.25 The Sun-Abraham estimator requires excluding from the estimation all time periods in which units in the control group are treated. Since the last province to announce a mandate is Prince Edward Island, on September 21, 2021, this means that we can use only data until September 20, at the latest (see Table C1). We present results for different control groups and corresponding sample periods in Table A1. We also compare our baseline results with the OLS TWFE estimates in Table A2. In addition, in Section 4 we report results from time-series analysis using all available data until October 31. ### 3.2 Results Fig. 3 displays an event study analysis of weekly first doses administered in the ten Canadian provinces, from six weeks before to five or more weeks after the announcement of a proof of vaccination mandate. *T* = 0 denotes the week starting at the announcement date. The reference point is one week before the mandate announcement (*T* = –1). We use the Sun and Abraham (2021) treatment effect heterogeneity robust estimator and use as the control group (latest treated) the last five provinces to announce a mandate (Alberta, New Brunswick, Saskatchewan, Newfoundland and Prince Edward Island). We chose this control group as the latest-in-time representative set of provinces containing a mix of smaller and larger provinces by population. The control group implies September 14, 2021 (the day before mandate announcement in Alberta and New Brunswick, see Table C1) as the (latest possible) sample end date in our baseline specification. We present results with different control groups, corresponding to both earlier and later end dates, in Table A1. ![Figure 3:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/01/13/2021.10.21.21265355/F3.medium.gif) [Figure 3:](http://medrxiv.org/content/early/2022/01/13/2021.10.21.21265355/F3) Figure 3: Canadian provinces – event study Notes: Sun and Abraham (2021) treatment effect heterogeneity robust estimates. Sample period: June 15 to September 14, 2021 using Alberta, New Brunswick, Saskatchewan, Prince Edward Island and Newfoundland as the control group (latest treated). The outcome variable, *V**it* is log weekly first doses per 100,000 people administered for dates *t −* 6 to *t* inclusive. The figure plots the estimates from a variant of equation (1) where the mandate announcement variable *P**it* is replaced by the interaction of being in the ‘treatment’ group (announced mandate) with a series of dummies for each week ranging from 6 weeks before (*T* = *−* 6) to 5 or more weeks after the announcement (*T* = 5), where *T* = 0 is the week of the announcement. The reference point is one week before the announcement (*T* = –1). The dotted lines correspond to 95% confidence intervals. We make several observations on Fig. 3. First, there is no mandate-associated pre-trend – the Sun-Abraham DID estimates before the mandate announcement are statistically indistinguishable from zero. This addresses the potential endogeneity concern that provinces that announced a mandate may have had a different trend in first-dose vaccinations than provinces that did not announce a mandate. Second, the impact of the mandate announcement on first-dose vaccine uptake is realized relatively quickly. Third, the mandate effect on the pace of first-dose vaccinations is large in magnitude — an increase of 42% (35 log points, CI 9–62) in weekly first doses in the first post-announcement week and 71% (54 log points, 21–86) in the second week, each relative to one week before the announcement.26 The observed quick increase in uptake mitigates possible concerns that vaccine supply or scheduling constraints may be affecting our results (see also Fig. 2 for a raw data plot of the post-announcement increase in first-dose vaccinations by province). Fourth, the average policy effect persists over time over the 6-week post-announcement period we analyze (*T* = 0 to *T* = 5). Unfortunately, the limitations of the data (the timing of the announcements) and the need for a not-yet-treated control group do not allow us to investigate longer horizons. We return to this in Section 4 using time-series analysis of all the data until October 31. Table 1 shows the Sun-Abraham DID estimates of equation (1) along with p-values from wild bootstrap standard errors clustered by province in the square brackets.27 Columns (1) and (2) (without and with controlling for weekly cases and deaths) show that the mandate announcement is associated with an average increase of about 66 percent (50.6 log points, CI 25-77 in column 2) in weekly first doses. In column (3) of Table 1, we report estimates from a variant of equation (1) where the policy variable *P**it* is decomposed into six binary variables (one for each week after the announcement date) to account for dynamic effects. The results indicate a sharp increase in weekly first doses of 43 percent (36 log points, CI 14–58, *p <* 0.01) in the week beginning at the announcement date (week 0). The increase is sustained until the sample end date as all estimates in column (3) are positive (*p <* 0.02, except for “week 5+”, *p <* 0.06) and larger than in week 0. We note that the later estimates are based primarily (week 3) or only (week 4) on post-mandate vaccinations in Quebec, the province with the earliest announcement. Thus, the DID analysis yields no evidence that short-term net intertemporal substitution (which would be manifested as a negative policy estimate for the later weeks) is the mechanism behind the observed boost in vaccinations in the studied period. View this table: [Table 1:](http://medrxiv.org/content/early/2022/01/13/2021.10.21.21265355/T1) Table 1: Canadian provinces – difference-in-differences results In Table A1, reporting policy estimates by week after announcement and the upper left panel of Fig. B5, reporting the average estimate for the post-announcement period, we use different sets of provinces as control group, from only Prince Edward Island (PEI) (using data up to Sep. 20), to all provinces except Quebec, British Columbia and Manitoba (using data up to Aug. 31). We find that our main results from Table 1 remain robust to using alternative control groups. ### 3.3 Robustness analysis #### Alternative initial dates In Fig. B5, upper right panel, we show that our estimates of the effect of a mandate announcement on first dose vaccine uptake are not very sensitive to the choice of initial sample date between May 1 and Jul. 15, 2021. This provides further reassurance that vaccine supply constraints are not a major concern for our results. #### Lags In equation (1), we assume no lag between the mandate announcement *P**it* and a person’s ability to receive the first dose (the outcome *V**it*). In practice, a small delay may occur (e.g., from making an appointment to receiving the vaccine) even in the absence of supply constraints. The lower left panel of Fig. B5 displays the coefficient estimates ![Graphic][5] of the policy effect when assuming a lag of zero (our baseline) to seven days. The mandate effect estimates remain large and statistically significant when varying the lag length, with a slight decrease for the longer lags. The lower right panel of Fig. B5 displays the policy estimates ![Graphic][6] when assuming a lag of zero (our baseline) to 14 days in the information variables *I**i**t* (log weekly cases and deaths). The mandate effect estimates remains large and statistically significant when varying the lag length.28 #### Other robustness checks We perform additional robustness checks in Table A2. For easier comparison, column (1) replicates our baseline estimates from columns (2) and (3) of Table 1. Column (2) of Table A2 omits cases and deaths, the coefficients for which are not significant in the baseline specification. In column (3), we control for log weekly deaths and log average weekly hospitalizations as information. Column (4) reports the estimates of a weighted specification using the provincial populations as weights and the baseline information variables (cases and deaths). The estimates are smaller than in the unweighted baseline (column 1), suggesting that smaller provinces contribute more to the the estimated effect of mandate announcements on vaccine uptake. In column (5), we re-estimate equation (1) using the standard OLS two-way fixed effects (TWFE) estimator which, as discussed in Section 3.1, can be biased in the presence of heterogeneous treatment effects over cohorts and/or time. In our setting, we find that these estimates differ relatively little from our baseline Sun-Abraham treatment effect heterogeneity robust estimates. In Table A2 column (6), we use log of daily (instead of weekly) first doses as the outcome and log of daily cases and deaths as information and find similar estimates to those in our baseline specification, although larger in week 0 since the outcome is not a moving sum. In column (7), we use the *level* of weekly first doses per 100,000 people as the outcome. The estimates are noisier than in the baseline and indicate that a mandate announcement increases weekly first dose uptake by roughly 482 per 100,000 people on average (CI -11– 975, *p <* 0.06) after the announcement. Overall, all these alternative specifications confirm the robustness of our main finding of statistically significant and large impact of mandate announcements on vaccine uptake. #### Randomization inference Difference-in-differences inference from the model in equation (1) relies on asymptotic approximations requiring a sufficiently large number of provinces. While we do account for the fact that there are only 10 provinces in our sample by using wild bootstrap to compute the estimates’ p-values, as an additional robustness check, we implement a variant of Fisher’s randomization test (Fisher, 1935). Specifically, we estimate equation (1) 5,000 times by randomly assigning the mandate announcement date for each of the ‘treated’ provinces in our baseline specification. Fig. B6 plots the histogram of the randomized inference (placebo) mandate effect estimates, along with the 95% confidence band. Only 2.88% of the 5,000 placebo estimates are larger than our baseline estimate 0.506 from column (2) of Table 1 (the solid vertical line on Fig. B6), offering assurance that our main result does not hinge on asymptotic inference. ### 3.4 Second doses In Table A4, we estimate the specifications in Table 1 using log of weekly *second doses* as the outcome. There are two potential mechanisms through which proof of vaccination mandates may have an impact on second dose uptake: (i) people that already had a first dose decide to receive their second dose soon after the announcement or implementation (e.g., because most mandates require full vaccination) and (ii) people that took a first dose because of the mandate take their second dose roughly 4 weeks after the first. We do not find a statistically significant effect of mandate announcements on second dose uptake in our sample period. This is consistent with second doses being more spread out after the announcement. We also plot weekly second doses over time on Fig. B7 and, unlike the sharp increases for first doses, we only see relatively small and gradual second-dose increases after the mandate announcements in some of the (smaller) provinces. ### 3.5 Mandate implementation Our main results use the regional variation in the timing of announcement of proof of vaccination mandates to examine their effect on vaccine uptake. In addition, there is variation in the interval between the mandates’ announcement and *implementation* (coming into force), as well as in the implementation dates themselves – see Table C1 and Fig. B4.29 We checked whether we can detect additional effects on first-dose uptake, associated with the mandates’ implementation, by adding to equation (1) an indicator variable that equals to 1 post-implementation and 0 otherwise. The estimate on this indicator variable is positive but not statistically significantly different from zero (not displayed). However, we cannot conclude that there do not exist implementation effects, since even the latest feasible sample end date (Sep. 20, 2021) necessitated by the requirement to have a not yet treated control group is only soon after the mandate implementation date in 3 of the 10 provinces (see Table C1). ## 4 Time series analysis ### 4.1 Estimation method We complement and extend our panel data analysis for Canada by time-series analysis for each of the ten Canadian provinces, as well as for France, Italy and Germany. While the Sun and Abraham (2021) difference-in-differences approach in Section 3 allows us to reliably estimate the impact of mandate announcements on new first doses in Canada by exploiting the variation in the timing of the mandates across provinces, it can only be used with data from before the last treatment date and hence it does not allow us to evaluate the effect of the mandates beyond the short term. In contrast, under stronger identification assumptions, the time-series approach allows us to use all our data until October 31, 2021 to estimate the mandates’ effect on vaccine uptake and the cumulative additional vaccinations compared to the counterfactual of no mandate separately for each province and country. We use the ‘interrupted time-series analysis’ (ITSA) framework (e.g., Linden, 2015) and estimate the following equation for each country or province30 ![Formula][7] where *V**t* are log weekly first doses for the week ending at date *t, c* is a constant, *I**t* is proxy for information analogous to its counterpart in equation (1), and *η**t* is the error term. In our baseline time-series specification, we construct the policy variable *P**t* as the weekly average from date *t −* 6 to *t* of the ‘mandate announced’ indicator *P**it* defined in Section 2.1.31 We include 7-day and 14-day lagged values of *V**t* (instead of *t −* 1 or *t −* 2 autoregressive terms) since the outcome variable *V**t* is a weekly total. We include two time trends in equation (2): *T**t* is a linear daily time trend initialized at the sample start date *t* = 0 and ![Graphic][8] is an ‘interaction’ time trend which takes value 0 at all dates until the announcement date (inclusive) and increases by 1 for each day afterwards. The coefficients *τ*1 (slope) and *τ*2 (change in slope at the announcement date) characterize respectively the baseline (pre-announcement) time trend and the post-announcement time trend (with slope *τ*1 + *τ*2) in weekly first doses. The time-series estimation requires stronger identifying assumptions than the DID approach since there is no control group. Specifically, we need to assume that the time-series process of weekly first doses changed after the mandate announcement only because of the policy, i.e., that it would have followed the same pre-trend if no mandate had been announced. Another needed assumption is exogeneity of the announcement date. We also cannot control for fixed effects in a flexible way beyond including a constant and time trends. Under these assumptions, the coefficient on the policy variable *P**t* captures the average level shift in first-dose uptake attributed to the mandate announcement. Similarly, the coefficient on the interaction time trend ![Graphic][9] measures the slope change in the time trend of first doses after the mandate announcement, relative to the pre-announcement trend. ### 4.2 Structural break at the mandate announcement date We first show the impact of the proof of vaccination mandate announcements on first dose uptake in Table A5, where we perform a structural break test (Chow (1960) test for a known break point) for the log weekly first doses, *V**t*. The presence of a break point in *V**t* at the mandate announcement date indicates an abrupt change (shift) in the first-dose time series process. We use a bandwidth of 50 days before and 35 days after the mandate announcement date.32 The before-after bandwidth was chosen to reduce the size distortion of the test since the outcome variable is a weekly sum and the error terms are serially correlated.33 Columns (1) and (2) in Table A5 use the log of weekly first doses, *V**t*. For all ten Canadian provinces and all three countries, we strongly reject the null hypothesis that the mandate announcement date is not a structural break (*p <* 0.01). We also perform the structural break test using first-differenced weekly first doses in Table A5 column (3). The differenced series is stationary, and the error terms are not serially correlated, which alleviates concerns about size distortion in the Chow test. The null hypothesis of no structural break at the announcement date is rejected with *p <* 0.05 in most locations except Saskatchewan and three Atlantic provinces.34 Overall, the structural break test results suggest that a mandate announcement is strongly associated with a trend break in first-dose vaccine uptake in all or most locations depending on the specification. ### 4.3 Results In Table 2, we report the results from estimating the time-series model (2) for each country and province in our data.35 We include log weekly cases and deaths as information, *I**t*. In columns (1)-(3), we report the estimated coefficients on the mandate announcement variable, *P**t*, the time trend, ![Graphic][10], and the interaction time trend, *T**a*, in equation (2). Since the outcome and information variables are level variables that can introduce non-stationarity, we use the augmented Dickey-Fuller (ADF) unit root test to test whether the residuals *η**t* are stationary, to avoid a spurious regression.36 We reject the null hypothesis that the residual contains a unit root at the 95% confidence level for all locations except Newfoundland, where we can reject the null at the 90% level. View this table: [Table 2:](http://medrxiv.org/content/early/2022/01/13/2021.10.21.21265355/T2) Table 2: Time series estimates The coefficient estimates ![Graphic][11] on the mandate announcement variable *P**t* in Table 2 are large and statistically significantly positive (*p <* 0.01) for all three countries. Italy’s estimated initial rise in weekly first doses after the mandate announcement is the largest (102.5 log points, CI 53-152), i.e., 179% average increase relative to the pre-announcement trend, followed by Germany (43.8 log points, CI 31–56 or 55% increase) and France (15.4 log points, CI 5–26 or 17% increase).37 The baseline time trend is downward sloping ![Graphic][12] in all countries and indicates a steep decrease in vaccinations of 1.7% to 2.9% per day. The post-announcement trend in first doses turns less steep in all three countries (the post-announcement slope change estimate ![Graphic][13] is positive, although only statistically significant for Germany), which suggests a lack of net intertemporal substitution as of October 31. Turning to the Canadian provinces in Table 2 we obtain statistically significantly positive and large estimates of the increase in weekly first doses after mandate announcement, ![Graphic][14] for all provinces except Quebec.38 The statistically significant policy estimates range from 0.293 (0.082–0.505) in Ontario, equivalent to a 34% average increase in first doses relative to the pre-announcement trend, to 1.455 (1.102–1.808) in Alberta (a 328% increase). The range of estimates is consistent with our DID estimate in Table 1 for the average policy effect for Canada, although the two estimation methods are not directly comparable. The estimated effect of the mandate announcement tends to be larger for provinces experiencing a surge in cases at the same time (for which we control in *I**t*), namely Alberta, Saskatchewan and New Brunswick. These provinces also announced their mandates relatively late (mid-September) and set a relatively short time interval between mandate announcement and implementation (see Table C1). A larger fraction of eligible unvaccinated people at the time of mandate announcement is also positively associated with a larger policy effect estimate. We illustrate these associations in Fig. B8. As in the results for the countries, the estimates of the base time trend in first doses, *τ*1, are negative for all ten provinces (although not statistically significant in the two smallest provinces, Newfoundland and Prince Edward Island). The estimates ![Graphic][15] of the post-announcement change in slope of the time trend are large and statistically significantly negative for Alberta, New Brunswick, Saskatchewan, and PEI (British Columbia’s estimate is also negative but not statistically significant). This result is consistent with the mandate effect on new first doses diminishing over time in these provinces.39 In Table A6, we also present results from two alternative specifications of equation (2): using log weekly deaths and log average weekly hospitalizations as information, in columns (1) and (2), and using a binary (instead of weekly-averaged) policy variable *P**t*. The results are very similar to our baseline findings in Table 2 and confirm the positive and large estimated effect of mandate announcements on first dose vaccine uptake.40 ### 5 Counterfactual analysis We use our estimates of the mandate effect on vaccine uptake from Table 2 to compute the total increase in new vaccinations for each province and for France, Italy and Germany, relative to in the counterfactual scenario of absence of mandate. Calling ![Graphic][16] the mandate announcement date, we compute the counterfactual log weekly doses per 100,000 people, ![Graphic][17] iteratively as ![Formula][18] where ![Graphic][19] is the coefficient estimate of the mandate announcement variable *P**t* from column (1) of Table 2, ![Graphic][20] is the estimate of the interaction time trend ![Graphic][21] from column (2) of Table 2, and where the terms multiplied by the estimates ![Graphic][22] and ![Graphic][23] account for the the lagged terms *V**t−* 7 and *V**t−* 14 in equation (2). The policy effect estimate ![Graphic][24], as well as ![Graphic][25] and ![Graphic][26] are specific for each respective country or province. The counterfactual simulations assume that all explanatory variables except the mandate announcement (e.g., deaths or time fixed effects) remain fixed at their observed values, and that the estimated model parameters remain stable. These assumptions are more plausible over relatively short periods and hence the simulations should be interpreted primarily as an illustration of the estimated impact of the proof of vaccination mandates on vaccine uptake rather than policy guidance. ### 5.1 Canadian provinces We next use the time-series estimates in Table 2 to compute counterfactual weekly doses, ![Graphic][27], as defined in (3), and counterfactual total first doses for each province. The results are displayed in Fig. 4. The left vertical axis displays total first doses in millions. The right axis displays the total doses in terms of the percentage of eligible people (age 12 and above; see Table C3). The number next to the province name is the estimated percentage-point (pp) increase in first doses relative to the no-mandate counterfactual, as of October 31, 2021. In Fig. B9 we also display these results in terms of actual vs. counterfactual weekly doses. In Table A7 we report the gains in thousands of doses, along with estimated gains from alternative specifications. ![Figure 4:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/01/13/2021.10.21.21265355/F4.medium.gif) [Figure 4:](http://medrxiv.org/content/early/2022/01/13/2021.10.21.21265355/F4) Figure 4: Canadian provinces – observed vs. no-mandate counterfactual first doses as of October 31, 2021 (time-series estimates) Notes: The figure plots the observed (black diamonds) and mean counterfactual (dotted red line) cumulative first doses for each province by date. The counterfactuals are computed using the estimates in Table 2. The shaded areas denote 5–95 percentile confidence bands computed computed using 1000 draws from the estimated asymptotic joint distribution of the parameters in (3). The vertical dashed line denotes the mandate announcement date. The number next to each province name indicates the mean estimated percentage point increase in first doses relative to the no-mandate counterfactual, as of October 31, 2021. Fig. 4 shows a relatively quick divergence and sizable gap between the observed and counterfactual cumulative first-dose numbers. The largest absolute gains in new vaccinations (left axis of Fig. 4 and column (1) in Table A7) are naturally in the four largest provinces (Ontario, Quebec, Alberta and British Columbia). As a proportion of the eligible population (right axis), however, we estimate the largest mandate increases in vaccine uptake to be in Saskatchewan (5 p.p., CI 4.0–5.6) and New Brunswick (5 p.p., 3.9–5.8), followed by Alberta (4.2 p.p., 3.2–4.9) and Nova Scotia (4.2 p.p., 3.6–4.7), see Fig. 5. These large gains in new first-dose vaccinations occur within 6 to 10 weeks of the mandate announcement for most provinces (13 for Quebec) and are on top of a very high base vaccination rates (above 80% of all eligible, on average), see Table C3. For example, Saskatchewan’s 5 p.p. estimated gain amounts to about one quarter of all remaining unvaccinated eligible people in the province as of the mandate announcement date (Sep. 16, 2021). ![Figure 5:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/01/13/2021.10.21.21265355/F5.medium.gif) [Figure 5:](http://medrxiv.org/content/early/2022/01/13/2021.10.21.21265355/F5) Figure 5: Counterfactuals – percentage-point increase in vaccine uptake Notes: The Figure plots the total estimated percentage-point increase (mean and 90% confidence interval) in the first dose vaccination rate after mandate announcement, as of October 31, 2021. We do not find evidence of net intertemporal substitution (decrease in the cumulative first-dose gains due to pulling vaccinations forward in time) as of October 31, 2021, except in PEI. However, the strongly negative *τ*2 estimates in Table 2 and the trends in Fig. B9 indicate that Alberta, New Brunswick and Saskatchewan are projected to exhibit net intertemporal substitution after the end of October 2021. That said, given the flattening of the counterfactual curves on Fig. 4, the cumulative gains are likely to remain significant. In Fig. B10 we also plot a counterfactual computed using the DID estimate for Canada.41 We estimate that, as of Sep. 14, 2021, the sample end date in Table 1, the proof-of-vaccination mandates in the five Canadian provinces that announced mandates by that date have led to 287,000 additional first doses (CI 239,000–333,000) or a vaccination rate increase of 0.9 p.p. (0.7–1.0) for the eligible population42, relative to the no-mandate counterfactual. ### 5.2 Countries We also use the time-series estimates in Table 2 to estimate the gains in vaccine uptake for France, Italy and Germany, relative to the counterfactual assuming the absence of a proof of vaccination mandate. The results are displayed in Fig. 5, Fig. 6 and Table A7. The estimated increases in first-dose uptake after the mandate announcements are large in all three countries: 8 p.p. (4.3–10.8) or 4.59 mln (2.47–6.25) doses in France, 12 p.p. (5.0–15.1) or 6.48 mln (2.67–8.14) additional first doses in Italy, and 4.7 p.p. (4.1–5.1) or 3.47 mln (3.06–3.81) doses in Germany. The larger percentage-point increase in Italy could be partly attributed to the complementary vaccination requirements for inter-regional travel and for employment introduced in September and October. ![Figure 6:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/01/13/2021.10.21.21265355/F6.medium.gif) [Figure 6:](http://medrxiv.org/content/early/2022/01/13/2021.10.21.21265355/F6) Figure 6: Countries – observed vs. no-mandate counterfactual first doses Notes: The figure plots the observed (black diamonds) and mean counterfactual (dotted red line) cumulative first doses for each country by date. The counterfactuals are computed using the estimates from Table 2. The shaded areas denote 5–95 percentile confidence bands computed using 1000 draws from the estimated asymptotic joint distribution of the parameters in (3). The vertical dashed line denotes the mandate announcement date. The number next to each country name indicates the mean estimated percentage point increase in first doses relative to the no-mandate counterfactual, as of October 31, 2021. ## 6 Discussion and conclusions We evaluate the impact of government-mandated proof of vaccination requirements on COVID-19 vaccine uptake. Using difference-in-differences analysis and the variation of timing of mandate announcements across Canadian provinces, we estimate a statistically significant and large (66% on average) increase in weekly first-dose uptake in Canada over the first weeks after the mandate announcement, relative to in absence of mandate. We obtain similar results from time-series analysis for each Canadian province and for France, Italy and Germany — a sizable increase in new first-dose vaccinations and lasting cumulative gains after a mandate announcement, relative to the pre-announcement trend. The estimated mandate effect varies across the Canadian provinces, with those exhibiting a larger initial gain tending to drop back toward the baseline pace of vaccination more quickly. Using simulations based on our time-series results, we estimate that, as of October 31, 2021, all ten provinces and all three countries achieved large net increases in the first-dose vaccination rate relative to the hypothetical scenario of absence of proof of vaccination mandate: 2.9 p.p. (1.3–3.8) of the eligible population in total for Canada, 8 p.p. (4.3–10.8) for France, 12 p.p. (5.0–15.1) for Italy and 4.7 p.p. (4.1–5.1) for Germany. These estimates may include effects from complementary measures (e.g., vaccination requirements for health workers or for inter-regional travel, etc.) adopted after the original proof of vaccination mandates, particularly in the European countries. In terms of external validity, in June 2020, Lazarus et al. (2021) conducted a survey on vaccine hesitancy across 19 countries comprising around 55% of the world population.43 Canada, Italy and Germany placed around the middle in the self-reported vaccine hesitancy rate (29% to 31%), while France had a higher hesitancy rate (41%). In this regard, our results on the large impact of proof of vaccination mandates on first-dose uptake in these four countries can be useful to public health authorities in other places looking for an effective strategy to increase vaccine uptake. Our objective is not to engage in normative or ethical debates related to vaccination mandates (e.g., Gostin, 2021), but to provide a benchmark range of estimates for their effectiveness in terms of new first dose uptake, which can be compared to other approaches such as monetary incentives or behavioural nudges. Monetary incentives for vaccination have been criticized by many for the ‘optics’ of putting a (low) dollar value on being vaccinated,44 because of the perceived unfairness in rewarding people who delayed their vaccination, or, if payments are unavailable to those that were vaccinated earlier, because of potential moral hazard problems (e.g., expecting future payments). Related to this, Chang et al. (2021) find that financial incentives may even have the perverse effect of validating vaccine concerns among unvaccinated individuals aged 40 or above and those that supported Donald Trump in the 2020 U.S. presidential election (see also Hsieh, 2021). Others have argued that, given the already high vaccination rates in developed countries, behavioural nudges may not be very effective (Thaler, 2021), which is consistent with the findings of Chang et al. (2021) and Campos-Mercade et al. (2021). Naturally, a full cost-benefit analysis of proof of vaccination mandates is beyond the scope of this paper. In particular, the costs of imposing and enforcing the mandates – economic, political, or personal – are very hard to estimate, as is the social value of vaccinating an additional person.45 Our results are a first step toward quantifying the benefits of proof of vaccination requirements. ## Data Availability All data used in this manuscript is based on information available in public domain. Data sources and links are provided within the manuscript. ## Author contributions A.K., D.K., S.L. and H.S. contributed equally to the conceptualization, writing and statistical analysis. ## Competing interests All authors declare no competing financial or other interest in the presented research. ## Data availability All data sources are described in Appendix C. The data used in the study are available at: [https://github.com/C19-SFU-Econ/dataV](https://github.com/C19-SFU-Econ/dataV). ## Acknowledgements We thank Serena Canaan, Hiro Kasahara and Pierre Mouganie for helpful suggestions. ## Appendix A. Additional Tables View this table: [Table A1:](http://medrxiv.org/content/early/2022/01/13/2021.10.21.21265355/T3) Table A1: Canadian provinces - different control groups View this table: [Table A2:](http://medrxiv.org/content/early/2022/01/13/2021.10.21.21265355/T4) Table A2: Robustness View this table: [Table A3:](http://medrxiv.org/content/early/2022/01/13/2021.10.21.21265355/T5) Table A3: Robustness – standard errors View this table: [Table A4:](http://medrxiv.org/content/early/2022/01/13/2021.10.21.21265355/T6) Table A4: Second doses View this table: [Table A5:](http://medrxiv.org/content/early/2022/01/13/2021.10.21.21265355/T7) Table A5: Test for structural break at the mandate announcement date View this table: [Table A6:](http://medrxiv.org/content/early/2022/01/13/2021.10.21.21265355/T8) Table A6: Time series estimates – alternative specifications View this table: [Table A7:](http://medrxiv.org/content/early/2022/01/13/2021.10.21.21265355/T9) Table A7: Counterfactuals – increase in first doses relative to no mandate as of Oct. 31, 2021 ## Appendix B. Additional Figures ![Figure B1:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/01/13/2021.10.21.21265355/F7.medium.gif) [Figure B1:](http://medrxiv.org/content/early/2022/01/13/2021.10.21.21265355/F7) Figure B1: Share of people with at least one dose – example countries Notes: The Figure plots the share of the population who have received at least one dose of a COVID-19 vaccine in selected countries. Source: Our World in Data. ![Figure B2:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/01/13/2021.10.21.21265355/F8.medium.gif) [Figure B2:](http://medrxiv.org/content/early/2022/01/13/2021.10.21.21265355/F8) Figure B2: France – first-dose vaccination appointments Notes: The figure plots the daily first-dose vaccination appointments by date made on Doctolib, a booking website accounting for about 2/3 of cumulative COVID-19 vaccinations in France as of Q4 2021. Source: doctolib.fr. The dashed red line denotes the mandate announcement date, July 12, 2021. ![Figure B3:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/01/13/2021.10.21.21265355/F9.medium.gif) [Figure B3:](http://medrxiv.org/content/early/2022/01/13/2021.10.21.21265355/F9) Figure B3: Canada – proof of vaccination mandates over time Notes: The figure plots the cumulative fraction of provinces and the cumulative fraction of Canada’s population for which a proof of vaccination mandate has been announced. See Table C1 for the exact dates of mandate announcement in each province. ![Figure B4:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/01/13/2021.10.21.21265355/F10.medium.gif) [Figure B4:](http://medrxiv.org/content/early/2022/01/13/2021.10.21.21265355/F10) Figure B4: Canadian provinces – first doses per 100,000 people Notes: The figure plots the weekly administered COVID-19 vaccine first doses per 100,000 people for dates *t −* 6 to *t*, where *t* is the date on the horizontal axis. The dashed red lines denote the proof of vaccination mandate announcement date for each province. The dotted blue lines denote the mandate implementation (enforcement) date for each province (see Table C1). ![Figure B5:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/01/13/2021.10.21.21265355/F11.medium.gif) [Figure B5:](http://medrxiv.org/content/early/2022/01/13/2021.10.21.21265355/F11) Figure B5: Robustness Notes: The upper left panel shows the estimates ![Graphic][28] and 95% confidence intervals for the mandate announcement variable *P**it* in equation (1) with different sample end dates and corresponding control groups, including the baseline (Sep. 14, in bold). The upper right panel shows the estimates for *P**it* with different initial sample dates (May 1 to July 15, 2021), including the baseline (June 15, 2021, in bold). The lower left panel shows the estimates from a variant of equation (1) when using no lag in the policy announcement *P**it* (in bold, our baseline) and using lag of up to 7 days, i.e., *P**it− k* for *k* = 1, 2, …, 7. The lower right panel displays the estimates from a variant of equation (1) when using no lag for the information (cases and deaths) *I**it* (in bold, our baseline) or using lag of up to 14 days, i.e., *I**it− k* for *k* = 2, …, 14. ![Figure B6:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/01/13/2021.10.21.21265355/F12.medium.gif) [Figure B6:](http://medrxiv.org/content/early/2022/01/13/2021.10.21.21265355/F12) Figure B6: Random assignment of announcement dates Notes: We estimate equation (1) 5,000 times using the Sun and Abraham (2021) treatment effect heterogeneity robust estimator after randomly assigning the date of mandate announcement for each province which has announced a mandate by Sep. 14. The figure plots the histogram of these placebo inference estimates, along with the 2.5-th and 97.5-th percentiles (dashed lines). The solid vertical red line corresponds to the baseline estimate from column (2) in Table 1. ![Figure B7:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/01/13/2021.10.21.21265355/F13.medium.gif) [Figure B7:](http://medrxiv.org/content/early/2022/01/13/2021.10.21.21265355/F13) Figure B7: Canadian provinces - second doses per 100,000 people Notes: The figure plots the weekly administered COVID-19 vaccine *second* doses per 100,000 people for dates *t−* 6 to *t*, where *t* is the date on the horizontal axis. The dashed red lines denote the vaccination proof mandate announcement date for each province. The dotted blue lines denote the mandate implementation (enforcement) date for each province (see Table C1). ![Figure B8:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/01/13/2021.10.21.21265355/F14.medium.gif) [Figure B8:](http://medrxiv.org/content/early/2022/01/13/2021.10.21.21265355/F14) Figure B8: Time-series policy estimates – correlations Notes: The figure plots the time-series policy estimates, ![Graphic][29] from Table 2 column (1) against the number of days between mandate announcement and implementation (left) and the percent remaining unvaccinated eligible people at the announcement date (right). The figure is for illustration; no causal claims are made. ![Figure B9:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/01/13/2021.10.21.21265355/F15.medium.gif) [Figure B9:](http://medrxiv.org/content/early/2022/01/13/2021.10.21.21265355/F15) Figure B9: Canadian provinces - observed vs. no-mandate counterfactual weekly first doses as of October 31, 2021 (time-series estimates) Notes: The figure plots the observed (black diamonds) and the expected no-mandate counterfactual (dashed red lines) log weekly first doses per 100,000 people. We use the estimates from Table 2 to compute the counterfactuals, as specified in (3). The dashed blue vertical lines denote the mandate announcement date for each province. ![Figure B10:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/01/13/2021.10.21.21265355/F16.medium.gif) [Figure B10:](http://medrxiv.org/content/early/2022/01/13/2021.10.21.21265355/F16) Figure B10: Canada – observed vs. no-mandate counterfactual first doses as of September 14, 2021 (DID estimate) Notes: The figure plots the observed (black diamonds) and the mean no-mandate counterfactual (dotted red line) cumulative first doses (in millions) by date, with 5–95 percentile confidence bands (the shaded area). The counterfactual uses the *P**it* coefficient estimate from column (2) of Table 1. The number in the caption indicates the percentage point increase in first doses relative to the no-mandate counterfactual as of September 14, 2021. ## Appendix C. Supplementary Information View this table: [Table C1:](http://medrxiv.org/content/early/2022/01/13/2021.10.21.21265355/T10) Table C1: Proof of vaccination mandate announcement and implementation dates View this table: [Table C2:](http://medrxiv.org/content/early/2022/01/13/2021.10.21.21265355/T11) Table C2: Canada – age 12+ eligibility dates for first vaccine dose View this table: [Table C3:](http://medrxiv.org/content/early/2022/01/13/2021.10.21.21265355/T12) Table C3: Vaccine eligible population (age 12 or older) View this table: [Table C4:](http://medrxiv.org/content/early/2022/01/13/2021.10.21.21265355/T13) Table C4: Data sources View this table: [Table C5:](http://medrxiv.org/content/early/2022/01/13/2021.10.21.21265355/T14) Table C5: Proof of vaccination mandates scope ## Footnotes * Minor revision improving the presentation of some results. * 1 Scobie et al. (2021), Grannis et al. (2021), Andrews et al. (2021) and Gupta et al. (2021). * 2 For example, by early August, all four of Canada’s Western provinces had removed their mask-wearing mandates for public indoor spaces. By mid-September, all re-imposed a mask mandate, either preemptively or after a COVID surge. The largest provinces Ontario and Quebec did not remove their mask mandates. * 3 While the mandates are sometimes colloquially called “vaccine passports”, we deliberately avoid this politicized term. * 4 Among the jurisdictions considered in this study, the mandates in France, Italy, Germany, Alberta and Saskatchewan allowed such options during the analyzed period – see the notes to Table C5 for details. * 5 Several countries have also planned or implemented vaccination mandates *for employees* (USA for all firms with more than 100 employees, Italy for all employees, Canada for all federal employees), and a broader mandate is under consideration in Austria. We do not analyze these alternative mandates. * 6 To avoid potential bias from constrained vaccine supply affecting the pre- or post-mandate pace of vaccination, we use only data after June 15, 2021, when it is safe to assume that any age-eligible person (12 or older in Canada) was able to receive a first dose of COVID-19 vaccine with minimal delay; see Table C2. * 7 In Canada, the provinces are separate public health jurisdictions with extensive powers over health policy while the vaccines are procured by the federal government and allocated to the provinces in proportion to their population. All COVID-19 vaccines used in Canada during the study period, namely BNT162b2 (Pfizer /BioNTech Comirnaty), mRNA-1273 (Moderna SpikeVax) and AZD1222 (Oxford–AstraZeneca Vaxzevria), were originally considered two-dose vaccines. * 8 Throughout the paper, we report 95% confidence intervals for parameter estimates and 90% confidence intervals for counterfactual simulation results. * 9 A comparison of Fig. 1 and Fig. B2 suggests that the relatively low French estimate for the initial policy effect may reflect a lag between appointment booking and vaccine administration. * 10 Mills and Rüttenauer estimate about 4 mln additional vaccinations in Italy and about 8.6 mln in France for the period 20 days before to 40 days after mandate implementation. * 11 When using total vaccinations instead of first doses, we also do not find a statistically significant effect for Germany. * 12 The interval between mandate announcement and implementation in the ten Canadian provinces and three countries in our sample varies from 5 to 41 days. * 13 In an RCT study in Sweden, Campos-Mercade et al. (2021) find that a modest payment of SEK 200 (USD 24) is associated with a 4.2 p.p. increase in vaccinations, relative to a baseline rate of 71.6% in the control group, while none of three behavioral nudges had impact. However, the RCT subjects were *yet to become eligible* to be vaccinated, and hence the setting is not comparable to ours in which weeks have elapsed after widespread vaccine availability. * 14 See Table C4 for the data sources and details. We collected the Canadian data from the official provincial dashboards or equivalent sources. We use the Our World In Data dataset for the country data. * 15 Some locations, e.g., British Columbia and Nova Scotia, do not report vaccination data on weekends and then report the total for several days at once (e.g., Monday’s number contains 3 days of data). In these cases, we distribute the reported total equally over the affected days. * 16 In Table A4, we also report results using second doses as the outcome. We find no statistically significant effect on second-dose uptake following the announcement of mandate. * 17 We use Sep. 17, 2021 as the mandate announcement date for Newfoundland and Labrador, the date on which the premier (provincial head of government) announced the upcoming mandate. The mandate details were officially unveiled on Oct. 7 after consultations with the business and other sectors. However, using the latter date would be problematic because of a potentially strong anticipation effect following the first announcement. Our main results in Table 1 and Fig. 3 are not affected by this choice. Throughout the paper, “Newfoundland” refers to the province of Newfoundland and Labrador. * 18 Another possible information variable is hospitalizations. However, it is very strongly correlated with the COVID-19 cases and deaths. To avoid multicollinearity, we do not include it in our baseline specification. See Table A2 for results using deaths and hospitalizations as information instead of deaths and cases. * 19 To tackle zero weekly values, which sometimes occur in the smaller provinces for deaths or cases (4.4% of all observations for cases and 10.7% for deaths), we replace log(0) with -1 as in Chernozhukov et al. (2021). * 20 In Canada, the provinces opened registration for first-dose vaccination for any person of age 12+ between May 10, 2021 in Alberta and May 27, 2021 in Nova Scotia, see Table C2. First-dose availability in France, Italy and Germany was similar by mid-June, at least for the 18+ age group. * 21 British Columbia, Ontario, Nova Scotia, Saskatchewan and Newfoundland exhibit a second peak later. * 22 For our main model (difference-in-differences), we perform sensitivity analysis using alternative lags of up to 7 days (see Section 3.3 and Fig. B5); the results affirm our baseline choice of zero lag. * 23 We use the Stata function *boottest* (see Roodman et al., 2019) clustered by province with 4,999 repetitions. Alternative ways of computing the standard errors are displayed in Table A3. * 24 See also Athey and Imbens (2018), de Chaisemartin and D’Haultfoeuille (2018), Callaway and Sant’Anna (2021) and Sun and Abraham (2021). * 25 We use the Stata function *eventstudyinteract* provided by the authors. * 26 A coefficient estimate *x* is equivalent to 100*x* log points or a 100(*e**x**−* 1) percent increase in weekly doses. The smaller *T* = 0 estimate reflects the fact that the outcome variable is a backward-looking weekly sum. * 27 In Table A3 we also report alternative standard error specifications: regular clustering at the province level (Stata command “cluster”), wild bootstrap standard errors clustered at the province level (Stata command “boottest”) and wild bootstrap standard errors two-way clustered by province and date. * 28 We also checked a specification in which we lag the information variables *I**it* (log weekly cases and deaths) with the same lag as the policy *P**it*, and the results are nearly identical. * 29 The mandates were implemented in all provinces between Sep. 3 and Oct. 22, 2021. * 30 We omit the subscripts *i* for simplicity since all variables refer to the same location. * 31 Using a weekly average of the mandate indicator is consistent with the weekly vaccination and information variables *V**t* and *I**t* and improves the fit in the time-series regressions. We also present results without weekly averaging in Tables A6 and A7. * 32 The results are also robust to picking June 15, 2021 as the initial date for the test. * 33 As shown in Giles and Scott (1992), the size of the Chow test is distorted upward (over-rejection of the null) in the presence of positive serial correlation in the error terms. The size distortion is mitigated if the sample split is unequal before vs. after the break point. * 34 The power of the Chow test is weaker in this specification since the first-differenced *V**t* series is a growth rate being used to test for a level shift. The differenced series is also noisier as it captures daily fluctuations. * 35 The estimates’ p-values are calculated using the Newey-West heteroskedasticity and autocorrelation (HAC) robust estimator (Newey and West, 1987) with 3 lags. The lag length was chosen as the closest integer to *T* 1*/*4, where *T* is the sample size. * 36 The ADF test results in column (4) use 13 lags, chosen as the integer part of 12(*T/*100)1*/*4, see Schwert (1989). For fewer lags the test tends to yield smaller p-values. * 37 France’s estimate is sensitive to including log weekly cases. We obtain a larger estimate, ![Graphic][30] increase, using deaths and hospitalizations as information variables, see Table A6. * 38 The estimate for Quebec is 0.389 (0.265–0.514) and statistically significantly different from zero when we use deaths and hospitalizations as information *I**t*; see Table A6. * 39 We note that with hospitalizations replacing cases as control variable (see column (2) of Table A6), ![Graphic][31] is negative, albeit sometimes without statistical significance, for all provinces but Manitoba and Nova Scotia. Estimates for the latter two provinces show very negative ![Graphic][32], possibly due to the abrupt decrease in vaccinations about four weeks before their mandate announcements (see Fig. B4). Thus, we cannot rule out that a negative change in trend may be quite general for Canadian provinces. * 40 The policy effect estimates are positive but not statistically significantly different from zero in one specification in column (1) and 3 specifications in column (3) of Table A6, out of 26 total specifications. * 41 We compute the counterfactual mean as ![Graphic][33] where ![Graphic][34] is the DID estimate on the policy variable *P**it*, 0.506 in Table 1, column (2). * 42 This number equals approximately 4.4% of all unvaccinated eligible people of age 12 or older at the time of the first proof-of-vaccination mandate announcement in Canada (August 5, 2021 in Quebec). * 43 The participants were asked: “If a COVID-19 vaccine is proven safe and effective and is available, will you take it?” * 44 For example, in July 2021 U.S. President Joseph Biden advocated that state and local governments pay $100 to anyone willing to receive a first dose, however, the public health value of vaccination is likely much higher (see Castillo et al., 2021). * 45 One component of the social value of vaccination is avoided healthcare costs. Barber and West (2021) estimate that the Ohio vaccine lottery program saved the state USD 66 million in intensive care unit costs. * Received October 21, 2021. * Revision received January 12, 2022. * Accepted January 13, 2022. * © 2022, Posted by Cold Spring Harbor Laboratory The copyright holder for this pre-print is the author. All rights reserved. The material may not be redistributed, re-used or adapted without the author's permission. ## References 1. [1].Alsan, M. and S. Eichmeyer (2021), “Experimental Evidence on the Effectiveness of Non-Experts for Improving Vaccine Demand”, NBER Working Paper No. 28593. 2. [2].Andrews, N., E. Tessier, J. Stowe, C. Gower, F. Kirsebom, R. Simmons, E. Gallagher, M. Chand, K. Brown, S. Ladhani, M. Ramsay and J. Bernal (2021), “Vaccine effective-ness and duration of protection of Comirnaty, Vaxzevria and Spikevax against mild and severe COVID-19 in the UK”, medRxiv preprint. 3. [3].Athey, S. and G. Imbens (2018), “Design-based Analysis in Difference-In-Differences Settings with Staggered Adoption”, NBER Working Paper No. 24963. 4. [4].Barber, A. and J. West (2021), “Conditional Cash Lotteries Increase COVID-19 Vaccination Rates”, working paper, UC Santa Cruz. 5. [5].Brehm, M., P. Brehm and M. Saavedra (2021), “The Ohio Vaccine Lottery and Starting Vaccination Rates”, working paper, Oberlin College. 6. [6].Campos-Mercade, P., A. Meier, F. Schneider, S. Meier, D. Pope, and E. Wengström (2021), “Monetary incentives increase COVID-19 vaccinations”, Science eabm0475. 7. [7].Callaway, B. and P. Sant’Anna (2021), “Difference-in-Differences with Multiple Time Periods.” Journal of Econometrics Forthcoming. 8. [8].Cameron, A., J. Gelbach, and D. Miller (2008), “Bootstrap-based improvements for inference with clustered errors”, Review of Economics and Statistics 90: 414–427. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1162/rest.90.3.414&link_type=DOI) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000258260500003&link_type=ISI) 9. [9].Castillo, J., A. Ahuja, S. Athey, A. Baker, E. Budish, T. Chipty, R. Glennerster, S. Kominers, M. Kremer, G. Larson, J. Lee, C. Prendergast, C. Snyder, A. Tabarrok, B. Tan and W. Wiecek (2021), “Market design to accelerate COVID-19 vaccine supply”, Science 371: 1107–1109. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6Mzoic2NpIjtzOjU6InJlc2lkIjtzOjEzOiIzNzEvNjUzNC8xMTA3IjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjIvMDEvMTMvMjAyMS4xMC4yMS4yMTI2NTM1NS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 10. [10].Chang, T., M. Jacobson, M. Shah, R. Pramanik, and S. Shah (2021), “Financial Incentives and Other Nudges Do Not Increase COVID-19 Vaccinations among the Vaccine Hesitant”, NBER Working Paper 29403. 11. [11].Chernozhukov, V., H. Kasahara, and P. Schrimpf (2021), “Causal Impact of Masks, Policies, Behavior on Early COVID-19 Pandemic in the U.S.”, Journal of Econometrics 220(1): 23–62. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.jeconom.2020.09.003&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=33100476&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F01%2F13%2F2021.10.21.21265355.atom) 12. [12].Chow, G. (1960), “Tests of equality between sets of coefficients in two linear regressions”, Econometrica 591–605. 13. [13].de Chaisemartin, C. and X. D’Haultfœuille (2020), “Two-Way Fixed Effects Estimators with Heterogeneous Treatment Effects”, American Economic Review 110(9): 2964–2996. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1257/aer.20181169&link_type=DOI) 14. [14].Dai, H., S. Saccardo, M. Han, L. Roh, N. Raja, S. Vangala, H. Modi, S. Pandya, M. Sloyan, and D. Croymans (2021), “Behavioural nudges increase COVID-19 vaccinations”, Nature 597: 404–409. [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F01%2F13%2F2021.10.21.21265355.atom) 15. [15].Fisher, R. (1935), The Design of Experiments, Oliver and Boyd, Edinburgh. 16. [16].Giles, D. and M. Scott (1992), “Some consequences of using the Chow test in the context of autocorrelated disturbances”, Economics Letters 38(2): 145–150. 17. [17].Goodman-Bacon, A. (2021), “Difference-in-Differences with Variation in Treatment Timing”, Journal of Econometrics Forthcoming. 18. [18].Gostin L. (2021) “COVID-19 Vaccine Mandates – A Wider Freedom”, JAMA Health Forum 2(10):e213852. 19. [19].Grannis, S., E. Rowley, T. Ong, E. Stenehjem, N. Klein, M. DeSilva, A. Naleway, K. Natarajan and M. Thompson (2021), “Interim Estimates of COVID-19 Vaccine Effectiveness against COVID-19-Associated Emergency Department or Urgent Care Clinic Encounters and Hospitalizations Among Adults During SARS-CoV-2 B.1.617.2 (Delta) Variant Predominance – Nine States, June–August 2021”, Morbidity and Mortality Weekly Report 70(37): 1291–1293. 20. [20].Gupta, S., J. Cantor, K. Simon, A. Bento, C. Wing and C. Whaley (2021), “Vaccinations Against COVID-19 May Have Averted Up To 140,000 Deaths In The United States”, Health Affairs 40(9). 21. [21].Hsieh, P. (2021), “Perks and incentives for COVID-19 Vaccination May Backfire”, Forbes, May 28, 2021. 22. [22].Karaivanov, A., S. Lu, H. Shigeoka, C. Chen and S. Pamplona (2021), “Face Masks, Public Policies and Slowing the Spread of COVID-19: Evidence from Canada”, Journal of Health Economics 78, 102475. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/J.JHEALECO.2021.102475&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F01%2F13%2F2021.10.21.21265355.atom) 23. [23].Klüver, H., F. Hartmann, M. Humphreys, F. Geissler and J. Giesecke (2021), “Incentives can spur COVID-19 vaccination uptake”, Proceedings of the National Academy of Sciences 118(36). 24. [24].Lazarus, J., S. Ratzan, A. Palayew, L. Gostin, H. Larson, K. Rabin, S. Kimball and A. El-Mohandes (2021), “A global survey of potential acceptance of a COVID-19 vaccine”, Nature Medicine 27: 225–228. [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F01%2F13%2F2021.10.21.21265355.atom) 25. [25].Linden, A. (2017), “Conducting interrupted time-series analysis for single-and multiple-group comparisons”, The Stata Journal 15(2): 480–500. 26. [26].Mills, M. and T. Rüttenauer (2021), “The effect of mandatory COVID-19 certificates on vaccine uptake: synthetic-control modelling of six countries”, Lancet Public Health published online Dec. 13, 2021, [https://doi.org/10.1016/S2468-2667(21)00273-5](https://doi.org/10.1016/S2468-2667(21)00273-5). 27. [27].Newey, W., and K. West (1987), “A Simple, Positive Semi-definite, Heteroskedasticity and Autocorrelation Consistent Covariance Matrix”, Econometrica 55(3): 703–708. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.2307/1913610&link_type=DOI) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1987H597700014&link_type=ISI) 28. [28].Roodman D, M. Nielsen, J. MacKinnon and M. Webb (2019), “Fast and wild: Boot-strap inference in Stata using boottest”, The Stata Journal 19(1):4–60. 29. [29].Scobie, H., A. Johnson, A. Suthar, R. Severson, N. Alden, S. Balter, D. Bertolino, D. Blythe, S. Brady, B. Cadwell, I. Cheng, S. Davidson, J. Delgadillo, K. Devinney, J. Duchin, M. Duwell, R. Fisher, A. Fleischauer, A. Grant, J. Griffin, M. Haddix, J. Hand, M. Hanson, E. Hawkins, R. Herlihy, L. Hicks, C. Holtzman, M. Hoskins, J. Hyun, R. Kaur, M. Kay, H. Kidrowski, C. Kum, K. Komatsu, K. Kugeler, M. Lewis, B.C. Lyons, S. Lyons, R. Lynfield, K. McCaffrey, C. McMullen, L. Milroy, S. Meyer, L. Nolen, M. Patel, S. Pogosjans, H. Reese, A. Saupe, J. Sell, T. Sokol, D. Sosin, E. Stanislawski, K. Stevens, H. Vest, K. White, E. Wilson, A. MacNeil, M. Ritchey and B. Silk (2021), “Monitoring Incidence of COVID-19 Cases, Hospitalizations, and Deaths, by Vaccination Status – 13 U.S. Jurisdictions, April 4–July 17, 2021”, CDC Morbidity and Mortality Weekly Report 70(37): 1284–1290. 30. [30].Sehgal, K. (2021), “Impact of Vax-a-Million lottery on COVID-19 vaccination rates in Ohio”, The American Journal of Medicine Forthcoming. 31. [31].Sun, B. and S. Abraham (2021), “Estimating dynamic treatment effects in event studies with heterogeneous treatment effects.” Journal of Econometrics Forthcoming. 32. [32].Schwert, G. (1989), “Tests for unit roots: A Monte Carlo investigation.” Journal of Business and Economic Statistics, Vol. 7, No. 2. 33. [33].Thaler, R. (2021), “More than nudges are needed to end the pandemic”, The New York Times, August 5, 2021. 34. [34].Walkey, A., A. Law, N. Bosch (2021), “Lottery-Based Incentive in Ohio and COVID-19 Vaccination Rates”, JAMA 326(8): 766–767. [1]: /embed/inline-graphic-1.gif [2]: /embed/inline-graphic-2.gif [3]: /embed/inline-graphic-3.gif [4]: /embed/graphic-3.gif [5]: /embed/inline-graphic-4.gif [6]: /embed/inline-graphic-5.gif [7]: /embed/graphic-6.gif [8]: /embed/inline-graphic-6.gif [9]: /embed/inline-graphic-7.gif [10]: /embed/inline-graphic-8.gif [11]: /embed/inline-graphic-9.gif [12]: /embed/inline-graphic-10.gif [13]: /embed/inline-graphic-11.gif [14]: /embed/inline-graphic-12.gif [15]: /embed/inline-graphic-13.gif [16]: /embed/inline-graphic-14.gif [17]: /embed/inline-graphic-15.gif [18]: /embed/graphic-8.gif [19]: /embed/inline-graphic-16.gif [20]: /embed/inline-graphic-17.gif [21]: /embed/inline-graphic-18.gif [22]: /embed/inline-graphic-19.gif [23]: /embed/inline-graphic-20.gif [24]: /embed/inline-graphic-21.gif [25]: /embed/inline-graphic-22.gif [26]: /embed/inline-graphic-23.gif [27]: /embed/inline-graphic-24.gif [28]: F11/embed/inline-graphic-25.gif [29]: F14/embed/inline-graphic-26.gif [30]: /embed/inline-graphic-27.gif [31]: /embed/inline-graphic-28.gif [32]: /embed/inline-graphic-29.gif [33]: /embed/inline-graphic-30.gif [34]: /embed/inline-graphic-31.gif