Effectiveness of vaccination against symptomatic and asymptomatic SARS-CoV-2 infection: a systematic review and meta-analysis
=============================================================================================================================

* A. Meggiolaro
* M. Sane Schepisi
* G. Nikolaidis
* D. Mipatrini
* A. Siddu
* G. Rezza

## ABSTRACT

**OBJECTIVE** To assess the effectiveness of SARS-CoV-2 vaccines in terms of prevention of disease and transmission. The evaluation was narrowed to two mRNA vaccines and two modified adenovirus vectored vaccines.

**METHODS** A frequentist random effects meta-analysis was carried out after data extraction. Risk of bias of the included studies was assessed using New-Castle-Ottawa Scale. The overall risk of SARS-CoV-2 infection confirmed by real time Polymerase Chain Reaction (PCR) was estimated in partially and fully vaccinated individuals. The effect size was expressed as Relative Risk (RR) and RRR (RR reduction) of SARS-CoV-2 infection after vaccination. Potential sources of heterogeneity were investigated through between-study heterogeneity analysis and subgroup meta-analysis.

**RESULTS** The systematic review identified 27 studies eligible for the quantitative synthesis. Partially vaccinated individuals presented a RRR=73% (95%CI=59%-83%) for any positive SARS-CoV-2 PCR (RR=0.27) and a RRR=79% (95%CI=30%-93%) for symptomatic SARS-CoV-2 PCR (RR=0.21). Fully vaccinated individuals showed a RRR=94% (95%CI=88%-98%) for any SARS-CoV-2 positive PCR (RR=0.06) compared to unvaccinated. According to the subgroup meta-analysis, full BNT162b2 vaccination protocol achieved a RRR=84%-94% against any SARS-CoV-2 positive PCR and a RRR=68%-84% against symptomatic positive PCR. The RR for any SARS-CoV-2 positive PCR remained higher within elderly groups aged ≥69 years (RR=0.12-0.15) compared to younger individuals (RR=0.05-0.12). The RR against B.1.351 infection approached 0.40 for any positive PCR and 0.36 for symptomatic SARS-COV-2 while the RR of any B.1.1.7 infection was 0.14.

**CONCLUSION** The current licensed vaccines may be transmission blocking, especially after full vaccination protocol. Given the substantial heterogeneity, results should be interpreted with caution. Subgroups meta-analyses suggested that the risk of any SARS-CoV-2 infection may be higher for non-B.1.1.7 variants and individuals aged ≥69 years. Further data and longer follow-up are required to investigate additional sources of heterogeneity and the effectiveness of SARS-CoV-2 vaccination within population subgroups.

**STRENGTHS**

*   Real-world data suggest that the current licensed vaccines may be transmission blocking, in particular after full vaccination protocol.

*   The risk of any SARS-CoV-2 infection either symptomatic or asymptomatic, may be higher for non-B.1.1.7 variants and individuals aged ≥69 years.

**LIMITATIONS**

*   Given the substantial heterogeneity encountered in this meta-analysis, results should be interpreted with caution

*   Fur ther evidence on the impact of SARS-CoV-2 variants are vital in order to monitor mutations associated with vaccine escape

## INTRODUCTION

Since December 2020, infections with SARS-CoV-2 and the respective disease, COVID-19 (Coronavirus Disease 2019), have spread worldwide. On 11 March 2020, WHO declared the COVID-19 outbreak a pandemic. The Coronavirus disease 2019 (COVID-19) has represented a serious threat to public health reporting nearly 4,038,342 deaths and 186,821,815 confirmed cases globally. As of 5 July 2021, 3,114,766,865 vaccine doses have been administered.[1]

Coronaviruses (CoVs), including SARS-CoV, MERS-CoV, and SARSCoV-2, are positive-sense, single stranded RNA viruses with four structural proteins (S protein, envelope (E) protein, membrane (M) protein, and nucleocapsid (N) protein).[2] The spike protein of SARS-CoV-2 binds to ACE2 receptors on target cells and acts as immunodominant antigen, eliciting both antibody and T-cell responses.[3]

Most COVID-19 candidate vaccines have been developed including mRNA, adenoviral vectored vaccines, inactivated virus, and adjuvanted.[4] During the COVID-19 pandemic, the European medicines Agency (EMA) has applied a conditional marketing authorisation for the fast-track approval of safe and effective COVID-19 vaccines in the EU. Initially, EMA has recommended three COVID-19 vaccines to prevent COVID-19: Comirnaty (BNT162b2), Vaxzevria (ChAdOx1/AZD1222), Moderna (mRNA-1273). On March 2021, EMA has further expanded the arsenal of COVID-19 vaccines available to European member states with the COVID-19 vaccine Janssen (Ad26.COV2.S). Each licensed vaccine should meet rigorous standards for safety, efficacy and quality, hence, despite the earlier market authorization, a comprehensive monitoring process is generated post-approval.[5]

The vaccines currently authorised in the EU have proven high efficacy in Phase III clinical trial – such as Moderna’s COVID-19 vaccine with 94.5% efficacy and Pfizer’s with 95% efficacy – and likewise, they are expected to be highly effective in the real world. The effort to measure vaccine effectiveness is crucial to encouraging the vaccine uptake and to improving vaccine technology.[6, 7] While efficacy estimate, in terms of degree to which a vaccine prevents disease and/or transmission, is measured under ideal circumstances such as Controlled Randomized Trials (RCTs), effectiveness refers to an estimate of the vaccine performance in the real world. Effectiveness is generally measured through observational studies where participants are not randomly assigned to an intervention versus a control group.[8]

The COVID-19 vaccination campaign has offered different options and the vaccines’ effect is not necessarily equal.[9-12] The controversy on whether the vaccination is sufficient to prevent any SARS-CoV-2 infection or it merely converts infections from symptomatic to asymptomatic has remained unsolved. Asymptomatic infection following vaccination could still spread the virus and nosocomial COVID-19 infections have been seemingly persistent within health care workers (HCWs), despite the vaccination.[13]

The main objective of this study was to estimate the effectiveness of conditional market licensed COVID-19 vaccines in terms of prevention of disease and transmission. The vaccine effectiveness (VE) was measured as relative risk (RR) and relative risk reduction (RRR) of any SARS-CoV-2 infection, confirmed by polymerase chain reaction (PCR) test, on partially and fully vaccinated individuals respect to unvaccinated individuals. The meta-analysis estimated the overall RR of asymptomatic and symptomatic SARS-CoV-2 infections after partial and full vaccination protocol. The evaluation was narrowed to two mRNA vaccines (Comirnaty and Moderna), a modified adenovirus vaccine (COVID-19 Vaccine AstraZeneca or Vaxzevria) and a recombinant human adenovirus vector (Ad26.COV2.S or Janssen or J&J).

Worldwide mass vaccination requires evidence synthesis on SARS-CoV-2 VE in order to drive health policy strategies. In particular, VE findings may address policy makers’ choices over mitigation measures following mass vaccination campaign, vaccine supply and deployment, vaccination protocol implementation.

## METHODS

### Protocol registration

The systematic review was prospectively registered on PROSPERO (n. CRD42021240143).

### Systematic Review

The systematic review started on March 2021 and was concluded on May 15 2021. The main purpose of this research was to assess the risk of COVID-19 occurrence (any positive RT-PCR test) among vaccinated subjects. Moreover, we aimed at quantifying the risk of developing a symptomatic COVID-19 after vaccination. The evaluation was narrowed to BNT162b2, mRNA-1273, ChAdOx1/AZD1222 and Ad26.COV2.S vaccines. This research followed the PRISMA 2020 statement.[14]

The search strategy was presented in details in Annex 1. We used seven different web engines, including early stage research platforms: Pubmed, Cochrane, clinicaltrial.gov, and COVID-NMA, medRxiv, SSRN, and Authorea. No restriction on language, setting or publication date was imposed.

The papers were selected initially by titles and abstracts. The full texts suitable for the quantitative synthesis were collected in an excel database for the data extraction. The exclusion criteria included lack of suitable data, and study design. We narrowed the quantitative synthesis to non-experimental studies only.

### Data extraction

The data extraction was performed by two authors, independently. The results of the respective phase III RCTs were used as reference for data extraction.[6, 7, 15, 16] The information derived from each full text was classified into four categories of information: 1) outcome, 2) study characteristics (design, publication status, year of publication), 3) participants characteristics (mean age, severity of COVID-19 symptoms, dose of SARS-CoV-2 vaccine, SARS-CoV-2 lineage) and 4) risk of bias. Disagreements were resolved by discussion or, if necessary, consultation with a third party.

#### Outcome

The primary endpoint of this reasearch was to measure the overall relative risk (RR) of any SARS-CoV-2 infection 14 days after the first dose (partially vaccinated), seven days after the second dose (fully vaccinated) and 14 days after at least one dose uptake.

As secondary endpoint, we aimed at measuring the risk of symptomatic infection following SARS-CoV-2 vaccination, and, if possible, quantifying the risk of hospitalization and death after partial and full vaccination protocol.

To this purpose, any COVID-19 infection, either symptomatic or asymptomatic, confirmed through PCR was considered as a COVID-19 case. Studies which did not declare to routinely perform a PCR test after vaccination were excluded. Documented SARS-CoV-2 infections (any positive rtPCR) at the baseline were excluded from the effectiveness analysis.

The risk of infection between vaccinated and unvaccinated groups was calculated as relative risk (RR). In order to quantify how much SARS-CoV-2 vaccination reduced the risk of infection relative to the control group, we computed the relative risk reduction (RRR), according to the formula 100x(1-⍰RR).

#### SARS-CoV-2 infection episodes

The primary analysis included the RR of any new positive infection episode among vaccinated and unvaccinated groups confirmed through PCR test. No restriction was imposed to PCR cycle threshold. The impact of SARS-CoV-2 vaccination on infection severity was investigated in secondary analysis as RR of symptomatic positive PCR, and possibly, RR of hospitalization and death following partial and full vaccination protocol. Both self-reported symptoms and symptoms ascertained through clinical visit were included.

Finally, COVID-19 episodes were classified as compatible with the B.1.1.7 variant and incompatible (i.e. B.135, non-B.1.1.7, unspecified).

#### Experimental group

All adults eligible to undergo SARS-CoV-2 mass vaccination were included in the experimental group. BNT162b2 has been the first vaccine authorized for the emergency rollout, hence, more evidence on effectiveness was expected in the scientific literature. For the definition of case in the vaccinated groups we referred to Polack et al. Phase III RCT.[6] The same rules were applied to ChAdOx1/AZD1222 and mRNA-1273. Concerning ChAdOx1/AZD1222, only few studies on VE after the first dose administration were available, as the vaccine was approved later for the emergency rollout.[15] Ad26.COV2.S single dose administration was considered as full vaccination protocol, although the induction time lasts two weeks (Fig. 1).[16, 47]

![Figure 1)](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/08/28/2021.08.25.21262529/F1.medium.gif)

[Figure 1)](http://medrxiv.org/content/early/2021/08/28/2021.08.25.21262529/F1)

Figure 1) 
Experimental group follow-up: the dashed line represents the *induction time* for the first and the second vaccine doses respectively. The solid line underpins the follow-up period.

For each vaccine, the 14 days ensuing the first jab were considered as induction time for building COVID-19 immunogenity, therefore, SARS-CoV-2 cases occurred within two weeks from the first dose were not included.[6, 17] In order to assess the effectiveness of full vaccination protocol, the upper follow-up limit was set at seven days after the second dose. (Fig. 1) Only SARS-CoV-2 cases occurred at least one week after the second dose were attributed to the lack of vaccine protection.[6] Moreover, cases occurred within these seven days were still attributed to the first dose effectiveness (Fig. 1).

VE against any positive SARS-COV-2 on at least one dose, was tested after 14 days from the first dose administration.

#### Control group

In the general population, the mean incubation period for COVID-19 symptoms onset was estimated as 5.8 days.[18] Insofar, vaccination has been prioritized towards older and at-risk individuals and this aspect might bias the comparison with the unvaccinated cohort, which generally was younger and healthier. Overall, cohort studies drew the control group from the same population as the vaccinated group, while in few cases the cohort was the same community observed before-after the vaccination. For reasons mostly related to dearth of detailed information on unvaccinated individuals, the beginning of the follow-up for the control group was established at T0 (Fig. 1).

#### Risk of bias

In order to assess the risk of bias, we employed the Newcastle-Ottawa Scale (NOS) in cohort and case-control studies, and the NOS adapted for cross-sectional studies. For systematic review and meta-analysis, we used AMSTAR 2 checklist.[49]

### Statistical analysis

We performed a frequentist meta-analysis using the inverse variance (IV) method.[19] In addition, the Mantel-Haenszel (MH) method was applied because we expected to encounter sparse data, especially in estimating the RR of symptomatic cases after vaccination. The analysis was executed on R (version 4.0.5).

In the early phases of the mass vaccination campaign we expected the selected studies to be statistically heterogeneous. The main assumption underpinning the fixed effect model (FE) is that the true effect size is the same in all studies. This assumption was implausible in this systematic review, as the SARS-CoV-2 vaccination campaign has targeted priority population subgroups in the early stages. Therefore, the random effect (RE) meta-analysis was considered more appropriate to estimate the overall effect size. However, both FE and RE outputs were reported. In order to test for the overall heterogeneity both the Cochran’s Q (chi-squared statistic) and I-squared values (I2) were calculated. The Der Simonian and Laird method estimated the between studies variance τ2.[20]

As the between-study heterogeneity can be caused by studies with extreme effect size, low quality or small sample size, the outliers test was implemented in order to detect which studies did not fit in the meta-analysis. Moreover, we explored the robustness of our meta-analyses results using influence analyses.[21] The Baujat Plot displayed which studies overly contributed to the heterogeneity,[22] while the leave-one-out analysis recomputed the pooled effect size and the I2 with one study omitted at each step (Annex 3). The Graphic Display of Heterogeneity (GOSH) with Knapp-Hartung adjustment fits the meta-analysis model over all possible subsets of the included studies.[23] Three clustering algorithms detect potential patterns in the data distribution. GOSH analysis allows the identificantion of the studies that contribute to the overall heterogeneity and, possibly, to derive subgroup analysis therefrom (Annex 3).[20]

The subgroups meta-analysis was performed to identify further heterogeneity sources.[24] As we expected a limited number of studies in some subgroups (n<10), we opted for a FE model in subgroup meta-analysis. The subgroup levels were considered fixed and exhaustive. Nevertheless, RE and mixed model were added to the output, in order to catch discrepancies that might yield to inconclusive results (reported in Annex 4.1). In mixed model (or fixed plural model), subgroups are assumed to share a common estimate of the between-study heterogeneity. However, the general approach was to evaluate a separate τ2 across subgroups when the number of studies was greater than five, because the estimate of τ2 may be imprecise when the number of studies in a subgroup is smaller (i.e. n≤5).[20, 25]

The subgroups meta-analysis enclosed vaccine type, quality, age of vaccinated population, and SARS-CoV-2 lineage detected through PCR tests. Although five or more studies were needed to consistently achieve power from random effects meta-analysis, given a need for a conclusion, two studies were considered sufficient for the statistical inference.[26, 27]

In order to test whether publication biases have influenced the study conclusions we performed a funnel plot (Annex 4.2).[28]

## Results

The web search provided 7,760 unduplicated records. Overall, 31 studies were selected for data extraction and qualitative data synthesis. (Fig. 5, Annex 2). There were 24 cohort studies, two case-control, one cross sectional, one matched observational, and three test negative case-control. Ten studies were performed in UK, eight in US, and five in Israel. RR of SARS-CoV-2 infection among Health care workers (HCWs) was examined by 13 studies; four studies analysed long term care facilities (LCTF) residents and nine the general population. (Tab.3, Annex 2)

### Qualitative synthesis

Four studies met the inclusion criteria for the systematic review but were not included in the quantitative analyses. [50, 53, 54, 55]

A study from Israel assessed the BNT162b2 effectiveness through national surveillance system. The study reported significant VE for asymptomatic SARS-CoV-2 infection (90.4%), symptomatic COVID-19 disease (96.3%), hospitalization (96.0%), severe and critical hospitalization (96.2%), and death (93.3%).[50]

Drury et al. conducted a study through the COVID Symptom Study app in UK. The results showed a progressive risk reduction for SARS-CoV-2 infections among vaccinated over time. Both BNT162b2 (risk reduction increased from 58% to 72%) and ChAdOx1/AZD1222 (risk of infection increased from 39% to 60%) single dose effectiveness was tested against unvaccinated subjects.[53]

In a prospective cohort study conducted among frontline HCWs in US, Thompson et al. reported 90% VE for full immunization and 80% for partial immunization.[54]

In a prospective cohort-study conducted on population registries in Scotland, Vasilieu et al. evaluated the effectiveness of BNT162b2 and ChAdOx1/AZD1222 single dose administration against COVID-19 hospitalization. VE was 85% for BNT162b and 94% for ChAdOx1 /AZD1222.[55]

### Meta-analysis

In the following meta-analysis a RE model with inverse variance (IV) method was used throughout. The evidence synthesis was based on RR as measure of effect. Additionally, the FE estimates as well as the Mantel-Haenszel outputs were reported in Table 1. The between-study variance τ2 was computed for each RE meta-analysis.

View this table:
[Table 1)](http://medrxiv.org/content/early/2021/08/28/2021.08.25.21262529/T1)

Table 1) 
Meta-analyses: relative risk of SARS-CoV-2 of positive PCR test after vaccination. Fixed (FE) and Random Effect (RE), inverse variance method (IV) and Mantel-Haenszel (MH) method. RR of any positive PCR and symptomatic positive PCR after SARS-CoV-2 vaccination. Any SARS-CoV-2 vaccination protocol.

Overall, 27 out of 31 studies were included in the meta-analysis. Despite some small sample sizes and few infection cases expected among vaccinated individuals, no study presented zero cells during the data extraction. ChAdOx1/AZD1222 effectiveness was investigated in 4 out of 27 studies,[31, 32, 51, 52] while only one study estimated Ad26.COV2.S effectiveness.[47]

The Newcastle-Ottawa Quality Assessment Scale showed a satisfactory quality, with a median NOS score of six, a minimum of five and a maximum of eight.[29]

#### RR of SARS-CoV-2 infection following vaccination

The meta-analysis on first dose VE against any positive PCR pooled 17 studies and 22 entries. Jones’ analysis spanned over a follow-up of two weeks (A) and, additionally, an extended period of six weeks (B).[30] Bernal et al. evaluated the effectiveness of one and two doses of BNT162b2 vaccine on adults aged ≥ 80 years (A80 and B80) and ≥70 years (B70 and C70) in England. Additionally, the effectiveness of ChAdOx1/AZD1222 single dose (A70) and BNT162b2 one dose at least (C70) were tested on adults aged ≥ 70 years (A70).[31] Shotri et al. estimated the protective effects of BNT162b2 (A) and ChAdOx1/AZD1222 (B) first dose against any SARS-CoV-2 infection.[32] One study from Denmark investigated BNT162b2 VE in two cohorts, namely long-term care facility residents (R) and HCWs (H).[33] Abu Raddad et al. measured the effectiveness of BNT162b2 against B.1.1.7 (A), B.1.351 (B), which became predominant within Qatar in the earliest 2021, and any different (C) SARS-CoV-2 variants.[34] According to the RE meta-analysis results (IV), partially vaccinated showed a RRR=73% of any SARS-CoV-2 positive PCR compared to unvaccinated (RR=0.27). The FE model yielded lower effectiveness estimates after partial vaccination (RR=0.41; RRR=59%). Although both FE and RE (IV) showed a statistically significant result (p< 0.0001), the heterogeneity was considerable (τ2=1.08; H = 18.43 [17.52; 19.38]; I2 = 99.7%); therefore the true size of the effect remained uncertain (Table 1, Fig. 2a).

![Figure 2)](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/08/28/2021.08.25.21262529/F2.medium.gif)

[Figure 2)](http://medrxiv.org/content/early/2021/08/28/2021.08.25.21262529/F2)

Figure 2) 
Forest plot, partially vaccinated. Any positive SARS-CoV-2 PCR RR [95%-CI] ≥ 14 days from first dose uptake (a). Forest plot, fully vaccinated. Any positive PCR RR [95%-CI] ≥7 days from full vaccination (b). Forest plot, any positive PCR RR [95%-CI], ≥14 days from vaccination with at least one dose. IV method (c).

In the evaluation of the RR for any positive SARS-CoV-2 PCR following full vaccination protocol, only data on mRNA vaccines (BNT162b2 and mRNA-1273) were available. Sixteen studies and 17 entries were included in this meta-analysis. One study evaluated Ad26.COV2.S effectiveness, whilst “Pritchard” indistinctly assessed the effectiveness of ChAdOx1/AZD1222 and BNT162b2 after full vaccination protocol.[35] In “Abu Raddad” study, data on PCR test were available after two weeks from the BNT162b2 second dose administration.[34] Four studies did not distinguish between BNT162b2 and mRNA-1273 vaccines.[36-39]

Basing on the RE meta-analysis results (IV), the RRR for any positive SARS-CoV-2 PCR following full vaccination protocol was 94% (95%CI=88%-98%) with RR=0.06 compared to unvaccinated individuals (Table 1, Fig. 2b). FE yielded a RRR=88%. Although both FE and RE showed a statistically significant protective effect of full COVID-19 vaccination (p< 0.0001), we should carefully draw conclusion over the true size effect because the heterogeneity was considerable (τ2=2.6; H = 10.07 [9.22; 11.00]; I2=99.0%). The test for heterogeneity was significant at 1% (Q= 1621.96, p=0). (Table 1, Fig. 2b).

A longer follow-up period represented the main advantage of VE estimation after at least one dose uptake. The mean length of follow-up was indeed 54 days after the first dose administration. The meta-analysis on RR of testing positive for any SARS-CoV-2 PCR included 18 entries and 14 studies. Menni et al. analysed the SARS-CoV-2 infection events on individuals who received one or two doses of BNT162b2 (A) and ChAdOx1/AZD1222 (B).[52] The results of RE meta-analysis (IV) produced a significant RRR=84% for vaccinated individuals compared to unvaccinated, with RR=0.26. The test for heterogeneity was significant at 1% (Q=1030.89, p<0.0001). (Table 1, Fig. 2c)

In order to assess the effectiveness of SARS-CoV-2 vaccination against symptomatic COVID-19 infection, we performed two meta-analyses, in partially and fully vaccinated individuals (Table1). The meta-analysis on full vaccination protocol included only studies estimating VE among individuals aged <69 years. The meta-analyses on VE against symptomatic SARS-CoV-2 infection pooled 9 studies and 17 entries in total. In particular, nine and eight entries pertained to partial and full vaccination protocols respectively. The IV method displayed significant results for both FE and RE (p< 0.05). According to the RE results (IV), partial vaccination achieved a RRR=78% (95%CI= 31%-93%) for symptomatic SARS-CoV-2 infection compared to unvaccinated, while fully vaccinated subjects exhibited a RRR= 94% (95%CI=84%-98%) (Fig. 3a, b). FE estimates of RRR were lower for both partially and fully vaccinated individuals (RRR=51% and RRR=76%, respectively). Both FE and RE were statistically significant (p<0.05).

![Figure 3)](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/08/28/2021.08.25.21262529/F3.medium.gif)

[Figure 3)](http://medrxiv.org/content/early/2021/08/28/2021.08.25.21262529/F3)

Figure 3) 
Forest plot, symptomatic positive SARS-CoV-2 PCR RR [95%-CI] ≥ 14 days from first dose uptake (a). Forest plot, fully vaccinated. Symptomatic positive PCR RR [95%-CI] ≥7 days from full vaccination (b). IV method.

Concerning hospitalization risk, a RE meta-analysis over 2 studies,[31, 40] produced a RR=0.38 (95%CI=0.2719; 0.5242) after a full BNT162b2 vaccination protocol. (Q= 2.04, p= 0.1529; I2 = 51.1% [0.0%; 87.6%]). Basing on Bernal et al. data, at least one dose of BNT162b2 vaccine ensured a RRR of death equal to 83%, compared to unvaccinated individuals.[31]

Considering the substantial heterogeneity and the significance of the heterogeneity test, we could not be overly confident that the RR estimate would be robust in every context; therefore, we performed a sensitivity analysis to address the between-study heterogeneity (Annex 3).

#### L’ Abbé plot

The L’ Abbé plots in Fig. 4 confirmed that, overall, infection events were in favour of the control group (unvaccinated). Regarding the meta-analysis on any positive SARS-CoV-2 PCR case after partial vaccination, no study laid above the 45° except “Dagan” and “Angel”, which overlapped the line of no-effect (Fig. 4a), while “Dagan” contributed the most to the overall effect size.[11, 46] Three studies clearly deviated from the effect-size line in Fig. 4a,[34, 38, 41] and this might explain the highest heterogeneity encountered in these data. Four studies were very small and less precise.[38, 41-43] In fully vaccinated meta-analysis (Fig4b), “Pawloski” crossed the no-effect line, while “Abu-Raddad B” and “Hall” evidently deviated from the dashed line of the pooled effect. In Fig. 4c “Hall” and “Lopez Bernal≥80” diverged from the overall RE estimate (light dashed line), while “Monge” appeared as the largest sample sized.[31, 44]

![Fig. 4)](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/08/28/2021.08.25.21262529/F4.medium.gif)

[Fig. 4)](http://medrxiv.org/content/early/2021/08/28/2021.08.25.21262529/F4)

Fig. 4) L’Abbè plots.
Partially vaccinated, any positive SARS-CoV-2 PCR RR ≥14 days from first SARS-CoV-2 dose administration (a). Fully vaccinated, any positive PCR RR, ≥7 days from full vaccination (b). At least one dose, any positive PCR RR ≥14 days from vaccination with first dose (c). Partially vaccinated, symptomatic positive PCR RR ≥14 days from first dose administration (d). Fully vaccinated, symptomatic positive SARS-CoV-2 PCR RR ≥7 days from full vaccination (e). The sizes of the plotted circles are proportional to the precision of the studies. The dashed lines marked the overall estimate of the log risk for RE (light) and FE (bold). The further the circle from the line of no effect, the greater the difference of event rates between intervention and control arms.

Concerning symptomatic SARS-CoV-2 infections, the L’Abbè plots in Fig. 4d) and 4e) confirmed that the infection rates were greater in unvaccinated groups than partially or fully vaccinated groups. Overall, “Dagan”, “Jones”, and “Abu-Raddad” were the largest sized studies (Fig. 4d and 4e).[11, 13, 45] In Fig. 4d, “Dagan” partially overlapped the 45° line. No study frankly diverged from the light dashed line (RE overall RR) except “Hyams”.[11, 45]

#### Subgroup meta-analysis

The GOSH analysis did not indicate any clear cluster that contributed to the pooled imbalance (Annex 3). In the subgroup meta-analysis, we grouped the studies by type of vaccine, mean age of the sample (≥69 and < 69 years), SARS-CoV-2 lineage, and bias assessment performed through Newcastle-Ottawa Scale (NOS). We classified NOS quality score into satisfactory (NOS ≤6) and good (NOS >6).[29] The meta-analysis was performed over two studies at least.[27] The results of FE model were presented in Tab.2 while mixed and RE subgroup meta-analyses were attached in Tab.10 and 11 (Appendix 4.1). The FE subgroup meta-analysis broadly explained the between-study heterogeneity because all subgroups produced a significant Q test (p< 0.0001). However, considered the residual heterogeneity and the small number of studies in several subgroups, results must be interpreted with caution.

The first subgroup analysis compared different vaccine technologies. Basing on FE subgroup estimates, the RRR after BNT162b2 first dose ranged from 46% to 49% for any positive SARS-COV-2 PCR and from 35% to 32% for symptomatic events. The discrepancy might be explained by the low number of studies included in the subgroup evaluating VE against symptomatic SARS-COV-2 infection (n=6) with respect to the group evaluating the effectiveness of BNT162b2 vaccine against any positive PCR (n=16) (Table 2). Only one study assessed the effectiveness of ChAdOx1/AZD1222 against symptomatic PCR (RR=0.34), however, the effect size was probably overestimated. Corchado et al. analysis suggested a RRR=77% for Ad26.COV2.S,[47] while “Pritchard” did not distinguish between BNT162b2 and ChAdOx1/AZD1222 in VE analysis.[35] In FE the RR estimates were greater within high quality studies (NOS>6), either symptomatic or not (RR=0.12 and RR=0.07 respectively) compared to satisfactory studies (NOS≤6, RR=0.67 and RR=0.48, respectively). Partial vaccination effectiveness appeared lower within studies which examined older population (≥69 years) compared to younger, either on symptomatic PCR (RR= 0.40 and RR= 0.49, respectively) or any positive PCR (RR=0.29 and RR= 0.48, respectively).

View this table:
[Table 2)](http://medrxiv.org/content/early/2021/08/28/2021.08.25.21262529/T2)

Table 2) 
Subgroup meta-analysis, any SARS-CoV-2 protocol. Fixed effect model.

Regarding the FE subgroup meta-analysis in fully vaccinated (Table 2), BNT162b2 produced a RRR=83% against any positive PCR and a RRR=68% against symptomatic SARS-CoV-2 infections. The RE and the mixed model produced a RRR=94% against any positive PCR and RRR=84% against symptomatic COVID-19. Further, the FE subgroup meta-analysis relative to the bias assessment (NOS) showed a greater RR for low quality studies with respect to high quality studies (RR=0.01 in symptomatic PCR and RR=0.33 in any positive PCR meta-analyses, respectively). Full vaccination effectiveness against any positive PCR appeared slightly lower within studies which examined older population (≥69 years) compared to younger (RR=0.15 and RR= 0.12, respectively).

Concerning partial vaccination effectiveness against SARS-CoV-2 variants, the RRR of any positive PCR for B.1.1.7 resulted nearly 60% in FE and 81% in mixed model. The RRR increased to 85% in FE and 92% in mixed model after full vaccination. Although only “Abu Raddad” assessed the effectiveness of BNT162b2 against B.1.351 lineage,[34] the RRR was lower for the B.1.351 variant in both partially (RRR=9%) and fully vaccinated subjects (RRR=60%). Regarding symptomatic PCR, the RRR within fully vaccinated individuals remained higher for B.1.1.7 (RRR=86%) with respect to B.1.351 (RRR=64%) (Table 5a, b, e).

#### Publication bias

The funnel plots were displayed in Fig. 26 (Annex 4.2). The contour-enhanced funnel plots did not exhibit publication bias. The majority of the studies fell into p<0.01 except “Tenforde” (Fig. 26 a, d). The Egger test indicated the presence of funnel plot asymmetry in meta-analysis for symptomatic SARS-CoV-2 RR after full vaccination protocol (intercept=-11.271, p=0.026), although it may lack the statistical power to detect bias, because the number of studies was small (n<10).

## DISCUSSION

Despite the considerable between-study heterogeneity, our findings provided evidence that any COVID-19 vaccine is highly effective outside the controlled conditions of the clinical trials. Overall, SARS-CoV-2 vaccination is effective in reducing the number of new COVID-19 events, either asymptomatic or symptomatic, with the greatest benefit achieved after completing the full vaccination protocol. Fewer real-world data were available for ChAdOx1/AZD1222 due to its later regulatory approval. Yet, vaccination with ChAdOx1/AZD1222 resulted in the rare onset of immune thrombotic thrombocytopenia; therefore, it underwent an additional careful monitoring process by European Union (EU) regulatory authorities.[48]

According to our findings, partially vaccinated individuals were only one-fourth as likely (RR=0.26) to develop any SARS-CoV-2 infection, as were unvaccinated individuals; whilst the overall risk of symptomatic infection was slightly lower (RR=0.22). The full vaccination schedule was effective at 94% against asymptomatic and symptomatic positive PCR tests. Concerning hospitalization risk, a meta-analysis over two studies,[31, 40] yielded a RRR=62% ensuing a full BNT162b2 vaccination protocol. Concerning mortality risk, our findings corroborated Bernal et al. study,[31] conducted over 7.5 million adults aged 70 years and older in UK. At least one dose of BNT162b2 was approximately 83%, effective at preventing death compared to unvaccinated individuals, while there was insufficient follow-up to assess ChAdOx1/AZD1222 impact on mortality because of the later rollout.

The subgroup analysis did not find any significant difference in effectiveness between the mRNA and the modified adenovirus vaccines, in part because the evidence, especially for ChAdOx1/AZD1222, was still very scarce. Up to May 15, only two studies evaluated ChAdOx1/AZD1222,[31, 32, 51, 52] and only one Ad26.COV2.S effectiveness.[47] As expected, COVID-19 VE appeared higher in adults aged <69 years. The evaluation of VE against the SARS-CoV-2 variants remained a key point that lacked of robustness in results. In fact, only one study investigated the effectiveness of BNT162b2 against two variants of concern.[34] The RR of infection following a full vaccination protocol appeared larger for B.1.351 (South Africa) variant compared to B.1.1.7 (UK). Although the evidence suggested a lower VE against SARS-COV-2 variants of concern, our findings were not sufficient to assert that COVID-19 full vaccination protocol is 60% effective against B.1.351 and 94% effective against B.1.1.7.

Our research aimed at addressing questions not answered by clinical trials, by including non-randomized studies undertaken after vaccine licensure. Moreover, we attempted to assess VE within different subgroups not included in the preauthorization vaccine clinical trials. Although imprecise, real world data confirmed the experimental evidence. Nevertheless, the lower level of protection assured against emerging variants of concern encouraged to pursue an accurate and robust VE validation.

This systematic review acknowledged several limitations. Undeniably, the expected heterogeneity stemmed from the observational design of the included studies. Unless a randomized process, it is unlikely to be reduced in any circumstances. Moreover, several studies lacked of data on population characteristics. For instance, the meta-analysis predominantly included Caucasian populations whereas vaccines may have different effectiveness across individuals from different ethnicities. Moreover, five studies did not declare to test systematically the vaccinated cohort during the follow-up. The absence of active laboratory surveillance within vaccinated individuals might have resulted in an underestimation of asymptomatic cases. However, we did not consider a different rate of testing as a bias of concern in terms of RR estimate, because both vaccinated and unvaccinated individuals underwent the same case investigation and contact tracing protocol. Moreover, when systematic testing was not performed, asymptomatic testing was available to different extent: workplace exposures (HCWs), out-of-state travellers or per-request.[37] In general, SARS-CoV-2 PCR testing was highly accessible and free of charge.[11, 50]

However, asymptomatic cases which progressed into symptomatic disease could act as source of bias in both vaccinated and unvaccinated groups.[37, 43] Finally, public health mitigation measures might contribute to the underestimation of SARS-CoV-2 vaccine effectiveness. Unfortunately, the extent to which primary prevention restrictions affected VE was beyond the scope of this study.

By way of conclusion, we can state that significant reduction in RR of asymptomatic infection within partially vaccinated was not corroborated by sufficient statistical robustness for the generalizability of results. However, full vaccination effectiveness over symptomatic and asymptomatic SARS-CoV-2 infections risk, confirmed the RCTs results, except larger CI.

To the best of our knowledge, this is the first evidence-based data synthesis on SARS-CoV-2 VE. Our findings support the maximization of the full vaccination coverage and encourage a better calibration of the statistical inference as the mass vaccination wears on. Long-term vaccine effectiveness remains unexplained and needs further investigation. Better evidence over the impact of SARS-CoV-2 variants are vital in order to monitor mutations associated with vaccine escape.

## Supporting information

Annex [[supplements/262529_file06.pdf]](pending:yes)

## Data Availability

This manuscript does not contain personal and/or medical information about identifiable individuals. The study protocol is available at PROSPERO web registry (n. CRD42021240143). The statistical analysis plan is available upon request from the Authors.

## FUNDING

This work was supported by the Italian Ministry of Health. COMPETING INTERESTS: none.

## ETHICS APPROVAL STATEMENT

No ethics committee approval was required.

## CONTRIBUTORSHIP STATEMENT

Each named author has substantially contributed to conducting the underlying research and writing the paper.

## DATA SHARING STATEMENT

this manuscript does not contain personal and/or medical information about identifiable individuals. The study protocol is available at PROSPERO web registry (n. CRD42021240143). The statistical analysis plan is available upon request from the Authors.

## PATIENT AND PUBLIC INVOLEMENT

patients were involved by primary studies.

## Web appendix

Supplementary material

*   Received August 25, 2021.
*   Revision received August 25, 2021.
*   Accepted August 28, 2021.


*   © 2021, Posted by Cold Spring Harbor Laboratory

The copyright holder for this pre-print is the author. All rights reserved. The material may not be redistributed, re-used or adapted without the author's permission.

## REFERENCES

1.  1.([https://COVID19.who.int/](https://COVID19.who.int/) Accessed 13.07.2021)
    
    

2.  2.Chen Y, Liu Q, Guo D. Emerging coronaviruses: genome structure, replication, and pathogenesis. Journal of medical virology. 2020 Apr;92(4):418–23.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/jmv.25681&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F08%2F28%2F2021.08.25.21262529.atom) 

3.  3.Peng Y, Mentzer AJ, Liu G, Yao X, Yin Z, Dong D, Dejnirattisai W, Rostron T, Supasa P, Liu C, Lopez-Camacho C. Broad and strong memory CD4+ and CD8+ T cells induced by SARS-CoV-2 in UK convalescent individuals following COVID-19. Nature immunology. 2020 Nov;21(11):1336–45.
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F08%2F28%2F2021.08.25.21262529.atom) 

4.  4.Ramasamy MN, Minassian AM, Ewer KJ, Flaxman AL, Folegatti PM, Owens DR, Voysey M, Aley PK, Angus B, Babbage G, Belij-Rammerstorfer S. Safety and immunogenicity of ChAdOx1 nCoV-19 vaccine administered in a prime-boost regimen in young and old adults (COV002): a single-blind, randomised, controlled, phase 2/3 trial. The Lancet. 2020 Dec 19;396(10267):1979–93.
    
    

5.  5.([https://www.ema.europa.eu/en/medicines](https://www.ema.europa.eu/en/medicines) Accessed 13.07.2021)
    
    

6.  6.Polack FP, Thomas SJ, Kitchin N, Absalon J, Gurtman A, Lockhart S, Perez JL, Marc GP, Moreira ED, Zerbini C, Bailey R. Safety and efficacy of the BNT162b2 mRNA COVID-19 vaccine. New England Journal of Medicine. 2020 Dec 10.
    
    

7.  7.Baden LR, El Sahly HM, Essink B, Kotloff K, Frey S, Novak R, Diemert D, Spector SA, Rouphael N, Creech CB, McGettigan J. Efficacy and safety of the mRNA-1273 SARS-CoV-2 vaccine. New England Journal of Medicine. 2021 Feb 4;384(5):403–16.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1056/NEJMoa2035389&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F08%2F28%2F2021.08.25.21262529.atom) 

8.  8.([https://www.gavi.org/vaccineswork/what-difference-between-efficacy-and-effectiveness](https://www.gavi.org/vaccineswork/what-difference-between-efficacy-and-effectiveness); Accessed 13.07.2021)
    
    

9.  9.Aran D. Estimating real-world COVID-19 vaccine effectiveness in Israel using aggregated counts. MedRxiv. 2021 Jan 1.
    
    

10. 10.Chodick G, Tene L, Patalone T, Gazit S, Ben-Tov A, Cohen D, Muhsen K. The Effectiveness of the First Dose of BNT162b2 Vaccine in Reducing SARS-CoV-2 Infection: Real-World Evidence. Available at SSRN 3769977. 2021 Jan 20.
    
    

11. 11.Dagan N, Barda N, Kepten E, Miron O, Perchik S, Katz MA, Hernán MA, Lipsitch M, Reis B, Balicer RD. BNT162b2 mRNA COVID-19 vaccine in a nationwide mass vaccination setting. New England Journal of Medicine. 2021 Apr 15;384(15):1412–23.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1056/NEJMOA2101765&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F08%2F28%2F2021.08.25.21262529.atom) 

12. 12.Rossman, H. et al. Patterns of COVID-19 pandemic dynamics following deployment of a broad national immunization program. (2021) doi: 10.1101/2021.02.08.21251325.
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoibWVkcnhpdiI7czo1OiJyZXNpZCI7czoyMToiMjAyMS4wMi4wOC4yMTI1MTMyNXYyIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjEvMDgvMjgvMjAyMS4wOC4yNS4yMTI2MjUyOS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

13. 13.Jones NK, Rivett L, Seaman S, Samworth RJ, Warne B, Workman C, Ferris M, Wright J, Quinnell N, Shaw A, Goodfellow IG. Single-dose BNT162b2 vaccine protects against asymptomatic SARS-CoV-2 infection. Elife. 2021 Apr 8;10:e68808.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.7554/eLife.68808&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=33830018&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F08%2F28%2F2021.08.25.21262529.atom) 

14. 14.Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. Journal of clinical epidemiology. 2009 Oct 1;62(10):e1–34.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.jclinepi.2009.06.006&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=19631507&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F08%2F28%2F2021.08.25.21262529.atom) 

15. 15.Voysey M, Clemens SA, Madhi SA, Weckx LY, Folegatti PM, Aley PK, Angus B, Baillie VL, Barnabas SL, Bhorat QE, Bibi S. Safety and efficacy of the ChAdOx1 nCoV-19 vaccine (AZD1222) against SARS-CoV-2: an interim analysis of four randomised controlled trials in Brazil, South Africa, and the UK. The Lancet. 2021 Jan 9;397(10269):99–111.
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F08%2F28%2F2021.08.25.21262529.atom) 

16. 16.Sadoff J, Gray G, Vandebosch A, Cárdenas V, Shukarev G, Grinsztejn B, Goepfert PA, Truyers C, Fennema H, Spiessens B, Offergeld K. Safety and efficacy of single-dose Ad26. COV2. S vaccine against COVID-19. New England Journal of Medicine. 2021 Jun 10;384(23):2187–201.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1056/NEJMoa2101544&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F08%2F28%2F2021.08.25.21262529.atom) 

17. 17.Rothman KJ, Greenland S, Lash TL. Modern epidemiology. Philadelphia: Wolters Kluwer Health/Lippincott Williams & Wilkins; 2008 Sep 20.
    
    

18. 18.McAloon C, Collins Á, Hunt K, Barber A, Byrne AW, Butler F, Casey M, Griffin J, Lane E, McEvoy D, Wall P. Incubation period of COVID-19: a rapid systematic review and meta-analysis of observational research. BMJ open. 2020 Aug 1;10(8):e039652.
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYm1qb3BlbiI7czo1OiJyZXNpZCI7czoxMjoiMTAvOC9lMDM5NjUyIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjEvMDgvMjgvMjAyMS4wOC4yNS4yMTI2MjUyOS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

19. 19.Fleiss JL, Gross AJ. Meta-analysis in epidemiology, with special reference to studies of the association between exposure to environmental tobacco smoke and lung cancer: a critique. Journal of clinical epidemiology. 1991 Jan 1;44(2):127–39.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/0895-4356(91)90261-7&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=1995774&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F08%2F28%2F2021.08.25.21262529.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1991EY98900004&link_type=ISI) 

20. 20.Schwarzer G, Carpenter JR, Rücker G. Meta-analysis with R. Cham: springer; 2015 Oct 8.
    
    

21. 21.Viechtbauer W, Cheung MW. Outlier and influence diagnostics for meta-analysis. Research synthesis methods. 2010 Apr;1(2):112–25.
    
    

22. 22.Baujat B, Mahé C, Pignon JP, Hill C. A graphical method for exploring heterogeneity in meta-analyses: application to a meta-analysis of 65 trials. Statistics in medicine. 2002 Sep 30;21(18):2641–52.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/sim.1221&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=12228882&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F08%2F28%2F2021.08.25.21262529.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000178003300003&link_type=ISI) 

23. 23.Olkin I, Dahabreh IJ, Trikalinos TA. GOSH–a graphical display of study heterogeneity. Research synthesis methods. 2012 Sep;3(3):214–23.
    
    

24. 24.Higgins JP, Thompson SG. Controlling the risk of spurious findings from meta-regression. Statistics in medicine. 2004 Jun 15;23(11):1663–82.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/sim.1752&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=15160401&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F08%2F28%2F2021.08.25.21262529.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000221661600001&link_type=ISI) 

25. 25.Borenstein M, Hedges LV, Higgins JP, Rothstein HR. A basic introduction to fixed-effect and random-effects models for meta-analysis. Research synthesis methods. 2010 Apr;1(2):97–111.
    
    

26. 26.Jackson D, Turner R. Power analysis for random-effects meta-analysis. Research synthesis methods. 2017 Sep;8(3):290–302.
    
    

27. 27.Valentine JC, Pigott TD, Rothstein HR. “How many studies do you need? A primer on statistical power for meta-analysis”: Erratum.
    
    

28. 28.Egger M, Smith GD, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. Bmj. 1997 Sep 13;315(7109):629–34.
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEyOiIzMTUvNzEwOS82MjkiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMS8wOC8yOC8yMDIxLjA4LjI1LjIxMjYyNTI5LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

29. 29.Peterson J, Welch V, Losos M, Tugwell PJ. The Newcastle-Ottawa scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. Ottawa: Ottawa Hospital Research Institute. 2011:1–2.
    
    

30. 30.Jones NK, Rivett L, Seaman S, Samworth RJ, Warne B, Workman C, Ferris M, Wright J, Quinnell N, Shaw A, Goodfellow IG. Single-dose BNT162b2 vaccine protects against asymptomatic SARS-CoV-2 infection. Elife. 2021 Apr 8;10:e68808.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.7554/eLife.68808&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=33830018&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F08%2F28%2F2021.08.25.21262529.atom) 

31. 31.Bernal JL, Andrews N, Gower C, Stowe J, Robertson C, Tessier E, Simmons R, Cottrell S, Robertson R, O’Doherty M, Brown K. Early effectiveness of COVID-19 vaccination with BNT162b2 mRNA vaccine and ChAdOx1 adenovirus vector vaccine on symptomatic disease, hospitalizations and mortality in older adults in England. MedRxiv. 2021 Jan 1.
    
    

32. 32.Shrotri M, Krutikov M, Palmer T, Giddings R, Azmi B, Subbarao S, Fuller C, Irwin-Singer A, Davies D, Tut G, Bernal JL. Vaccine effectiveness of the first dose of ChAdOx1 nCoV-19 and BNT162b2 against SARS-CoV-2 infection in residents of long-term care facilities in England (VIVALDI): a prospective cohort study. The Lancet Infectious Diseases. 2021 Jun 23.
    
    

33. 33.Moustsen-Helms IR, Emborg HD, Nielsen J, Nielsen KF, Krause TG, Molbak K, Moeller KL, Berthelsen AS, Valentiner-Branth P. Vaccine effectiveness after 1st and 2nd dose of the BNT162b2 mRNA COVID-19 Vaccine in long-term care facility residents and healthcare workers–a Danish cohort study. MedRxiv. 2021 Jan 1.
    
    

34. 34.Abu-Raddad LJ, Chemaitelly H, Butt AA. Effectiveness of the BNT162b2 COVID-19 Vaccine against the B. 1.1. 7 and B. 1.351 Variants. New England Journal of Medicine. 2021 May 5.
    
    

35. 35.Pritchard E, Matthews PC, Stoesser N, Eyre DW, Gethings O, Vihta KD, Jones J, House T, VanSteenHouse H, Bell I, Bell JI. Impact of vaccination on new SARS-CoV-2 infections in the UK. medRxiv. 2021 Jan 1.
    
    

36. 36.Mody A. Review 1:” FDA-authorized COVID-19 vaccines are effective per real-world evidence synthesized across a multi-state health system”. Rapid Reviews COVID-19. 2021 Apr 26.
    
    

37. 37.Bouton TC, Lodi S, Turcinovic J, Weber SE, Quinn E, Korn C, Steiner J, Schechter-Perkins EM, Duffy E, Ragan EJ, Taylor BP. COVID-19 vaccine impact on rates of SARS-CoV-2 cases and post vaccination strain sequences among healthcare workers at an urban academic medical center: a prospective cohort study. medRxiv. 2021 Jan 1.
    
    

38. 38.Tenforde MW. Effectiveness of Pfizer-BioNTech and Moderna Vaccines Against COVID-19 Among Hospitalized Adults Aged≥ 65 Years—United States, January–March 2021. MMWR. Morbidity and mortality weekly report. 2021;70.
    
    

39. 39.Swift MD, Breeher LE, Tande AJ, Tommaso CP, Hainy CM, Chu H, Murad MH, Berbari EF, Virk A. Effectiveness of mRNA COVID-19 vaccines against SARS-CoV-2 infection in a cohort of healthcare personnel. Clinical Infectious Diseases. 2021 Apr 26.
    
    

40. 40.Mason TF, Whitston M, Hodgson J, Watkinson RE, Lau YS, Abdulrazeg O, Sutton M. Effects of BNT162b2 mRNA vaccine on COVID-19 infection and hospitalization among older people: matched case control study for England. medRxiv. 2021 Jan 1.
    
    

41. 41.Gras-Valentí P, Chico-Sánchez P, Algado-Sellés N, Jiménez-Sepúlveda NJ, Gómez-Sotero IL, Fuster-Pérez M, Cartagena-Llopis L, Sánchez-Valero M, Cerezo-Milán P, Martínez-Tornero I, Tremiño-Sánchez L. Effectiveness of the first dose of BNT162b2 vaccine to preventing COVID-19 in healthcare personnel. Revista Espanola de Salud Publica. 2021 Apr 29;95.
    
    

42. 42.Hall VJ, Foulkes S, Saei A, Andrews N, Oguti B, Charlett A, Wellington E, Stowe J, Gillson N, Atti A, Islam J. COVID-19 vaccine coverage in health-care workers in England and effectiveness of BNT162b2 mRNA vaccine against infection (SIREN): a prospective, multicentre, cohort study. The Lancet. 2021 May 8;397(10286):1725–35.
    
    

43. 43.Britton A, Slifka KM, Edens C, Nanduri SA, Bart SM, Shang N, Harizaj A, Armstrong J, Xu K, Ehrlich HY, Soda E. Effectiveness of the Pfizer-BioNTech COVID-19 vaccine among residents of two skilled nursing facilities experiencing COVID-19 outbreaks—Connecticut, December 2020–February 2021. Morbidity and Mortality Weekly Report. 2021 Mar 19;70(11):396.
    
    

44. 44.Monge S, Olmedo C, Alejos B, Lapena M, Sierra MJ, Limia A. Direct and indirect effectiveness of mRNA vaccination against SARS-CoV-2 infection in long-term care facilities in Spain. medRxiv. 2021 Jan 1.
    
    

45. 45.Hyams C, Marlow R, Maseko Z, King J, Ward L, Fox K, Heath R, Turner A, Friedrich Z, Morrison L, Ruffino G. Assessing the effectiveness of BNT162b2 and ChAdOx1nCoV-19 COVID-19 vaccination in prevention of hospitalizations in elderly and frail adults: a single centre test negative case-control study.
    
    

46. 46.Angel Y, Spitzer A, Henig O, Saiag E, Sprecher E, Padova H, Ben-Ami R. Association between vaccination with BNT162b2 and incidence of symptomatic and asymptomatic SARS-CoV-2 infections among health care workers. JAMA. 2021 May 6.
    
    

47. 47.Corchado-Garcia J, Hughes T, Cristea-Platon T, Lenehan P, Pawlowski C, Bade S, O’Horo JC, Gores GJ, Williams AW, Badley AD, Halamka J. Real-world effectiveness of Ad26. COV2. S adenoviral vector vaccine for COVID-19.
    
    

48. 48.Covid EA. Vaccine: EMA Finds Possible Link to very Rare Cases of Unusual Blood Clots with Low Blood Platelets.
    
    

49. 49.Ma LL, Wang YY, Yang ZH, Huang D, Weng H, Zeng XT. Methodological quality (risk of bias) assessment tools for primary and secondary medical studies: what are they and which is better?. Military Medical Research. 2020 Dec;7(1):1–1.
    
    

50. 50.Haas EJ, Angulo FJ, McLaughlin JM, Anis E, Singer SR, Khan F, Brooks N, Smaja M, Mircus G, Pan K, Southern J. Impact and effectiveness of mRNA BNT162b2 vaccine against SARS-CoV-2 infections and COVID-19 cases, hospitalizations, and deaths following a nationwide vaccination campaign in Israel: an observational study using national surveillance data. The Lancet. 2021 May 15;397(10287):1819–29.
    
    

51. 51.Lumley SF, Rodger G, Constantinides B, Sanderson N, Chau KK, Street TL, O’Donnell D, Howarth A, Hatch SB, Marsden BD, Cox S. An observational cohort study on the incidence of SARS-CoV-2 infection and B. 1.1. 7 variant infection in healthcare workers by antibody and vaccination status. medRxiv. 2021 Jan 1.
    
    

52. 52.Menni C, Klaser K, May A, Polidori L, Capdevila J, Louca P, Sudre CH, Nguyen LH, Drew DA, Merino J, Hu C. Vaccine side-effects and SARS-CoV-2 infection after vaccination in users of the COVID Symptom Study app in the UK: a prospective observational study. The Lancet Infectious Diseases. 2021 Apr 27.
    
    

53. 53.Drury RE, O’Connor D. Symptom study app provides real-world data on COVID-19 vaccines. Lancet Infect Dis. 2021 Jul;21(7):890–891. doi: 10.1016/S1473-3099(21)00264-4. Epub 2021 Apr 27. PMID: 33930321; PMCID: PMC8078875.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S1473-3099(21)00264-4&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=33930321&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F08%2F28%2F2021.08.25.21262529.atom) 

54. 54.Thompson MG, Burgess JL, Naleway AL, Tyner HL, Yoon SK, Meece J, Olsho LEW, Caban-Martinez AJ, Fowlkes A, Lutrick K, Kuntz JL, Dunnigan K, Odean MJ, Hegmann KT, Stefanski E, Edwards LJ, Schaefer-Solle N, Grant L, Ellingson K, Groom HC, Zunie T, Thiese MS, Ivacic L, Wesley MG, Lamberte JM, Sun X, Smith ME, Phillips AL, Groover KD, Yoo YM, Gerald J, Brown RT, Herring MK, Joseph G, Beitel S, Morrill TC, Mak J, Rivers P, Harris KM, Hunt DR, Arvay ML, Kutty P, Fry AM, Gaglani M. Interim Estimates of Vaccine Effectiveness of BNT162b2 and mRNA-1273 COVID-19 Vaccines in Preventing SARS-CoV-2 Infection Among Health Care Personnel, First Responders, and Other Essential and Frontline Workers - Eight U.S. Locations, December 2020-March 2021. MMWR Morb Mortal Wkly Rep. 2021 Apr 2;70(13):495–500. doi: 10.15585/mmwr.mm7013e3. PMID: 33793460; PMCID: PMC8022879.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.15585/mmwr.mm7013e3&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=33793460&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F08%2F28%2F2021.08.25.21262529.atom) 

55. 55.Vasileiou E, Simpson CR, Shi T, Kerr S, Agrawal U, Akbari A, Bedston S, Beggs J, Bradley D, Chuter A, de Lusignan S, Docherty AB, Ford D, Hobbs FR, Joy M, Katikireddi SV, Marple J, McCowan C, McGagh D, McMenamin J, Moore E, Murray JL, Pan J, Ritchie L, Shah SA, Stock S, Torabi F, Tsang RS, Wood R, Woolhouse M, Robertson C, Sheikh A. Interim findings from first-dose mass COVID-19 vaccination roll-out and COVID-19 hospital admissions in Scotland: a national prospective cohort study. Lancet. 2021 May 1;397(10285):1646–1657. doi: 10.1016/S0140-6736(21)00677-2. Epub 2021 Apr 23. PMID: 33901420; PMCID: PMC8064669.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S0140-6736(21)00677-2&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=33901420&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F08%2F28%2F2021.08.25.21262529.atom)