A Simplified Bayesian Analysis Method for Vaccine Efficacy

1. Introduction

A key parameter of a vaccine against an infectious disease is the protective vaccine efficacy, usually denoted VE_S. This parameter is intended to represent the reduction due to the vaccine of the chance of contracting the disease, and is typically estimated using controlled double-blind clinical trials [1]. Simple point-estimates of VE_S may be produced by person-time exposure-weighted ratios of vaccine group incidence to placebo group incidence. Such estimates are known to be affected by correctable biases [2]. From a statistical perspective, however, interval estimates are more valuable than point estimates, as they yield information concerning the uncertainty in the estimates. Frequentist methods for estimating confidence regions have been proposed [e.g. 3, 4] and compared for relative conservativeness and accuracy of their coverage [5, 6].

Bayesian credible region estimation methods have also been proposed [7, 8]. Such methods are straightforward to implement and interpret, although they usually raise issues of choice of prior distribution over the parameters — both over VE_S and over another “nuisance” parameter, often the disease incidence in the placebo group. In [7] the prior density is chosen to be a product of log-normals, with “uninformative” widths. The work described in [8] examines the use of reference priors, as a means of exhibiting principled notion of “uninformativeness” [9]. It is worth noting that one reason for concern with prior choice is the extra degree of freedom represented by the nuisance parameter, since the posterior density must be marginalized over the nuisance parameter in order to obtain inferences that concern VE_S alone. It is a matter of due-diligence to demonstrate that such inferences are robust despite different prior choices, that is, that credible regions and point estimates suffer negligible variation across plausible families of prior densities. In many cases this desirable property of robustness is expected a priori for parameter estimation in the limit of abundant data, since in this case the likelihood is typically so sharply peaked that all priors of interest are for all intents and purposes constant in the parameter-space region where the mass of the posterior is concentrated [10]. However, the higher the dimension of the parameter space in which parameter inference tasks are performed, the greater the risk of loss of robustness to prior choice. In particular, if the nuisance parameter could be summarily ejected from the Bayesian estimation of VE_S, the resulting estimates should be expected to be more robust against prior choice, as well as simpler to obtain.

In this article we briefly describe a reparametrization of the Poisson likelihood that permits precisely this simplification: the choice of a new, reparametrized nuisance parameter λ — the total expected infections across both vaccine and placebo group — results in a factorization of the likelihood into a factor depending only on VE_S and a second factor depending only on λ. As a result, one obtains a simple, intuitive result for the reduced likelihood ℒ_e (VE_S), which in turn permits very simple inferences about credible regions, that are better insulated against variations due to prior choice.

We describe the method in §2. In §3 we exhibit it’s use in a study that compares the efficacy of three recently-announced vaccines agains SARS-CoV-2: The Pfizer/BioNTech mRNA vaccine [11], the Moderna mRNA vaccine [12], and the Sputnik V adenovirus-based vaccine [13]. We supply some final discussion in §4.

2. The Factorized Likelihood

We assume that infections in the placebo and vaccine samples are well-described by Poisson models with different means µ_P and µ_V, respectively. For the sake of notational compactness, we will denote the protective vaccine efficacy VE_S by the symbol e. We introduce a model for e by assuming that where r is the vaccine-to-placebo ratio of person-times at risk – that is, if the vaccine and placebo groups represent P_V and P_P person-times of exposure to the pathogen respectively, then r ≡ P_V / P_P. If the infection numbers recorded during the trial are N_V and N_P respectively for the vaccine and placebo groups, then the likelihood under the model is the product of the Poisson likelihoods for the two groups, that is

This form of the likelihood entangles the nuisance parameter µ_P with the quantity of interest, the efficacy e, so that after introducing a prior distribution and using Bayes’ theorem, the posterior distribution over e, µ_P is similarly entangled. Rather than submitting to the necessity of explicitly marginalizing the posterior over µ_P, we may introduce a factorized form of the likelihood by means of a change of variables. We introduce the parameter λ, defined as

The new parameter λ is the expected number of overall infections, summed over the two groups — this is clear, since the sum of two Poisson random variables is itself a Poisson random variable whose mean is the sum of their respective means. We may use Equation (2.3) to substitute λ for µ_P in Equation (2.3). The resulting likelihood is where we have introduced the reduced likelihood for e,

The advantage of the form of the likelihood in Equation (2.3) is that the dependence on e is factorized from the dependence on λ. If we now introduce a factorized prior π_λ (λ) ×π_e (e), then we obtain a factorized posterior distribution. Marginalization over λ is trivial, and we obtain for the marginalized posterior the formula where κ is a normalization constant.

From this point of the discussion, we assume, for simplicity, a proper uniform prior π_e (e) = 1 on the domain e ∈ [0, 1]. In §3.3 we will re-examine the effect of prior choice on this model.

One may easily show that if N_V 0 then the posterior in Equation (2.3) has a maximum at , which accords with intuition – the relative incidence 1 − e_MP is given by the ratio of the person-time adjusted infection rates. On the other hand, when N_V = 0 the posterior increases monotonically, attaining a domain maximum at e_MP = 1. This also accords with intuition: if no vaccinated subjects become infected, the most probable efficacy is 1.0 — perfect efficacy. Of course, in this case the distribution may still have substantial mass at much lower efficacies, depending on the value of N_P.

The normalization constant κ may be calculated by integrating Equation (2.3) with respect to e, in the interval [0, 1]. This may be done by the change of variables e →g = 1 + (1 − e) r, expanding the resulting integrand using the binomial theorem. The result is

The usage of the formula for the posterior in Equation (2.3) is very straightforward. One may simply bin the interval 0, 1 into N_Samp equally-spaced sample points, compute the factorized likelihood at each point (multiplying by a non-uniform prior if so desired), and compute the normalization κ either by Equation (2.3) (for a uniform prior) or by a straightforward numerical quadrature. Then, after normalization, various desired interval estimates (such as credible regions, or upper/lower bounds of prescribed probability) can be obtained by sorting the array containing the 1-dimensional discretized posterior over e, and processing it as appropriate.

We now demonstrate the use of these procedures using public data from SARS-Cov-2 vaccine trials.

3. Bayesian Analysis of SARS-CoV-2 Vaccine Trial Data

At the time of this writing (early December 2020), the final data from the Phase III trials of the mRNA vaccines against SARS-CoV-2 from Pfizer/BioNTech [11] and from Moderna [12] are available from public information, although not yet from scientific journal articles. The two data releases include results for entire trials, and for subsamples of infections leading to “severe” COVID-19 symptoms. In addition, preliminary data from the Sputnik V vaccine trials are also available [13]. The data are summarized in Table 1.

The results of applying the reduced likelihood Bayesian analysis (with a uniform prior) to these samples and sub-samples are summarized in Table 2, and displayed in the panels of Figure 3.1. The figures display the posteriors over the efficacy e (blue solid line), the maximum-posterior efficacies (cyan dashed line), equal-posterior-density 90% credible regions (red solid region), and 99% lower bounds (hatched regions). These are all obtained by straightforward application of the method described in §2, implemented in a short Python script that makes use only of the Numpy [14] and Matplotlib [15] modules. The code is available for public download under an MIT license at https://github.com/CarloGraziani/BayesVaccineEfficacy.

• Download figure
• Open in new tab

Figure 3.1:

Efficacy estimates: Maximum a posteriori estimates (dashed line), 90% credible regions (solid shaded regions), and 99% lower bounds (hatched regions). Top: Pfizer/BioNTech, overall (left) and severe (right). Middle: Moderna overall (left) and severe (right). Bottom: Interim Sputnik V data.

We now discuss these results in more detail.

4. Discussion

As illustrated in §3, the reduced likelihood approach leads to an easily-implemented 1-dimensional posterior density on the unit interval for the vaccine efficacy. Simple and intuitive results are easily extractable from such posteriors, and with the numbers characteristic of vaccine trials such as the three considered above, the overall efficacy estimates are quite robust to prior choice.

The most cogent take-away conclusion from the Bayesian analysis in §3 appears to be that on present data, the three vaccines for SARS-CoV-2 cannot be said to differ greatly in their efficacies. The differences are largely in the degree to which those efficacies are constrained by their respective trial data.

More thought needs to be given to prior choice in the event where subsamples have low counts. In the Pfizer/BioNTech Severe case, where there were 10 total infections, use of a skeptical prior such as the “show-me” prior can at least provide some conservatism to estimates. In such cases it is possible that more principled prior choice approaches, such as the reference prior approach advocated in [8] could lead to better-constrained, but still reasonable results. This is of course true irrespective of whether one adopts the reduced likelihood approach advocated here or another standard Bayesian nuisance parameterization.

Some generalization of the approach is possible: to test time-varying immunogenesis (whether short-term ramping up of immune response, or long-term decline of same, as in e.g. [16]), it is straightforward to introduce an empirical parametrized time dependence to the efficacy, i.e. e = g(t ; θ), where θ are parameters. If the data is in time bins, then the time-dependence g (t ; θ) could be integrated with respect to t over each time bin, and the likelihood computed as the product of the individual bin reduced likelihoods, which are now functions of θ. The resulting analysis no longer leads to simple manipulations of 1-dimensional posterior distributions on the unit interval, and MCMC methods may be advisable, depending on the dimensionality of θ. However, one benefit is that the nuisance parameter is still banished from the analysis, so that the parameter space has its dimensionality reduced by 1, and is limited to the empirical parameters of interest.

Declaration of Interests

The author declares that he has no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Government License

The submitted manuscript has been created by UChicago Argonne, LLC, Operator of Argonne National Laboratory (“Argonne”). Argonne, a U.S. Department of Energy Office of Science laboratory, is operated under Contract No. DE-AC02-06CH11357. The U.S. Government retains for itself, and others acting on its behalf, a paid-up nonexclusive, irrevocable worldwide license in said article to reproduce, prepare derivative works, distribute copies to the public, and perform publicly and display publicly, by or on behalf of the Government. The Department of Energy will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan. http://energy.gov/downloads/doe-public-access-plan.