Association between SARS-Cov-2 infection during pregnancy and adverse pregnancy outcomes: a re-analysis of data from Wei et al. (2021)

Background

Wei et al. (1) have recently published a systematic review and meta-analysis (MA) which aimed to “evaluate the association between SARS-Cov-2 infection during pregnancy and adverse pregnancy outcomes”.

The authors show that SARS-CoV-2 infection is associated with preeclampsia, preterm birth and stillbirth. They also stated that they used Mantel-Haenszel method, but the results of figure 2 are related to a random-effects inverse-variance model, with DerSimonian-Laird estimate of tau² and continuity correction.

Higgin’s I2 was used to assess heterogeneity. Although this approach is widely mentioned, the point estimate I2 should be interpreted cautiously when a MA has few studies (2), and the confidence interval should be provided.

MA relies on several within and between study distributional assumptions that are sometimes hidden (3).

Several methods are available to assign weights in meta-analyses, e.g. Mantel-Haenszel for fixed-effect MA or inverse-variance for fixed-effect or random-effects MA. For random-effects MA, there are several ways to estimate the between-study variance, e.g. DerSimonian and Laird (DL), restricted maximum likelihood (REML). Finally, there are also different ways to estimate the confidence interval for the summary effect (e.g. Wald or Hartung-Knapp-Sidik-Jonkman (HKSJ) method). The same apply to confidence interval for the between-study variance (e.g. Q-Profile method). See (4,5) for more information.

Performing MA with low event rates or with few studies, or both, may be challenging. For example, estimating between-study heterogeneity is difficult in this case, and inaccurate estimation of this heterogeneity may lead to too narrow confidence intervals.

Statistical methods can never completely resolve the issue of sparse data. Different methods may give different results, and using a suboptimal approach may lead to erroneous conclusions (6).

Methods

In order to assess the robustness of the results provided by Wei et al. (1), we performed a sensitivity analysis using a range of frequentist and Bayesian meta-analysis methods. We also estimated fragility indexes (and Fragility Quotient, which is fragility index divided by the total study sample size). This intuitive measure of the robustness of a trial, is a number indicating how many patients would be required to convert a trial from being statistically significant to not significant (7). This index has been adapted for MA (8) and is currently being refined methodologically (19). We estimated FI with DL method and restricted maximum likelihood with Knapp-Hartung (KNHA) test.

Wei et al. (1) reported three outcomes: preeclampsia, preterm and stillbirth. We focused on preeclampsia and stillbirth which contain sparse data.

For frequentist MA, several methods were used (9,10), including that of Hemni and Copas, which was found to be less sensitive to publication bias than DL method (11,12) (see tables). Although not optimal, continuity correction (0.5) was applied to studies with zero cells, in order to compare our results with the work of Wei et al. (1).

According to IntHout et al (18), we have also presented prediction interval, only for the model used by Wei et al. (1). It is the interval within the effect size of a new study would fall, if this study was selected at random from the same population of the studies already included in the MA.

For Bayesian MA, we used a binomial-normal hierarchical model (BNHM), i.e., modelling probabilities of success in each group (13), instead of modelling estimates of log odds-ratios directly (normal-normal model) for the between trial heterogeneity. We also used a beta-binomial (BB) model, which has shown good statistical properties for meta-analysis of sparse data (14,15). With these approaches, no continuity correction (of any type) is required as these models are based on the exact distributional assumptions unlike (commonly used) normal-normal hierarchical model. Several priors were used as sensitivity analysis (16,17).

Statistical analyses were performed with Stata (frequentist framework) and R software (Bayesian framework).

Results

All estimates are shown in tables 1 and 2 and in appendix 1 (tables 3 and 4).

In some cases, no estimation was possible: this is the case for permutations, when less than six studies were included in the meta-analysis. Under these conditions, Bayesian estimates are more appropriate.

For eclampsia in general (comparing patients with Covid-19 vs without Covid-19), the confidence intervals of most frequentist MA models contain 1 (table 1).

The prediction interval, based on DL method ranges from 0.75 to 2.38.

The fragility index is 2 for the DL method used by the authors and the same with the REML-HKSJ method.

As for the Bayesian approach, all three beta-binomial models of MA lead to credible intervals containing 1. For binomial normal models, credible intervals do not contain one, but the upper bound of the interval is very close to 1.

For eclampsia when comparing patients with severe vs mild Covid-19, the frequentist sensitivity analysis confirms the authors’ results, except for one model out of 12 (Kenward-Roger’s model).

The prediction interval, based on DL method ranges from 0.84 to 20.61.

The fragility index is 2 for DL method and 1 with REML-HKSJ method.

When an heterogeneity of 75% is taken into account, the difference between the two groups is no longer significant.

Concerning the Bayesian analysis, only the binomial-normal models confirmed Wei et al. (1) initial results. None of the three beta binomial models results in credible intervals excluding 1.

For preterm (global analysis comparing patients with Covid-19 vs without Covid-19, or subgroup analysis comparing patients with symptomatic vs asymptomatic Covid-19), all frequentist models used in our sensitivity analysis lead to confidence or credible intervals excluding 1, except inverse variance heterogeneity model and Hemni-Copas’s method (table 3, appendix).

But the prediction interval, based on DL method is very broad, ranges from 0.84 to 20.61.

The fragility index ranges from 27 (global analysis with for the DL method) to 10 (subgroup analysis comparing symptomatic vs asymptomatic patients, with REML-HKNHA method). Details on these fragility indexes and quotients are in table 4 (Appendix).

All Bayesian credible intervals also exclude 1.

Finally for stillbirth, the confidence intervals of most frequentist meta-analysis (MA) models contain 1 (table 2).

The prediction interval, based on DL method ranges from 0.52 to 8.49.

The fragility index is 3 for the DL method and 3 with REML-HKNHA method.

As for the Bayesian approach, all six MA models lead to credible intervals containing one.

Interpretation

We performed a large sensitivity study taking into account different approaches (frequentist and Bayesian) and different statistical models in order to estimate several parameters including odds ratio, I2, Tau and their respective confidence intervals.

With the exception of preterm infants, regardless of the frequentist or Bayesian statistical approach, confidence intervals of odds ratio frequently overlap 1 and thus show no association between Covid-19 and the occurrence of eclampsia or stillbirth. The prediction intervals are very wide and contain 1, for all three endpoints (eclampsia, stillbirth and preterm). Finally, the fragility indexes are only 1, 2 or 3 for eclampsia and stillbirth.

Concerning our Bayesian sensitivity analysis, our results are robust to variation of the priors.

Some authors have proposed the use of generalized linear mixed model (GLMM) to perform MA with sparse data (20). We have not used GLMM. Here again, several approaches are available. We considered that our sensitivity analysis was sufficiently broad. Finally, we did not take into account the quality of the studies via risk-of-bias analysis, which is possible through meta-regression.

Conclusion

Given the available data and the results of our broad sensitivity analysis we can only suggest that SARS-CoV-2 infection during pregnancy is associated to preterm, and may be associated to preeclampsia. For stillbirth, more data are needed as none of the Bayesian analyses are conclusive.

Contributions

Ludwig Serge Aho Glele (LSAG) and Emmanuel Simon (EM) contributed to the conception and design of the study.

LSAG performed the statistical analysis.

LSAG, EM,Philippe Kadhel and Paul Sagot drafted the manuscript.

Camille Bouit, Maeva Serrand, Laurence Filipuzzi and Karine Astruc reviewed the manuscript.

All of the authors contributed to the interpretation of the data, gave final approval of the version to be published and agreed to be accountable for all aspects of the work.