Brief communication: A meta-analysis of randomized trials of hydroxychloroquine for the prevention of COVID-19
==============================================================================================================

* Xabier García-Albéniz
* Julia del Amo
* Rosa Polo
* José Miguel Morales-Asencio
* Miguel A Hernán

## Abstract

**Background** Recruitment into randomized trials of hydroxychloroquine (HCQ) for prevention of COVID-19 is being threatened by a widespread conviction that HCQ is not effective for prevention. In the absence of an updated systematic review, we conducted a meta-analysis of randomized trials that study the effectiveness of HCQ to prevent COVID-19.

**Methods and Findings** A search of PubMed, Embase, and medRxiv with expert consultation found four completed randomized trials: two pre-exposure prophylaxis trials and two post-exposure prophylaxis trials. We obtained or calculated the risk ratio of COVID-19 diagnosis for assignment to HCQ versus no HCQ (either placebo or usual care) for each trial, and then pooled the risk ratio estimates. The risk ratio estimated for each of the individual trials were 0.74 (95% CI 0.50-1.10), 0.95 (0.25, 3.64), 0.83 (95% CI: 0.58-1.18), and 0.69 (95% CI: 0.35-1.37). The pooled risk ratio estimate was 0.78 (95% CI: 0.61-0.99) when using either a fixed effect or a standard random effects approach, and 0.78 (95% CI: 0.53-1.16) when using a conservative modification of the Hartung-Knapp random effects approach. All four trials found a similar rate of serious adverse effects in the HCQ and no HCQ groups.

**Discussion** A pooled analysis of these randomized trials cannot rule out a moderate reduction of COVID-19 risk after assignment to HCQ. Yet the findings from the randomized trials were widely interpreted as definite evidence of lack of effectiveness of HCQ, simply because they were not “statistically significant”. Completion of the ongoing trials is needed to generate more precise estimates of the effectiveness of HCQ to prevent COVID-19.

## Background

Hydroxychloroquine (HCQ) is not an effective treatment for established COVID-19 (1-3). Several ongoing studies will help determine whether HCQ is a viable option for prophylaxis of COVID-19. However, recruitment into these studies (some of them carried out by the authors of this report) is being threatened by a widespread conviction that HCQ is not effective for prevention. In the absence of an updated systematic review, we conducted a meta-analysis of randomized trials that study the effectiveness of HCQ to prevent COVID-19 either before known exposure to an infected individual (pre-exposure prophylaxis) or after known exposure to an infected individual (postexposure prophylaxis).

## Methods and Findings

A search of PubMed, Embase, and medRxiv, together with expert consultation, identified four completed randomized trials of prophylaxis with HCQ as of September 30, 2020 (4-7): two pre-exposure prophylaxis trials (4-5) and two post-exposure prophylaxis trials (6-7). For each of the completed trials, we obtained or calculated the risk ratio of COVID-19 for assignment to HCQ versus no HCQ (either placebo or usual care) among PCR-negative individuals at baseline. Our primary analysis is based on the definition of COVID-19 reported in the primary analysis of each study. Also, we tried to harmonize the definition of COVID-19 across studies by conducting separate meta-analyses among those studies that reported “laboratory-confirmed COVID-19 diagnosis” and among those that reported “laboratory-confirmed or clinically-probable COVID-19 diagnosis”.

We calculated the pooled risk ratio estimate and its 95% confidence or compatibility interval (CI) using a fixed effect approach and two types of random effects approaches (8-9). The standard random effects method has been shown to yield an anticonservative (that is, too narrow) 95% CI (9). The alternative Hartung-Knapp random effects method (8) has been shown to generally outperform the standard random effects method (10), but it results in an even more anticonservative 95% CI (narrower than the 95% CI from the standard method) when, as in our meta-analysis, the estimated between-study heterogeneity is small (tau-squared near zero) (11-12). We therefore used the Hartung-Knapp method with an ad hoc modification (13) designed to ensure that its 95% CI remains wider than that of the standard method, even though the resulting 95% CI is expected to be conservative (too wide) when, as in our meta-analysis, the number of studies is small (14). All analyses were conducted in Stata.

The two pre-exposure prophylaxis trials were double-blinded, placebo-controlled trials that included healthcare workers with ongoing exposure to patients with COVID-19. In the first one (available as a preprint) (4), the risk of COVID-19 was 5.9% (58 total cases, 11 confirmed by PCR) among 989 participants assigned to HCQ (two 400 mg doses on day 1, followed by 400 mg either once or twice weekly for 12 weeks) and 7.9% (39 total cases, 6 confirmed by PCR) among 494 participants assigned to placebo. The estimated risk difference was -2.0% (95% CI -4.8 to 0.8) and the risk ratio was 0.74 (95% CI 0.50-1.10). In the second trial (5), the risk of COVID-19 was 6.3% (4 cases confirmed by PCR) among 64 participants assigned to HCQ (three 200 mg doses daily for 8 weeks) and 6.6% (4 cases confirmed by PCR) among 61 participants assigned to placebo. The estimated risk difference was -0.3% (95% CI -8.9, 8.3) and the risk ratio 0.95 (95% CI 0.25-3.64).

The two post-exposure prophylaxis trials included asymptomatic contacts of confirmed COVID-19 cases. The first one was a double-blind, placebo-controlled trial (6) in which the risk of COVID-19 diagnosis (largely based on compatible symptoms) was 11.8% in 414 contacts assigned to HCQ (800 mg. on day 1, 600 mg. daily for four days) and 14.3% in 407 contacts assigned to placebo. The risk difference was -2.4% (95% CI: -7.0 to 2.2) and the risk ratio can be calculated as 0.83 (95% CI: 0.58-1.18). The second one (available as a preprint) (7) was an open-label, cluster-randomized trial in which the COVID-19 risk (based on confirmed PCR) was 3.0% in 958 contacts assigned to HCQ (800 mg. on day 1, 400 mg. daily for six days) and 4.3% in 1042 contacts assigned to usual care. The risk ratio (adjusted for age, gender, region, and time of exposure) was 0.69 (95% CI: 0.35-1.37); the corresponding risk difference cannot be directly calculated from the data provided.

The time from exposure to initiation of prophylaxis was relatively long in the two post-exposure prophylaxis trials: in one trial, about a third of participants were enrolled 4 days after exposure (none was enrolled later) (6); in the other, about 10% of participants were enrolled 7 or more days after exposure (7). For comparison, post-exposure prophylaxis for HIV is recommended in the first 6-72 hours after the exposure (15-16).

The Figure shows the risk ratio estimates from each of the trials and the pooled estimates. The pooled risk ratio of COVID-19 for assignment to hydroxychloroquine vs. no hydroxychloroquine was 0.78 (95% CI: 0.61-0.99) when using either a fixed effect or a standard random effects approach, and 0.78 (95% CI: 0.53-1.16) when using the ad hoc modification of the Hartung-Knapp approach. All four trials found a similarly low rate of serious adverse effects in the HCQ and no HCQ groups (4-7). As expected, the proportion of mild gastrointestinal side effects was greater in the HCQ group.

In an analysis of the outcome “laboratory-confirmed COVID-19”, the pooled risk ratio estimate was 0.88 (95% CI: 0.56-1.37) or 0.88 (95% CI: 0.42-1.80); in an analysis restricted to the three trials that reported “laboratory-confirmed or clinically-probable COVID-19” (4, 6-7), the pooled risk ratio estimate was 0.89 (95% CI: 0.73-1.08) or 0.89 (95% CI: 0.57-1.38) (Supplementary Figures 1 and 2).

## Discussion

When considered together, the available randomized trials yield a point estimate of an approximately 20% lower risk of COVID-19 for assignment to HCQ compared with no HCQ among PCR-negative individuals at randomization. Any effect between a 50% reduction in risk and a small increase in risk is highly compatible with the data from these trials.

When we attempted to harmonize the definition of COVID-19, the point estimates were closer to the null value than those of our primary analysis, with a similar or greater level of uncertainty. However, the available information was insufficient to adequately harmonize the outcomes, and thus our attempt at harmonization may have introduced some degree of selection bias or misclassification bias. Estimates for laboratory-confirmed COVID-19 are difficult to interpret because the limited access to testing for participants in two of the studies (4,6) implies that their estimates are based on a nonrandom subset of the cases that should have been laboratory confirmed. Also, some test results were self-reported. Estimates for “laboratory-confirmed or clinically-diagnosed COVID-19” are difficult to interpret because this definition includes only symptomatic individuals in two of the studies (4,6), but also asymptomatic individuals with a positive PCR in another one (7). Ideally, all studies would have used the same primary outcome as the trial by Mitjà et al. (7): symptomatic COVID-19 with laboratory confirmation.

Since our pooled estimates are very imprecise, we would need additional studies to draw more precise inferences about the prophylactic effect, if any, of HCQ. Unfortunately, recruitment for those studies is being impeded by incorrect interpretations of the available randomized trial evidence. The individual findings from the four completed trials have been widely (and incorrectly) portrayed as definite evidence of the lack of effectiveness of HCQ, simply because they were not “statistically significant”. Thus, the common confusion between the concepts “no effect” and “no statistical significance” have led many to conclude that HCQ has no prophylactic effect (which may be a true but premature conclusion) when we can only conclude that the effect estimate is still too imprecise (17-18).

Our pooled estimates are based on the design and analytic choices made by the investigators of each trial, sometimes in a context of limited access to testing for SARS-CoV-2. We encourage the investigators from these and future prophylaxis trials to coordinate analyses of individual-level data with standardized outcome definitions; corrections for differences in length of follow-up, treatment dose, and time since exposure; adjustment for losses to follow-up and other deviations from protocol (19); and adoption of a more causally-interpretable meta-analytic approach (20). We also encourage the authors of living systematic reviews to update their findings frequently when, as in this case, public opinion interferes with the generation of the very evidence that will feed their systematic review.

In summary, pending final publication of two of them, these randomized trials cannot rule out a moderate reduction of COVID-19 risk after assignment to HCQ. Completion of the ongoing trials is needed to generate more precise estimates of the potential effectiveness of HCQ to prevent COVID-19.

## Supporting information

Supplementary Material [[supplements/203869_file02.pdf]](pending:yes)

## Data Availability

Data used in the analysis are publicly available

![Figure.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2020/10/03/2020.09.29.20203869/F1.medium.gif)

[Figure.](http://medrxiv.org/content/early/2020/10/03/2020.09.29.20203869/F1)

Figure. 
Risk ratio estimates of COVID-19 for hydroxychloroquine vs. no hydroxychloroquine in randomized trials of pre- and post-exposure prophylaxis, pooled and by study. These estimates are based on the definition of COVID-19 reported in the primary analysis of each study.

## Acknowledgments

We thank Prof. Sander Greenland for technical advice and Prof. Richard Riley, Dr. Tim Morrison, and Mr. David Fisher for constructive feedback on estimation methods for random effects meta-analysis.

## Footnotes

*   An additional clinical trial was added to the meta-analysis

*   Received September 29, 2020.
*   Revision received October 3, 2020.
*   Accepted October 3, 2020.


*   © 2020, Posted by Cold Spring Harbor Laboratory

This pre-print is available under a Creative Commons License (Attribution-NonCommercial-NoDerivs 4.0 International), CC BY-NC-ND 4.0, as described at [http://creativecommons.org/licenses/by-nc-nd/4.0/](http://creativecommons.org/licenses/by-nc-nd/4.0/)

## References

1.  1.WHO discontinues hydroxychloroquine and lopinavir/ritonavir treatment arms for COVID-19. [https://www.who.int/news-room/detail/04-07-2020-who-discontinues-hydroxychloroquine-and-lopinavir-ritonavir-treatment-arms-for-covid-19](https://www.who.int/news-room/detail/04-07-2020-who-discontinues-hydroxychloroquine-and-lopinavir-ritonavir-treatment-arms-for-covid-19) (accessed October 2, 2020)
    
    

2.  2.Statement from the Chief Investigators of the Randomised Evaluation of COVid-19 thERapY (RECOVERY) Trial on hydroxychloroquine. Available at: [https://www.recoverytrial.net/files/hcq-recovery-statement-050620-final-002.pdf](https://www.recoverytrial.net/files/hcq-recovery-statement-050620-final-002.pdf) (accessed October 2, 2020)
    
    

3.  3.Horby P, Mafham M, Linsell L, et al., Effect of Hydroxychloroquine in Hospitalized Patients with COVID-19: Preliminary results from a multi-centre, randomized, controlled trial. RECOVERY Collaborative Group. MedRxiv preprint doi: [https://doi.org/10.1101/2020.07.15.20151852](https://doi.org/10.1101/2020.07.15.20151852)
    
    

4.  4.Rajasingham R, Bangdiwala AS, Nicol MR, et al. Hydroxychloroquine as pre-exposure prophylaxis for COVID-19 in healthcare workers: a randomized trial. medRxiv. doi:2020:2020.09.18.20197327.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=2020:2020.09.18.20197327&link_type=DOI) 

5.  5.Abella BS, Jolkovsky EL, Biney BT, et al. Efficacy and Safety of Hydroxychloroquine vs Placebo for Pre-exposure SARS-CoV-2 Prophylaxis Among Health Care WorkersA Randomized Clinical Trial. JAMA Intern Med.2020. doi:10.1001/jamainternmed.2020.6319
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jamainternmed.2020.6319&link_type=DOI) 

6.  6.Boulware DR, Pullen MF, Bangdiwala AS, et al. A Randomized Trial of Hydroxychloroquine as Postexposure Prophylaxis for Covid-19. N Engl J Med 2020;383(6):517–25.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1056/NEJMoa2016638&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F10%2F03%2F2020.09.29.20203869.atom) 

7.  7.Mitja O, Ubals M, Corbacho M, et al. A Cluster-Randomized Trial of Hydroxychloroquine as Prevention of Covid-19 Transmission and Disease. medRxiv. doi:2020:2020.07.20.20157651
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=2020:2020.07.20.20157651&link_type=DOI) 

8.  8.Hartung J, Knapp G. A refined method for the meta analysis of controlled clinical trials with binary outcome. Stat Med 2001; 20 (24): 3875–3889.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/sim.1009&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=11782040&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F10%2F03%2F2020.09.29.20203869.atom) 

9.  9.DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials 1986; 7 (3): 177–188
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/0197-2456(86)90046-2&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=3802833&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F10%2F03%2F2020.09.29.20203869.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1986F013900001&link_type=ISI) 

10. 10.IntHout J, Ioannidis JPA, Borm GF. The Hartung-Knapp-Sidik-Jonkman method for random effects meta-analysis is straightforward and considerably outperforms the standard DerSimonian-Laird method. BMC Medical Research Methodology 2014; 14:25
    
    

11. 11.Wiksten A, Rücker G, Schwarzer G. Hartung–Knapp method is not always conservative compared with fixed-effect meta-analysis. Statistics in Medicine 2016; 35: 2503–2515
    
    

12. 12.Jackson D, Law M, Rücker G, Schwarzer G. The Hartung-Knapp modification for random effects meta-analysis: a useful refinement but are there any residual concerns? Statistics in Medicine 2017; 36: 3923–3934
    
    

13. 13.Röver C, Knapp G, Friede T. Hartung-Knapp-Sidik-Jonkman approach and its modification for random-effects meta-analysis with few studies. BMC Medical Research Methodology 2015; 15:99
    
    

14. 14.Parlett C, Riley RD. Random effects meta-analysis: Coverage performance of 95% confidence and prediction intervals following REML estimation. Statistics in Medicine 2017: 36: 301–317
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/sim.7140&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=27714841&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F10%2F03%2F2020.09.29.20203869.atom) 

15. 15.Updated U.S. Public Health Service guidelines for the management of occupational exposures to HIV and recommendations for postexposure prophylaxis. U.S. Public Health Service Working Group. Department of Health & Human Services. Available at: [https://stacks.cdc.gov/view/cdc/20711](https://stacks.cdc.gov/view/cdc/20711)
    
    

16. 16.Polo Rodriguez R, Lozano F, González de Castro P, et al. Consensus Document on post-exposure prophylaxis against HIV, HBV and HCV in adults and children. Enferm Infecc Microbiol Clin 2016; 34(2):121.e1-15.
    
    

17. 17.Greenland S, Senn SJ, Rothman KJ, et al. Statistical Tests, P-values, Confidence Intervals, and Power: A Guide to Misinterpretations. Eur J Epidemiol 2016; 31 (4):337–50
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s10654-016-0149-3&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=27209009&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F10%2F03%2F2020.09.29.20203869.atom) 

18. 18.Amrhein V, Greenland S, McShane B. Scientists rise up against statistical significance. Nature 2019;567:305–307
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1038/d41586-019-00857-9&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=30894741&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F10%2F03%2F2020.09.29.20203869.atom) 

19. 19.Hernán MA, Robins JM. Per-protocol analyses of pragmatic trials. N Engl J Med 2017; 377(14):1391–1398
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=doi:10.1056/NEJMsm1605385&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=28976864&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F10%2F03%2F2020.09.29.20203869.atom) 

20. 20.Dahabreh IJ, Petito LC, Robertson SE, Hernán MA, Steingrimsson J. Towards causally interpretable meta-analysis: transporting inferences from multiple randomized trials to a new target population. Epidemiology 2020; 31(3):334–344