Abstract
BACKGROUND A recent trial (NCT04308668) found that post-exposure prophylaxis with hydroxychloroquine (HCQ) was associated with a reduced incidence of Covid-19 by 17% overall; 36% in younger subjects, 31% in household contacts and 49% given within one day. To understand these trends, we re-analyzed the released dataset.
METHODS Our protocol conformed to the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT). We compared the incidence of Covid-19 after HCQ or placebo, stratifying by intervention lag, age, and gender.
RESULTS Requesting additional data, we found that 52% of subjects received medication 1-2 days after the intended overnight delivery; 19% of them outside the intended four-day intervention lag. After re-analysis, there was a reduced incidence of Covid-19 associated with HCQ compared with placebo (9.6% vs. 16.5%) when received Early (up to 3 days) after exposure (RR 0.58, 95%CI 0.35 - 0.97; p=0.044; NNT 14.5) but not Late (RR 1.22, 95%CI 0.72 - 2.04).
We found a significant HCQ-associated Covid-19 reduction in subjects 18 to 45 years old with Early (RR 0.54, 95%CI 0.29-0.97; p=0.0448, NNT 11.5) but not Late (RR 1.02, 95%CI 0.55-1.89) prophylaxis, attenuated in older subjects (RR 0.75, 95%CI 0-27-2.05) and by co-morbidities. There were reductions associated with Early prophylaxis in household contacts (RR 0.35, 95%CI 0.13-0.89; p=0.025, NNT 5.7) and Health Care Workers (RR 0.74, 95%CI 0.4-1.38). We did not detect effects of gender, folate, zinc, or ascorbic acid.
CONCLUSIONS Using novel data with a prospective post hoc re-analysis, hydroxychloroquine, in an age-dependent manner, was associated with reduced illness compatible with Covid-19 or confirmed infection when supplied for post-exposure prophylaxis between 1 and 3 days after high-risk or moderate-risk exposure, at higher loading and maintenance doses than in similar studies. This finding warrants prospective confirmation.
Registered with the Open Science Framework (last revised September 27, 2020, osf.io/fqtnw).
Short Summary A prospective re-analysis of a public dataset integrated with novel data found an HCQ-associated reduction of illness compatible with Covid-19 when received between 1 and 3 days after a high-risk or moderate-risk exposure (RR 0.58, 95% CI 0.35-0.97, p=0.044, NNT14.5).
Plain Language Summary A recent clinical trial examined the ability of hydroxychloroquine (HCQ) to prevent Covid-19 just after an exposure to a person confirmed to have Covid-19. There was an HCQ-associated reduction of Covid-19 by an overall 17%; 36% in younger subjects, and 49% in subjects given HCQ within one day of being exposed. Likely because the study had too few patients to find what may have been a medically and economically meaningful, reduction, this effect was not statistically significant.
Studying the trial data, we discovered an unintended and variable delay in the delivery of study drug which may have masked any drug effect. The investigators provided further information at our request that confirmed our theory. About half of the participants received drugs one or two days later than intended, about a fifth beyond the four days the investigators thought the drug might work.
When we factored in this new information, we found that if HCQ was given early (up to three days after exposure), it was associated with a statistically significant 42% reduction of Covid-19. Giving HCQ later had no effect. There was a greater effect in younger (less than 45 years) rather than older subjects (47% vs. 25%). Gender did not seem to affect the results, but there was a greater HCQ-associated reduction (65%) when it was given early to people exposed to Covid-19 in a household environment rather than to health care workers (26%). The effects associated with HCQ were better in people without co-existing conditions.
These re-calculations are important because the study, as originally analyzed, was the only randomized study that dealt with preventing Covid-19 cited by FDA to support a key public health decision made in June 2020 regarding HCQ. Although other studies have shown that the drug is not effective to treat established cases of Covid-19, our research suggests that that it is effective for prevention. Other prevention studies have failed to show a benefit of HCQ, possibly because they have used lower doses or have estimated the timing of dosing differently. Our research paves the way for our result to be confirmed under clinical trial conditions and for a re-examination of public health policy regarding this drug. Even with the introduction of vaccination, there remains a need for approaches like this to prevent Covid-19 while individual and community immunity develops, especially in subjects given a lower priority for vaccination.
Introduction
There have been (as of December 11, 2020) over 69.1 million cases of Covid-19 and over 1.57 million deaths worldwide,1 about one fifth of them within the USA.2 With early interest in deploying hydroxychloroquine (HCQ), the Food and Drug Administration (FDA) issued an Emergency Use Authorization (EUA) in March 2020.3 Lacking randomized clinical trial (RCT) data, emerging observational reports (with exceptions4) disfavored HCQ.5 With safety concerns, FDA cautioned6 against using HCQ outside hospital or trial settings on April 24.
HCQ became highly controversial, with suggestions that “to some extent the media and social forces — rather than medical evidence — are driving clinical decisions and the global Covid-19 research agenda.”7 Against this background, on June 15, FDA revoked3 HCQ’s EUA, citing only two just-published RCTs. The RECOVERY Trial8 was cited as offering “persuasive evidence of a lack of benefit of HCQ in the treatment of hospitalized patients.”
The second and only study9 addressing prevention examined post-exposure prophylaxis (the “PEP” study) with HCQ in 821 asymptomatic adults with a household or occupational exposure to Covid-19. Subjects received HCQ (1.4g first day, then 600 mg daily for 4 more days) or placebo (folate - USA; lactose - Canada). The study concluded that “…HCQ did not prevent illness […] when initiated within 4 days after […] exposure” (HCQ 11.8% vs. placebo 14.3%; RR 0.83, 95%CI 0.58-1.18, p=0.35).
We10 and others have criticized the study’s interpretation. Since this was a pragmatic trial, with typically greater heterogeneity and smaller effect sizes than in an explanatory trial,11 powering the study to detect a 50% reduction in Covid-19 may have been over-ambitious, especially given its early termination.12 A reduction of 17% is arguably13 clinically meaningful.14 Rather than targeting specific clinical goals, the authors suggested15 that the study was primarily powered to collect data quickly under pandemic conditions.
Non-statistically significant signals of HCQ-associated efficacy included an a 31% reduction among household cohabitees. There were age-dependent reductions found in other analyses16 to be statistically significant. The folate placebo and ex-protocol use of zinc and ascorbate may have been confounding (Supplement). With a reduction of 49% associated with early (“Day 1”) HCQ prophylaxis, we10 and others17 found a negative association between intervention lag and reduction of Covid-19.
We conjectured that post hoc exploratory re-analysis of the PEP study would inform a time- and age-nuanced approach to Covid-19 using HCQ, testable prospectively. Our objectives were to define: (a) any time- or (b) age-dependent effects associated with HCQ and, (c) any influence of gender, exposure type, use of zinc, ascorbate or folate on outcomes.
Methods
Dataset and Protocol Revisions
One protocol (NCT04308668) described separately reported PEP9 or early post-exposure treatment (PET)18 cohorts. The de-identified PEP dataset was released (covidpep.umn.edu/data) with revisions: September 9 (“9/9”), October 6 (“10/6”) and October 30 (“10/30”) 2020.
Using the Open Science Framework (OSF) protocol template (osf.io/jea94/), we conformed to the SPIRIT checklist (Standard Protocol Items: Recommendations for Trials19) integrating the WHO Trial Registration Data Set.20 Our protocol was registered on August 13, 2020 with revisions (Supplement), most recently September 27, 2020 (osf.io/vz8a7/10) prior to receiving data regarding the time to drug receipt in the 10/6 revision.
Four main areas required clarification (Supplement) related to: (i) exposure risk definition; (ii) study medication adherence; (iii) “intervention lag” (time from exposure to receipt of medication, resolved by the 10/6 revision); (iv) nomenclature for timing study events. Counting the date of highest reported exposure to Covid-19 as “Day 1,” this adopted clarification yields some inconsistencies with the original paper indicating the occurrence of study events to be one day later.
Analysis Plan
We re-stratified data by intervention lag and then by age, gender, exposure type, risk level, or use of zinc or ascorbate. An Intent-to-Treat (ITT) analysis was employed as in the original study. We analyzed data according to adherence to taking study medication, provision of outcome data, use of the folate placebo, and presence of co-morbidities (Supplement).
We retained the original primary outcome variable
“incidence of either laboratory-confirmed Covid-19 or illness compatible with Covid-19 within 14 days,” comparing treatment arms using Fisher’s Exact test. We examined the severity of symptoms at 14 days according to a visual analogue scale originally described as a secondary outcome (Kruskal-Wallis test).
We mirrored the original use of two-tailed tests without adjustment for multiple comparisons. This is further justified by the exploratory nature of our analyses. p-values ≤ 0.05 were considered statistically significant. Larger values are presented to identify trends. Microsoft Excel was used for data processing. Vassar Stats (vassarstats.net/) was used for verification. The original authors provided two calculations from which we verified our primary time stratification (Supplement).
Ethics Committee Approval
No ethics committee approval was required as we used a de-identified, publicly released dataset.
Results
Considering shipping schedules, we estimated10 that within each of the reported9 strata for “Time from exposure to enrollment” (1 to 4 days), there were overlapping variations in intervention lag. For example, some “Day 1” (range 1.4-4.4 days) and “Day 4” (range 4.4-7.4 days) subjects may have received drug after the same interval.
New data (9/9 revision) provided at our request broadly confirmed these estimates revealing a reduction in Covid-19 associated with HCQ given within 2 days of exposure (RR 0.35, 95%CI 0.13 – 0.93; p=0.0438) but not later (RR 0.98, 95%CI 0.67 – 1.45).10 Recognizing limitations (Supplement) to these estimates, further detail was requested and provided (10/6 revision). The PEP study protocol had intended to enroll only those receiving drug within 4 days from exposure, assuming overnight shipping. We found that 332 and 95 subjects (52% of all subjects) received medication one or two days later than this respectively, with 152/821 (19%) subjects receiving drug outside the intended 4-day window (Table S 3).
We stratified subjects according to the intervention lag, in keeping with the declared aim of the original protocol. We found an HCQ-associated reduction in Covid-19 when received “Early” between 1 and 3 days after exposure from 16.5% to 9.6% (RR 0.58, 95%CI 0.35 – 0.97; p=0.044; NNT 14.5) but not later (“Late”) (RR 1.22, 95%CI 0.72 – 2.04) (Table 1). We did not detect differences in symptom severity scores for either time strata (Supplement). A comparison of the demographic and clinical characteristics of the two groups shows a largely conserved balance between the Early and Late cohorts (Table 2, Table S 1).
Stratification of effect associated with hydroxychloroquine based on time from exposure to drug receipt (ITT population)
Demographic and clinical characteristics, stratified into Early and Late Cohorts
The data for the original cohort recreates data from the original paper, for comparison and quality control purposes. Several variables have been added. The data are stratified into Early (1-3 days) and Late (4-6 days) post exposure prophylaxis cohorts.
(I/S/%) - Shown in parentheses are interquartile ranges (1stand 3rdquartile), or standard deviations where indicated. All other values within parentheses indicate the percent contribution to the cohort total. See Table S 1 for full list of demographic and clinical characteristics.
Adopting the same age strata as the PEP study, we found in the Early cohort non statistically significant Risk Ratios of 0.53 (18-35 years), 0.52 (36-50 years), and 2.80 (> 50 years). With no a priori reason for selecting these strata, the data are less subjectively supportive of two age strata. Conservatively (Supplement), we set the boundary at 45 years. We found HCQ-associated reductions of Covid-19 when given Early in both younger (18-45 years) (RR 0.54, 95%CI 0.29-0.97; p=0.0448, NNT 11.5) and older (>45 years) (RR 0.75, 95%CI 0-27-2.05) subjects. Within the Early and Late cohorts, further stratification revealed no obvious gender-dependent effects (Table S 4).
Considering only subjects reporting no co-morbidities (particularly excluding asthma and co-morbidities classified as “other”), suggested stronger effects associated with HCQ within time- or age-related strata (Supplement).
Numerical differences in HCQ-associated effects of Early prophylaxis in the whole cohort between Household (RR 0.69) and HCW (0.92) contacts reached statistical significance after time-stratification with a reduction of Covid-19 in household contacts (RR 0.35, 95%CI 0.13-0.89; p=0.025, NNT 5.7, Table 4). Differences in the baseline incidence of Covid-19 and the relationship between contact type, risk level and changes in risk definitions are described in the Supplement.
Stratification of effect associated with hydroxychloroquine by age based on time from exposure to drug receipt (ITT population)
Stratification of effect associated with hydroxychloroquine by exposure type based on time from exposure to drug receipt (ITT population)
Directionally similar results were obtained after excluding subjects not contributing outcome data and subjects who did not take study medication. Stratifying into Early and Late prophylaxis cohorts revealed no discernible effect associated with folate (Supplement). With poorly detailed observational data, there did not appear to be an effect associated with zinc or ascorbate (Supplement). The use of zinc and ascorbate appears balanced between the groups both for the whole cohort and the Early and Late time strata (Table 1).
Discussion
In tackling our primary objective of defining any temporal effect of HCQ, we understood that HCQ prophylaxis had been “initiated within 4 days after […] exposure.”9 Although others,13,16,17 including the authors of NIH21 guidelines and the editorial7 accompanying the paper, shared this understanding, it requires revision for two main reasons.
Firstly, many participants received medication after the intended overnight delivery or after four days from exposure. A similar issue likely pertains to the companion PET study.18 Secondly, inconsistent terminology led to an overestimate by one day of the time from exposure to enrollment or to drug receipt.
Correcting these understandings yields a statistically significant reduction of Covid-19 by 42% associated with HCQ received between 1 and 3 days after exposure, but not later. The early use of HCQ is supported by estimates for an incubation period of 3-8 days.22
We found an age-dependent, statistically significant reduction of Covid-19 associated with Early prophylaxis. Re-analyzing the same PEP dataset without time stratification, we confirmed (Supplement) Luco’s16 report of HCQ-associated reductions in Covid-19 in subjects younger than 50 years reaching statistical significance in the high-risk exposure cohort. Interpretation of age-related effects is limited by a poorly understood relationship between age and susceptibility to Covid-19, increases in incubation period with age23 and a low representation of older subjects with a low baseline incidence of Covid-19.
Small population sizes within the co-morbidity subgroups prompt cautious interpretation. However, the presence of co-morbidities attenuated age- and time-dependent HCQ-associated effects. Although age-related responses associated with HCQ may be related to co-morbidity,16 excluding co-morbid patients did not yield equivalent effects in age strata. Asthma and co-morbidities classified as “other” contributed most to attenuating the HCQ-associated response. This finding is supported by the application of Multiple Correspondence Analysis and the Mantel test to the same dataset by Luco16 who described confounding clinical differences between arms particularly regarding asthma and “other” co-morbidities.
Mirroring the original data, we found a significant HCQ-associated effect in household contacts. This result may reflect differences in access to advanced PPE, hygiene training, likely multiple exposures, and the ability to quarantine after exposure. Thus, household contacts in this study may share much with first responders in the companion PrEP study,24 where a 64% reduction in Covid-19 associated with HCQ was observed (combined dose groups). Further, the changing risk definitions and their apparently inconsistent application between contact types may confound understanding of how contact type and risk level affect Covid-19 development. Whether the apparent lack of an HCQ-associated effect in moderate-risk exposures is a statistical aberration or is biologically meaningful is unclear.
With a small “no folate” cohort, we did not detect an effect of folate. Due a paucity of data, we could not determine whether there was an effect of zinc or ascorbate other than noting no differences associated with HCQ in subjects taking neither agent and the entire ITT cohort. Based on the apparently balanced use of zinc and ascorbate in the stratified cohorts, confounding due to these agents seems unlikely.
Our findings are made in the climate of concern25 for the reliability of publications related to Covid-19 and the controversy surrounding HCQ.26,27 This is partly fueled by a widening understanding of Covid-19 pathogenesis and the multiple, sometimes stage-dependent, mechanisms proposed for HCQ.28
In hospitalized patients RCT findings29 evincing HCQ’s ineffectiveness are supported by observational studies, notably two4,30 that report significant HCQ-associated reductions in mortality only with the use of zinc. The possibly synergistic use of steroids31 may further confound some studies. At earlier stages, any HCQ-associated effect appears independent of zinc, evinced (weakly) by the lack of synergy we observed. Further, using zinc may be futile in otherwise healthy, especially younger subjects with no zinc deficiency or dysregulation.
For prophylaxis, understanding differences in the ability to quarantine, testing methods, co-morbidities, and the possibility of multiple rather than single “index” exposures, appear important in reconciling apparently conflicting studies. Relying on different pharmacokinetic models, differences in dosing and intervention lag may be particularly significant (Table 5).
Comparison of HCQ-associated effects after early prophylaxis in similar PEP studies
Our findings are consistent with those of an Indian study32 in which subjects mostly exposed to laboratory confirmed COVID-19 cases could opt to receive standard care alone or HCQ prophylaxis. With lower loading and maintenance doses than in the PEP study (Table 5), there was an HCQ-associated 45% reduction of Covid-19 at 4 weeks. A single “index” exposure could not be identified. Subjects enrolled after the primary case positive report and prophylaxis began at the earliest within 48 hours of knowing about the high-risk contact (D. Dhibar personal communication).
A well-executed cluster-randomized trial33 from Spain reported a small HCQ-associated reduction of Covid-19 (aRR 0.86, 95%CI 0.52-1.42), with the same loading dose as the Indian32 study but more frequent dosing over a shorter period (Table 5). The mean age was higher (48.6 years) than in the PEP study (41.6 years) where we have described possible age-dependent effects. There were poor HCQ-associated effects with an intervention lag of <= 3 days (RR 0.89), 4-6 days (RR 0.93) and later (RR 4.09). Notably, there were differences according to PCR testing status at baseline. In PCR-positive subjects there was no HCQ-associated effect (aRR 1.02; 95%CI 0.64–1.63), whereas for PCR-negative patients there was an effect signal (aRR 0.68, 95%CI 0.34–1.34). A change in PCR status is likely to be a function of the moment beyond which a drug is unlikely to be effective, possibly more accurately than an estimate of intervention lag. Accordingly, this study supports our findings suggesting a beneficial effect of early intervention.
Another well-executed household-randomized trial34 examined the effect of HCQ given to subjects (PCR negative at baseline) exposed to an infected person. Daily, subjects self-collected nasal swabs for PCR testing. By day 14, no difference in infection rate was associated with HCQ even for intervention lags under 72hrs. Underestimation of intervention lag using the time to dosing from last contact, rather than from first contact, may mask the ability to observe any time-dependent effect. As the authors acknowledge, the dosing, which is the lowest among comparable studies (Table 5), may have been insufficient. The substantive lack of a loading dose may have extended the intervention lag beyond an effective range. The possibly anti-viral effect of the ascorbate35 placebo may have confounded the results. Although data were not age-stratified, the populations was of a similar age to the PEP study, but with a higher representation of Hispanic subjects.
Differences in HCQ-associated responses between “Early” and “Late” PEP illustrate the poorly defined position that PEP occupies on a continuum between PrEP and PET. The variable possibility of other exposures occurring before and after a single “index” exposure means that the Early PEP cohort has much in common with the population of the companion PrEP study.24 Although the PrEP study was hampered by poor recruitment, once or twice weekly use of HCQ (after a loading dose) in HCW was associated with reduced development of Covid-19 by 27%, compared with folate placebo (HR 0.73, CI 0.48-1.09, p=0.12, combined groups). Age related HCQ-associated effects were of a similar order of magnitude (34-45%) in the PEP and PrEP studies (Supplement Table S 6), consistent with the findings of an observational study involving mainly younger HCW.36 Comparison with another PrEP RCT37 in HCW, with a very small population (n=132), early termination, and a low incidence of Covid-19, lacks meaning.
Limitations
Given the enormity of the pandemic, we adopted a drug-development approach to glean efficacy signals possibly lost in the original study due to Type II errors related to underpowering, early termination and an environment of poorly understood pathogenesis. This could well justify shifting the threshold of toleration for Type I errors and other statistical challenges related to sub-group analysis.38 These are partially offset by our use of two-sided tests, when directionality in the original data may have justified otherwise. Our primary time stratification based on newly-acquired data essentially represents the a priori analysis intended by the original authors. Nonetheless, this does remain a post hoc study; results should be interpreted cautiously, and hypotheses should be tested in prospective studies sufficiently powered to accommodate multiple comparisons in sub-groups.
Our study retains the limitations acknowledged by its original authors related to the availability and access to testing, the use of a clinical case definition of Covid-19, the reliance on self-reported data and the generally young population studied. There are other limitations. The study poorly represents African-American and Hispanic or Latino populations. The rapidly executed study overcame several logistical challenges to allow the collection of real-world data, having both advantages and disadvantages of a pragmatic design.11,39 Self-selection bias inherent in this type of study may have been compounded by FDA cautions regarding HCQ.6 Unlike similar studies,33,34 the PEP study was not cluster-randomized.39
Limitations not already discussed relate to the estimation of the interval between exposure and treatment with 24-hour windows of uncertainty on either side. The earlier window is due to subjects providing only the date of their highest risk exposure. The later window is due to de-identification of shipping data, and the unknown interval before ingesting the first dose. The original authors (personal communication) attempted to minimized this by delivering medication to where the participant knew they would be at its expected arrival time.
Time-related or other biases may be associated with the exclusion of 100 randomized subjects who became symptomatic before medication was received and aggregated into the companion treatment study.18 Lastly, analysis of the effect of risk level is confounded by an inability to discriminate between nuances within the risk categories
Conclusions
Analysis40 of the PEP,9 companion24,18 and other33,34 studies raise no significant safety concerns for using HCQ in the populations studied. Integrating a public dataset with new unpublished data, we found that, especially in younger subjects, hydroxychloroquine was associated with significantly reduced illness compatible with Covid-19 when initiated between 1 and 3 days after a high-risk or moderate-risk exposure at higher loading and maintenance doses than in similar studies. With relevance to pre-exposure prophylaxis, these findings warrant prospective confirmation until individual and community immunity is achieved with widespread availability of vaccination.
Funding and Conflicts of Interest
There is no external support for this study and the sponsor is entirely responsible for its design and conduct. The sponsor and principal investigator have no financial or other conflicts of interest in the subject matter of this study. DMW is the president of Synechion, Inc. and KevMed, LLC, providing services for the medical industry and marketing medical products, respectively, outside of the area of this work. See ICMJE forms for further details. DM is the president of ZSX Medical, LLC. developing surgical devices and a Principal at Third Eye Associates, a technical consulting company. PK and SAS report no conflicts.
Data Sharing
Microsoft Excel files will be available on reasonable request up to one year after publication.
Acknowledgements
We thank Dr. David Boulware and his colleagues for clarifying our questions, providing insight into their study, collecting additional data at our request related to shipping times and providing confirmatory calculations for our analysis. We also thank Drs. Marcio Watanabe, Juan Luco and Philip Lavin for their valuable comments. This acknowledgment does not imply endorsement of our work.
Footnotes
The following revisions have been made since version 1 of this manuscript28 was pre-printed December 2, 2020. i)Additional comments related to just published full paper version of Barnabas et al.,29 previously cited as abstract version. Discussion of dose and loading dose effects between comparable studies, with added Table 5. ii)Revision of discussion to reflect age strata revision in PrEP study from pre-print to published versions We had noted in our pre-printed protocol22 that the age strata used in the pre-printed companion PrEP study24 differed from those used in the companion PEP27 and PET30 studies. These strata have been revised in the published version25 to match the other two studies. The pre-printed Version 1 of this manuscript also reflected the earlier age strata, which we now revise and add a comparison (Table S 6) between the PEP and PrEP studies. iii)Table (Table S 5) added for effect of HCQ by risk level and contact type. Differences in the baseline incidence of Covid-19 and the relationship between contact type, risk level and changes in risk definitions are further discussed in main body and Supplement. iv)Discussion revised concerning statistical limitations. v)Comments added regarding remaining need for non-vaccine approaches. vi)Citation of Mitja31 study: Some RR values changed to aRR to reflect adjustment noted in Mitja Table 2 and apparent discrepancy with values in Mitja Figure 1. vii)Table 4: non-stratified totals added viii)Additional discussion in limitations. ix)Use of the term intervention lag to denote the time between highest risk exposure and receipt of study medication. x)Updated citations from pre-print to published versions. xi)Various typos, minor grammatical errors, reduction of word count. xii)Typo in Luco analysis, p value corrected to 0.0293 from 0.293. xiii)Verification calculations: minor differences likely due to rounding noted (Supplement) xiv)Description of selection of strata boundaries for age - expanded and moved form man text. (Supplement)