Safety of hydroxychloroquine, alone and in combination with azithromycin, in light of rapid wide-spread use for COVID-19: a multinational, network cohort and self-controlled case series study

Study design

Two study designs were developed and executed across a multinational, distributed database network. First, new user cohort studies were used to estimate the safety of HCQ compared to sulfasalazine (SSZ), and to assess the risks associated with the addition of AZM compared to amoxicillin (AMX) amongst users of HCQ in patients with rheumatoid arthritis (RA). SSZ and AMX were chosen as active comparators as they have similar indications as the target treatments (HCQ and AZM respectively). As a secondary analysis, self-controlled case series (SCCS) was used to estimate the safety of HCQ in the wider population, including uses for non-RA indications.

Data sources

Electronic health records and administrative claims databases from primary care and secondary care containing participants from Germany, Japan, Netherlands, Spain, the UK, and the USA were analysed in a distributed network, and are detailed in the Supplementary Appendix, Table S1.

Observational healthcare databases mapped to the Observational Medical Outcomes Partnership (OMOP) common data model collaborated in an international effort with the Observational Health Data Science and Informatics (OHDSI) community.^37,38 De-identified or pseudonymised data were obtained from routinely collected records from clinical practice in Germany, Spain, the UK, Japan, and the USA. Studies were performed locally and no patient level data shared using the following databases: IQVIA Disease Analyser Germany EMR (ambulatory EMR from Germany); JMDC (Japanese claims); IPCI (primary care EMR from Netherlands); SIDIAP (primary care EMR from Spain); CPRD and IMDR (primary care EMRs from UK); and CCAE, Optum, MDCR, MDCD, PanTher, IQVIA OpenClaims, Veteran Affairs (VA), and IQVIA US Ambulatory EMR (USA). SCCS were conducted on a subset of these as a secondary analysis: CCAE, CPRD, Optum, MDCD, and MDCR. Rather than pooling these data assets, all analyses were conducted in a distributed network, where analysis code was sent to participating sites and only aggregate summary statistics were returned, with no sharing of patient-level data between organizations.

Study Period and Follow-up

The study period started from 01/09/2000 and ended at the latest available date for all data sources in 2020. Follow-up for each of the cohorts started at an index date defined by the first dispensing or prescription of the target/comparator drug as described in the cohort definitions (Supplementary Table 2.1). Two periods were considered to define time-at-risk. First, for an intention-to-treat analysis, follow-up started one day after the index date and continued up until the first of: outcome of interest, loss to follow-up, or 30 days after the index date to resemble the likely duration of COVID-19 treatment regimens.²³ Secondly, for an on-treatment analysis, follow-up started one day after the index date and continued until the earliest of: outcome of interest, loss to follow-up, or discontinuation, with an added washout time of 14 days. Continued use of a same treatment was inferred by allowing up to 90-day gaps between dispensing or prescription records.

In the HCQ versus SSZ study, the index event was defined as the first recorded dispensing or prescription of the drug in a patient’s history. For the study of HCQ combined with AZM, follow up started when the second of the two co-administered treatments was initiated while still exposed to the first treatment (e.g. when AZM started during a period of HCQ use, or when HCQ started during a period of AZM use). HCQ use was assumed to be chronic in the management of RA, and AZM was assumed an acute prescription for infection treatment, and therefore inferred persistent exposure to AZM was assessed by allowing up to 30 days between dispensing or prescription records. Cohorts of combined HCQ and amoxicillin were generated using these same rules as an active comparator.

For SCCS, periods of inferred persistent exposure to HCQ were generated by allowing up to 90-day gaps between dispensing or prescription records. Individual SCCS analyses were executed separately for each of the proposed study outcomes, including both safety events and negative control outcomes. Patients were followed for their entire observation time (e.g. from enrolment to disenrollment in each database), and incidence rates of each of the study outcomes calculated in periods of inferred persistent exposure to HCQ and non-exposure periods.

Participants

For the new user cohorts, participants included those with a history of RA (a condition occurrence or observation indicating RA any time before or on the same day as therapy initiation), aged 18 years or over at the index event, with at least 365 days of continuous observation time prior to index event. Inclusion and start of follow-up started at the time one of the drugs of interest (HCQ, SSZ, or addition of AZM or AMX amongst users of HCQ) was initiated after a diagnosis of RA. For the SCCS study, all prevalent users of HCQ were included, regardless of RA history or indication for HCQ therapy.

Participants were identified using pre-specified code lists reviewed by a core team of clinicians, epidemiologists, vocabulary experts, and health data scientists with extensive expertise in the use of the OMOP CDM and the OHDSI tools. The code lists in the OMOP CDM used to identify participants are listed in Supplementary Table 2.2.

Exposures, outcomes and confounders

The proposed code lists for the identification of the study population and for the study exposures were created by clinicians with experience in the management of RA using ATLAS, and reviewed by 4 clinicians and 1 epidemiologist (Supplementary Table 2.1).³⁹

A total of 16 severe adverse events (SAEs) were analysed. Hospital-based events, not available in primary care records (CPRD, IMRD and SIDIAP), included gastrointestinal bleeding, acute renal failure, acute pancreatitis, myocardial infarction, stroke, transient ischaemic attack, and cardiovascular events (composite). Additionally, angina/chest pain, heart failure, cardiac arrhythmia, bradycardia, venous thromboembolism, end stage renal disease, and hepatic failure were analysed from both primary and secondary care data. Mortality outcomes were obtained only from data sources with reliable information on death date (CPRD, IMRD, IPCI, Optum, SIDIAP, VA) and cardiovascular events preceding death records (CPRD, IMRD, Optum, VA), with the former contributing to informing all-cause mortality, and the latter also used to assess to cardiovascular death. All codes for the identification of the 16 proposed study outcomes were based on a previously published paper, and are detailed in Supplementary Table 2.2.⁴⁰ Face validity for each of the outcome cohorts was further reviewed by exploring age- and sex-specific incidence rates compared to previous clinical knowledge and/or existing literature.

Two active comparator analyses were conducted in the cohort studies: first, incident users of HCQ were compared to new users of SSZ; second, new use of AZM amongst prevalent users of HCQ was compared to incident use of AMX during ongoing HCQ use.

Exposure commenced on the first day of dispensing or prescription recorded with at least 365 days of prior observation period to increase confidence that the exposure was incident. Exposure interval gaps of ≤90 days (HCQ and SSZ) and of ≤30 days (AZM and AMX) between drug dispensing or prescription records were allowed and inferred as persistent exposure. Drug discontinuation was considered in the HCQ study if a patient switched from one study drug to another. Patients who switched from target exposure to comparator exposure, or vice versa, contributed follow-up time to the exposure cohort that they entered first, and were censored at the time of switching in the ‘on treatment’ analysis.

A list of negative control outcomes was also assessed for which there is no known causal relationship with any of the drugs of interest. These outcomes were identified using a semi-automatic process based on data extracted from literature, product labels, and spontaneous reports, and confirmed by manual review by 2 clinicians.⁴¹ A full list of codes used to identify negative control outcomes can be found in Supplementary Table 3, and details on covariate/confounder identification are provided in Supplementary Table 4.

Study size

This study was undertaken using routinely collected data and all patients meeting the eligibility criteria above during the study observation period were included. No a priori sample calculation was performed; instead, a minimum detectable rate ratio (MDRR) was estimated for each drug-outcome pair in each of the available databases. The MDRRs for each of the databases for each drug pair-outcome analysis, as well as sample size for each of the comparisons are reported in full in an interactive web app (https://data.ohdsi.org/Covid19EstimationHydroxychloroquine/. Only analyses with 0 counts in either treatment group were excluded based on power, with all others contributing to meta-analytic estimates where applicable.

Statistical methods

PS stratification was used as the analytical strategy to adjust for imbalance between exposure cohorts in a comparison, using a large-scale regularized logistic regression ³⁶ fitted with a LASSO penalty and with the optimal hyperparameter determined through 10-fold cross validation. Baseline patient characteristics were constructed for inclusion as potentially confounding covariates.⁴² From this large set of tens of thousands of covariates, key predictors of exposure classification were selected for the propensity score. The predictor variables included were based on all observed patient characteristics and covariates available at each data source, including conditions, procedures, visits, observations and measurements. All covariates that occur in fewer than 0.1% of patients within the target and comparator cohorts were excluded prior to propensity score model fitting for computational efficiency. Patients in the target and comparator cohorts were stratified into 5 propensity score quintiles.

Plotting the propensity score distribution and assessment of covariate balance expressed as the standardized difference of the mean was undertaken for every covariate before and after propensity score adjustment. A standardized difference > 0.1 indicated a non-negligible imbalance between exposure cohorts.⁴³ The target and comparator cohort were compared using a univariate Cox proportional hazards model conditioned on the propensity score strata with treatment allocation as the sole explanatory variable. Negative control outcomes analyses and empirical calibration were used to further minimise potential unresolved confounding with calibrated HRs (CalHRs) and 95% confidence intervals estimated.^44,45

For SCCS, safety of HCQ therapy was assessed separately as a secondary analysis, regardless of indication, comparing exposed and unexposed time periods within the same individuals. The method is self-controlled in that it makes within-person comparisons of event rates during periods of hypothesized increase risk with other periods of baseline risk, with eliminates all time-invariant confounding. Because we do not compare between persons, the SCCS is robust to between-person differences, even including unmeasured differences (like genetics). However, the method is vulnerable to time-varying confounders: the time of exposure may be incomparable to the time when not exposed. To adjust for this, we included many time-varying co-variates in the models, including age, season, and other drug exposures. The effects of age and season were assumed constant within each calendar month and were modelled using bicubic splines with 5 knots. A conditional Poisson regression was used to fit the outcome model using the Cyclops package, with a hyperparameter selected through 10-fold cross-validation.⁴⁶

Study diagnostics (power, propensity score distribution, covariate balance, empirical null distribution) were evaluated by clinicians and epidemiologists to determine which database-target-comparator-outcome-analysis variants could produce unbiased estimates. Database-target-comparator-analysis variants with zero event outcomes in the time-at-risk window or contained analyses with baseline covariate with standardized mean difference>0.1 after stratification were excluded from analysis. Study diagnostics for all database-target-comparator-outcome-analysis will be provided as part of study, regardless of which effect estimation results are unblinded. All the proposed analyses were conducted for each database separately, with estimates combined in fixed effects meta-analysis methods where I2 is <=40%. No meta-analysis was conducted where I2 for a given drug-outcome pair is >40%. Of note, when running analysis in a distributed network, it was not possible to link across datasets, and to know the extent of overlap between data.

All analytical code is available at https://github.com/ohdsi-studies/Covid19EstimationHydroxychloroquine, with study diagnostics considered prior to the unblinding of estimation results. All study diagnostics are available for exploration at https://data.ohdsi.org/Covid19EstimationHydroxychloroquine/. All statistical analyses were conducted using tools previously validated by the OHDSI community. For the cohort analysis, the CohortMethod package was used (https://ohdsi.github.io/CohortMethod/) using a large-scale propensity score (PS) constructed through the Cyclops package (https://ohdsi.github.io/Cyclops).⁴⁶ All SCCS were run using the freely available package (https://ohdsi.github.io/SelfControlledCaseSeries/).⁴⁷

Participants

A total of 956,374 HCQ and 310,350 SSZ users were identified, with 323,122 and 351,956 contributing to the analyses of combination therapy of HCQ with AZM compared to HCQ with AMX respectively. Participant counts in each data source are provided in Appendix S5.

Users of HCQ were more likely female (e.g. 82.0% vs 74.3% in CCAE) and less likely to have certain comorbidities like inflammatory bowel disease (e.g. prevalence of Crohn’s disease 0.6% vs 1.8% in CCAE) or psoriasis (e.g. 3.0% vs 8.9% in CCAE). All these differences were however minimised after propensity score stratification, with all reported analyses balanced on all identified confounders including socio-demographics, comorbidities and concomitant drug/s use. Similarly, users of combination HCQ+AZM differed from those of HCQ+AMX, with a prevalence of acute respiratory disease appearing higher amongst azithromycin users (62.5% vs 50.7% in CCAE). Again, propensity score methods resolved these differences, and comparison groups became balanced for all observed confounders after stratification. Detailed baseline characteristics for HCQ vs SSZ and for HCQ+AZM vs HCQ+AMX after propensity score stratification in CCAE are detailed in Table 1 for illustrative purposes, and similar tables with a more complete list of features for each included database and comparing before and after propensity score stratification are provided as Supplementary Tables 6.1.1 to 6.1.14 for HCQ vs SSZ, and Supplementary Tables 6.2.1 to 6.2.13 for HCQ+AZM vs HCQ+AMX.

Propensity score distribution plots showing overlap between groups and figures depicting all covariate balance and empirical null distribution plots based on negative controls can be found in Supplementary Tables 9.1 to 9.14 (Evidence evaluation diagnostics), and interactive versions of these are available at https://data.ohdsi.org/Covid19EstimationHydroxychloroquine/

Outcome Data

We report here (Table 2) on database-specific counts and rates of key outcomes (cardiovascular mortality, chest pain/angina and heart failure) observed in the proposed 30-day intention-to-treat analysis.

Database-specific counts, incidence rates (IR) of all study outcomes stratified by drug use are detailed in full in Supplementary Table S7. Least common outcomes included bradycardia (e.g. IR 0.92/1,000 person-years (py) amongst HCQ users in CCAE) and end-stage renal disease (e.g. IR <0.92/1,000 py amongst HCQ users in CCAE), whilst most common ones were chest pain/angina (e.g. IR 82.41/1,000 py amongst HCQ users in CCAE) and composite cardiovascular events (e.g. IR 17.96/1,000 py amongst HCQ users in CCAE). As expected, most IRs appeared higher in data sources which included older populations (e.g. IR of composite cardiovascular events in HCQ users in MDCR of 91.39/1,000 py). Mortality rates ranged from 4.81/1,000 person-years in HCQ users in Optum to 17.13/1,000 py amongst HCQ users in VA, with cardiovascular-specific mortality ranging from IR 3.43/1,000 py in HCQ users in VA to <4.25/1,000 person-years in SSZ users in the same data source.

Database and outcome-specific HRs (uncalibrated as well as calibrated) are reported in full in the form of forest plots (Supplementary Figure Sections 8.1 and 8.2). None of the SAEs appeared consistently increased with the short-term use of HCQ (vs SSZ) in the intention-to-treat analyses (Figure 1), with meta-analytic calibrated HRs (CalHRs and 95%CI) ranging from 0.67 (0.45-1.01) for hepatic failure to 1.35 (0.51-3.63) for cardiovascular mortality (Figure 2).

• Download figure
• Open in new tab

Figure 1.

Source-specific and meta-analytic cardiovascular risk estimates for hydroxychloroquine vs sulfasalazine and azithromycin vs amoxicillin new users during 30-day follow-up

• Download figure
• Open in new tab

Figure 2.

Meta-analytic risk estimates for hydroxychloroquine vs sulfasalazine and azithromycin vs amoxicillin new users during on-treatment during 30-day and on-treatment follow-up

Consistent findings were seen with the long-term (on treatment) use of HCQ vs SSZ (Figure 3), with the exception of cardiovascular mortality, which appeared inconsistent in the available databases, but overall increased in the HCQ group when meta-analysed: pooled CalHR 1.65 (1.12-2.44).

• Download figure
• Open in new tab

Figure 3.

Source-specific and meta-analytic cardiovascular risk estimates for hydroxychloroquine vs sulfasalazine and azithromycin vs amoxicillin new users during on-treatment follow-up

Similar results were obtained in SCCS analyses, which looked at the effect of HCQ use (on-vs off-treatment) on all outcomes except mortality regardless of indication, and therefore included non-RA patients (Tables S10.1 to 10.6 for database-specific results).

All the obtained database- and outcome-specific calHRs for the association between short-term (1 month) use HCQ+AZM vs HCQ+AMX are depicted in the form of Forest plots in Supplementary Figure Sections 8.1 and 8.2. Three SAEs appeared increased with the short-term (30-day fixed follow-up) use of HCQ+AZM: chest pain/angina (meta-analytic CalHR 1.15 (1.05-1.26), heart failure (meta-analytic CalHR 1.22 (1.02-1.45)), and cardiovascular mortality (meta-analytic CalHR 2.19 (1.22-3.94) (Figure 1).