Abstract
Background There is an urgent need for an efficacious and safe treatment for COVID19. Several trials testing a variety of therapeutics are on-going. Some in-vitro studies found the anti-malarial drug chloroquine (CQ), and its derivative, hydroxychloroquine (HCQ), are effective against COVID19. However, systematic reviews and meta-analyses of clinical trials in humans have produced conflicting findings on the efficacy and safety of these drugs. Guidelines vary considerably and are hotly debated at political and scientific levels. Therefore, it has become necessary to provide a summary of the effectiveness and safety of these drugs in treating COVID19 infection, using an overview of the existing systematic reviews and meta-analyses.
Objective To synthesize the findings presented in systematic reviews and meta-analyses as well as to update the evidence using a meta-analysis in evaluating the efficacy and safety of CQ and HCQ with or without Azithromycin for the treatment of COVID19 infection.
Methods The design of this meta-review followed the Preferred Reporting Items for Overviews of Systematic Reviews including harms checklist (PRIO-harms). A comprehensive search included several electronic databases in identifying all systematic reviews and meta-analyses as well as experimental studies which investigated the efficacy and safety of CQ, HCQ with or without antibiotics as COVID19 treatment. Manual searches of the reference list of all included studies and a citation search of the top 20 papers supplemented the search. Findings from the systematic reviews and meta-analyses were reported using a structured summary including tables and forest plots. The updated meta-analysis of experimental studies was carried out using the distributional-assumption-free quality effects model. Risk of bias was assessed using the Assessing the Methodological Quality of Systematic Reviews (AMSTAR) tool for reviews and the MethodologicAl STandard for Epidemiological Research (MASTER) scale for the experimental studies. The main outcomes for both the meta-review and the updated meta-analysis were; mortality, transfer to the intensive care unit (ICU), intubation or the need for mechanical ventilation, worsening of illness, viral clearance and the occurrence of adverse events.
Results A total of 13 reviews with 40 primary studies comprising 113,000 participants were included. Most of the primary studies were observational (n=27) and the rest were experimental studies. Two meta-analyses reported a high risk of mortality with similar ORs of 2.5 for HCQ with Azithromycin. However, four other meta-analyses reported contradictory results with two reporting a high risk of mortality (OR ∼ 2.2 to 3.0) and the other two reporting no significant association between HCQ with mortality. Most reviews reported that HCQ with or without Azithromycin had no significant effect on virological cure, disease exacerbation or the risk of transfer to the ICU, need for intubation or mechanical ventilation. After exclusion of studies that did not meet the eligibility criteria, the updated meta-analysis contained eight experimental studies (7 RCTs and 1 quasi-experimental trial), with a total of 5279 participants of whom 1856 were on either CQ/HCQ or combined with Azithromycin. CQ/HCQ with or without Azithromycin was significantly associated with a higher risk of adverse events (RR 5.7, 95%CI 2.4-13.7, I2 =55%, n = 5 studies). HCQ was not effective in reducing mortality (RR 1.0, 95%CI 1.0-1.2, I2 =0%, n=6 studies), transfer to the ICU, intubation or need for mechanical ventilation (RR 1.1, 95%CI 0.9-1.4, I2 =0%, n=3 studies) virological cure (RR 1.0, 95%CI 0.9-1.2, I2 =55%, n=5 studies) nor disease exacerbation (RR 1.2, 95%CI 0.3-5.0, I2 =29%, n=3 studies).
Conclusion There is conclusive evidence that CQ and HCQ, with or without Azithromycin are not effective in treating COVID-19 or its exacerbation.
Registration PROSPERO: CRD42020191353
Introduction
The severe acute respiratory syndrome coronavirus 2 (SARS-CoV2) is currently known to be among one of the most contagious viruses in the history of pathogens (1). This novel human coronavirus emerged in Wuhan in December 2019 and has since spread from person to person in an efficient and sustained way to cause a global pandemic, with the disease named Coronavirus Disease-19 (COVID19) (1). To date, 16.9 million people have been infected with COVID19 and more than 660,000 have died as of 29th of July 2020 (2). Although the disease is largely controlled in China, where it originated, and Italy, which became the epicentre of the disease in Europe, subsequently, the cases and deaths continue to increase globally, with the highest increases in the USA, Brazil and India. Concerns for world economies has forced the relaxation of lockdowns and restrictive measures even in some of the worst affected countries (3-5). However, the fatalities from COVID19 are expected to rise further as lockdowns are eased and people begin to interact more (5, 6). This continued increase in deaths from COVID19 infection has warranted the need to investigate effective vaccines and therapies.
Of several therapeutic drugs that have been suggested, chloroquine (CQ), an anti-malarial drug in the class of 4-aminoquinolones with anti-inflammatory, antiviral and anti-thrombolytic properties, and its derivative, hydroxychloroquine (HCQ) have been re-purposed and are being used widely for the treatment of COVID19 (7, 8). CQ and HCQ are antimalarials as well as used as disease-modifying antirheumatic drugs (DMARDs) (9). There are several mechanisms which have been suggested for the expected effect of the two drugs against viruses in general and SARS-COV-2 in particular. One of these mechanisms is based on inhibiting the ability of the virus to enter the cell. SARS-COV-2 harbours a spike (S) protein, which is considered a crucial element in the virus replication cycle as it binds to angiotensin-converting enzyme 2 (ACE2) expressed in the lungs of the host cell receptor (10, 11). The antiviral effect of both CQ and HCQ is thought to be through their ability to interfere with the glycosylation of ACE2 and thus prevent the proper binding of the S protein. (9). Moreover, virus entry occurs through receptor-mediated endocytosis, which needs an acidic PH to complete the fusion and deliver the viral genome into the cell. Both CQ and HCQ are weak bases and thought to inhibit this process that the SARS COV-2 virus needs for replication (12). Apart from their direct antiviral properties, CQ and HCQ also seem to have immune-modulatory effects which help to reduce over-activation of the immune system from COVID19 (9). HCQ has been used more frequently than CQ in treating COVID19 as it has been reported to be more potent in vitro (13), can be used in higher doses for a longer time with a lower risk of adverse events compared to CQ and is more widely available (7, 14). Further, both drugs are relatively cheap, and their use in the treatment of COVID19 is based on promising results from in vitro studies (13, 15) and some observational studies (16, 17). Repurposing the drugs for use in treating COVID19 has been easy as the drugs are already in use for the treatment of malaria, are cheap and have been thought to have a relatively safe profile (7, 18). Due to the above reasons and the unprecedented situation and the urgency to curb the COVID19 pandemic, CQ and HCQ have been approved, on a fast track basis, by the United States Food and Drug Administration and other regulatory bodies, despite the lack of good quality evidence of their efficacy and safety (19, 20). Consequently, more than 80 trials of the two drugs are either ongoing or completed and it is necessary to evaluate the evidence so clinicians and regulatory bodies can make informed decisions on the use of these drugs for the treatment of COVID19 infection.
Initial reports suggested that HCQ was effective in the treatment of COVID19 associated pneumonia (21) and that HCQ in combination with the second-generation macrolide antibiotic, Azithromycin, resulted in a lower fatality (22), and 93% viral clearance of COVID19, by day 8 (17). However, subsequent findings from both observational studies (23, 24) and clinical trials have been contradictory (25-27). Further, although the two drugs are relatively safe in the treatment of malaria, concerns have been raised about the risk of adverse events associated with their use when treating COVID19 (28). The RECOVERY trial (29), one of the largest trials to-date investigating the optimal treatment for COVID19, issued a statement that they found no differences in mortality between participants on HCQ and those on usual care, and subsequently stopped the HCQ arm of the trial. In a similar move, the World Health Organization stopped the HCQ arm of the SOLIDARITY trial (30), citing the lack of efficacy from their interim analysis of the U.K. RECOVERY trial, the French DisCoVeRy trial (31) and an unnamed Cochrane review (32). Before the stopping of the RECOVERY and SOLIDARITY trials, the largest observational study to date, a multinational cohort study (33) of 96,000 participants, had previously reported a 6 to 8 fold increase in the mortality associated with CQ or HCQ, with or without macrolides. However, the study was subsequently retracted due to concerns over the veracity of the data and its analysis (34) leaving some degree of uncertainty in its wake.
Several systematic reviews and meta-analyses investigating the efficacy and the safety of CQ and HCQ have been published or are in the process of being published (8, 13, 16, 28, 35-54). Similar to the primary studies, systematic reviews and meta-analyses have presented contradictory findings. A key issue that has made it difficult to have conclusive results on the efficacy of CQ and HCQ with or without the macrolide antibiotics is that clinical trials were few, small and poorly designed. Most of the existing systematic reviews have therefore carried out meta-analyses combining data from observational and experimental studies without accounting for the quality of included studies. Several clinical trials, with acceptable quality, are now available. Therefore, it has become necessary to synthesize all available evidence to provide the best evidence-based assessment on the efficacy and safety of both CQ and HCQ, with or without macrolide antibiotics, in the treatment of COVID19 infection. In this respect, we conducted this umbrella review with two broad aims; (1) to assess the efficacy and safety of each of HCQ and CQ, with or without Azithromycin in the treatment of COVID19 by assessing the evidence from existing systematic reviews and meta-analyses, and (2) to carry out an updated meta-analysis of the existing experimental studies to assess the efficacy of these drugs.
Methods
Study Design
This study protocol is registered on PROSPERO (CRD42020191353). This study had two components; an overview of all existing systematic reviews and meta-analyses and an updated meta-analysis of all eligible experimental studies that investigated the efficacy of either CQ or HCQ with or without macrolide antibiotics in the treatment of COVID19 infection. The design of this overview followed the Preferred reporting items for overviews of systematic reviews including harms checklist guidelines (PRIO-harms) (55). The updated meta-analysis was carried out according to the Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) (56).
Search strategy
We conducted electronic searches for all experimental studies, systematic reviews and meta-analyses on the efficacy and safety of CQ and HCQ for the treatment of COVID19 up to the 3rd of June 2020. We searched for studies in the following electronic databases; Cochrane Database of Systematic Reviews (CDSR) (The Cochrane Library), Cochrane Central Register of Controlled Trials (CENTRAL), the China Academic Journals Full Text Database, EMBASE and MEDLINE through PubMed, the Database of Abstracts of Reviews of Effectiveness (DARE), Scopus, Web of Science, CINAHL, and the databases of preprints (medRXIV and bioRxiv). We hand-searched the databases of preprints (medRXIV and bioRxiv) and the websites for the World Health Organization Solidarity Trial and the U.K. Recovery Trial. We manually searched all references of all included studies and carried out a citation search of all included systematic reviews. The top 20 similar articles citation search of all included studies was performed on Pubmed to retrieve studies that might have been missed in the original electronic search.
We used the following search terms for chloroquine and hydroxychloroquine; “chloroquine” OR “hydroxychloroquine” OR “CQ” OR “HCQ”. We used the following search terms for COVID19 infection; “COVID19” OR “Coronavirus” OR “novel coronavirus” OR “SARS-CoV-2” OR “COVID” OR “COVID-19”. The following search terms were used for the study design; “clinical study”, “clinical trial”, “trial”, “RCT”, “controlled trial”, “randomized controlled trial”, “meta-analysis”, “rapid review”, “review”, “systematic reviews”. All searches had no limitations on language or location but were limited to studies published during the year 2020. Keywords and Medical Subject Headings (Mesh) were used for searches for COVID19, SARS-CoV2, chloroquine and hydroxychloroquine. The full electronic search strategy is given in Appendix 1 (Supplementary Table S1).
Eligibility
Systematic reviews and/or meta-analyses were included if they compared the efficacy and/or safety of HCQ or CQ. Such reviews should either have a minimum of two clinical studies comparing HCQ or CQ with or without Azithromycin to any other standard treatments, including placebo. Participants in the included index primary studies should have had confirmed COVID19, regardless of age or the severity of illness. Reviews were excluded if they were literature reviews, did not include at least two eligible primary studies and if their main scope was on prophylaxis. The updated meta-analysis only included experimental studies which investigated the efficacy of either CQ, HCQ or either with Azithromycin added in the treatment of COVID19. Studies were participants were diagnosed using symptoms were excluded as symptoms have been shown to have poor diagnostic accuracy (57). Observational studies and any studies without control groups were excluded. Studies on animals and in-vitro studies were also excluded. In the case that studies were duplicates, either the study with the most data was used and the other excluded or both studies were combined. Studies were included if the intervention included either CQ or HCQ alone or with Azithromycin in any dose combinations and any length of administration. Due to the heterogeneous nature of COVID19 treatments in different countries and different disease severity categories, any studies that had control groups that did not include either CQ or HCQ were acceptable for inclusion in this overview and meta-analysis. Studies were also included irrespective of either the severity of disease of included participants or the setting (i.e. either hospital-based or community or both).
Data extraction
Search results were uploaded on to the Rayyan platform (58) where 2 authors blindly screened titles and abstracts. Conflicts were discussed and resolved by consensus between authors. The full text of potentially relevant articles were screened against eligibility criteria for the final inclusion.
For each study, two authors independently extracted data and assessed quality. The data extracted from the reviews included; type of review (systematic review or meta-analysis), date of publication or submission to the preprint servers, countries with data included in the review, scope of the review, the number and type of index studies included, tools used to assess the risk of bias, risk of bias summary, main comparisons, total participants, mean age, outcomes measured and pooled measures of effect, statistics for heterogeneity, the review conclusion and limitations. Additional data included the number of and citations of all included primary studies in the review. The primary studies were further categorised as either experimental or observational, first based on their classification in the parent review and later after an independent assessment by two authors. All experimental studies identified from the assessment were considered for inclusion in the updated meta-analysis.
Data extracted from experimental studies included; study design (RCT or quasi-RCT), intervention (whether CQ or HCQ with or without Azithromycin), dosage, route of administration, control treatment description, any co-interventions, setting (community setting or hospital), the country where study carried out, proportions with severe sickness, proportions with comorbidities, mean age, gender distribution, and total with each out outcome in the intervention and control groups.
Outcomes
Due to the heterogeneous nature of the outcomes assessed in different systematic reviews, meta-analyses and primary studies, we grouped the outcomes into four main groups. The primary outcome was mortality assessed as all-cause mortality. Secondary outcomes were disease exacerbation, virological cure, adverse events and included a composite outcome that combined the events of any transfer to the intensive care unit (ICU), any need for intubation or any need for mechanical ventilation. Disease worsening was defined as any form of symptom worsening such as the need for oxygen, dyspnoea, hospitalization in the case of community-based studies. The virological cure was defined as a negative PCR any time after commencement of treatment. Safety was assessed as the occurrence of the known adverse events of CQ or HCQ or Azithromycin. These adverse events included gastric side effects such as diarrhoea and vomiting, ventricular tachycardia possibly as a result of QTc prolongation and headache, blurred vision and rash.
Assessment of study quality
Two authors independently assessed the quality of each included review using the Assessing the Methodological Quality of Systematic Reviews (AMSTAR) tool (59). Each review had a maximum score of 11 if the methodological quality is good and zero if the methodological quality is poor. Any disagreements were resolved by discussion between the authors.
For the experimental studies, the MethodologicAl STandard for Epidemiological Research (MASTER) scale (60) was used to assess the methodological quality of studies across safeguards listed within 7 standards. The 7 standards assessed were; equal recruitment, equal retention, equal ascertainment, equal implementation, equal prognosis, sufficient analysis and temporal precedence (60). A kappa interrater agreement was calculated for each study and the quality safeguard counts averaged if the Kappa was at least 0.70. Where the Kappa was below 0.7, the authors resolved disagreements by discussion or by referring to a third assessor if they did not resolve. The quality counts were used to rank studies for inclusion in a bias adjusted meta-analysis.
Synthesis of findings
The characteristics of the included reviews and experimental studies are summarised in Table 1 and Table 2 respectively. The quality assessment of the included reviews and experimental studies are shown in Supplementary Tables 2 and 3, respectively. A map showing the distribution of all the index studies included in the included reviews was created using Tableau software (61).
Synthesis of findings from different reviews was done using a combination of a structured summary of findings from the reviews and presentation in forest plots (62). A table with the findings of each review for each outcome was presented. For outcomes were several meta-analyses were available, forest plots were used to show the magnitude of the effect, the 95% confidence intervals (95%CI) and the number of included studies. Where there were no available meta-analyses, findings from systematic reviews were compared narratively. The overall score from the AMSTAR quality assessment and the I2 heterogeneity score were incorporated in the interpretation of the findings of each review. The forest plots were created in R statistical software (63).
For the updated meta-analysis, the quality effects model (64) was employed to pool the estimates (relative risks), as it is more robust and performs better than the random-effects model (65). In some studies where outcome assessed cases were zero, a post hoc continuity correction adding (0.5) to all cells was employed for valid estimation of the relative risk and its variance. Forest plots were used to depict the results of the pooled analysis. The quality effects method uses a relative quality rank from each of the studies to modify the study’s variance weight, thereby incorporating the quality of the study quantitatively into the results. The results of random effects analyses were reported in the supplementary material for comparison only if there was heterogeneity as without this it defaults to a fixed effect model similar to the quality effects model. The STATA software program (66) was utilized for meta-analysis. Forest plots were used to present the pooled risk ratios and their confidence intervals (CIs). The Cochran Q test p-value was used to test for and the I2 statistic to quantify heterogeneity (67). The I2 statistic measures consistency and is an indication of the variability in the estimates of the effects that are caused by heterogeneity instead of sampling error, and ranges from a 0% (no heterogeneity) to 100% (high heterogeneity). I2 values above 50% indicate substantial heterogeneity and above 75% high heterogeneity. Doi plots and LFK index were used for assessing publication bias as they are more reliable (68) than funnel plots and in this case funnel plots could not be used as they are not recommended when there are less than 10 studies in a meta-analysis (69).
Ethics
Ethics approval was not required as the study used published data.
Results
Search results and characteristics of included reviews
The search identified 158 reviews and 111 were excluded by screening the title and abstract (Fig 1). Most reviews were excluded because they were either literature reviews or review which did not include completed clinical studies (Fig 1 and Supplementary Table S2). The remaining 47 reviews were screened for eligibility through reading the full text and 24 were provisionally included. One additional review was identified through manual searching of references and the citations search. The final included reviews were 13, of which 5 (38.5%) were systematic reviews and meta-analyses and the remainder systematic reviews only (Table 1).
All the included reviews were conducted between 13 April and the 1st of June 2020. At the time that this meta-review was carried out, 7 of the included reviews were published (16, 28, 36, 37, 40, 46, 50) and the remaining 6 were preprints (35, 43, 47, 51-53). All the included reviews assessed the efficacy and the safety of the CQ and HCQ in the treatment of COVID-19, except Jankelson et al. (28) which only assessed adverse events associated with CQ and HCQ.
Most of the included reviews had above 7 safeguards and only 2 reviews (28, 53) had 6 or lower safeguard counts on the AMSTAR scale (Supplementary Fig. 1 & Supplementary Table S3). The same 2 reviews did not report that they assessed for risk of bias in their included index studies. One (53) of the two reviews which did not report assessing the risk of bias in included studies was not published at the time of this umbrella review. The reported quality of included index studies in the reviews was generally poor, with 7 reviews (16, 35-37, 40, 43, 47, 51) reporting a low count of safeguards in general. The most common issues were lack of safeguards against selection bias, lack of randomization, lack of allocation concealment bias, absence of blinding and performance and reporting biases. Most of the included reviews had the limitations of a small number of included index studies, lack of proper control groups or improper randomization among the included primary studies. An additional limitation of the reviews was the mixing of results from observational and experimental studies in the meta-analyses of efficacy.
Characteristics of primary studies included in reviews and the updated meta-analysis
The total number of index studies in the included reviews, after removing duplicate studies was 40. Most of the included primary studies were conducted in the USA (14 studies, 108 011 participants), followed by China (9 studies, 1975 participants) and France (9 studies, 1637 participants) (Fig 2). The total number of participants in all the reviews after removing duplicate studies was 113 786. All the included reviews included at least 3 experimental studies and the number of included observational studies ranged from zero in 3 reviews (36, 46, 53) to 20 in one review (40).
The observational index studies were excluded from the updated meta-analysis (Supplementary Table S4). After screening the 11 studies described as experimental studies in the included reviews, only 5 experimental studies (26, 27, 70-72) satisfied the criteria for inclusion in the updated meta-analysis (Fig. 1). The reasons for exclusion of the experimental studies were as follows; duplicate study (n=1) (73) which used data from Gautret et al. (27), one study had no full text available (74), one study compared high dose chloroquine to low dose chloroquine (75), and one study included participants who did not have COVID19 confirmed by PCR (76). Another experimental study was excluded because it was a letter to the editor describing 15 trials with a total of 100 patients but with no clear details about the trials, the participants or any outcomes (21). Finally one study (77) was described by the authors as quasi experimental as two institutions treated patients confirmed as positive with HCQ but study subjects in both institutions received HCQ. One institution received HCQ shortly after admission, and in the other institution did not receive HCQ until days later when PCR results came through so there was no real control arm and was not published with a submission version of the pdf circulated on the internet. Five trials were identified from the additional search, with three included (29, 78, 79) and two excluded (80, 81) as they included participants who were diagnosed using symptoms rather than PCR (Fig. 1).
The eight experimental studies included in the updated meta-analysis (7 RCTs (26, 29, 70-72, 78, 79) and 1 quasi-experimental study (27)), had a total of 5279 participants and 1856 on either CQ/HCQ or HCQ+ Azithromycin. Most of the studies (n=4) were from China (26, 72, 82, 83) and the remaining 4 studies from France (n =1) (27), the U.K. (29), Spain (79) and Taiwan (78). Only one study used CQ (70) as the experimental drug while the remaining seven used HCQ. All seven (26, 29, 70-72, 78, 79) included RCTs had safeguard counts (quality assessment) of at least 29 out of 36. The quasi-experimental study (27) had the lowest safeguard counts of 16. All the included studies, except Chen Z et al. (26), either did not blind or did not report sufficient information about blinding of either participants and research performers. Other notable deficiencies included; insufficient reporting of randomisation procedure and insufficient statistical analyses (Supplementary Fig.1B & Supplementary Table S3).
All-cause mortality
Findings from included reviews
A total of 4 meta-analyses, all with AMSTAR quality safeguard counts of at least 10, reported pooled effect sizes for mortality (16, 35, 47, 50) (Fig. 3). For HCQ alone, two meta-analyses; Yang et al. (52) (677 participants, 5 studies) and Singh et al. (50) (2042 participants, 10 studies) found a significantly higher risk of mortality with HCQ with odds ratios of 3.0 and 2.2, respectively, with no heterogeneity (I2 = 0% for both). The other 2 reviews, Shamshirian et al. (47) (103 486 participants, 21 studies) and Charko et al. (35) (4306 participants, 11 studies) found slight but non-significant increases in the risk of mortality with HCQ use, with significant heterogeneity in both meta-analyses. Only 2 reviews (47, 52) reported pooled analyses on HCQ+ Azithromycin and both found a significantly higher risk of mortality associated with the use of HCQ+AZI, with similar odds ratios of 2.5, with little heterogeneity in one and more heterogeneity in the other (Fig. 3).
Of the reviews without meta-analysis, 4 concluded that there was evidence for higher risk of mortality in the HCQ group (40, 43, 51, 52) and one review (53) reported evidence of lower mortality in the HCQ group. Two reviews reported evidence of higher mortality in the chloroquine group (40, 46) (Supplementary Table S6).
Updated meta-analysis of experimental studies
Six experimental studies, five RCTs (29, 71, 72, 78, 79) and one quasi-experimental studies (27), assessed mortality, with a total of 5195 participants, of which 1815 were in the intervention group. There was no significant difference in risk of mortality between participants who received HCQ with or without Azithromycin and those on standard care (RR 1.07, 95%CI 0.97 – 1.19; Fig 4), which was consistent across studies (I2 = 0%). The Doi plot (Supplementary Fig. 3A) showed minor asymmetry. As Horby et al. had the most weight in the meta-analysis, we carried out a sensitivity analysis without this study and the effect of HCQ with or without Axithromycin on all-cause mortality remained non-significant (RR 1.02, 95%CI 0.19 -5.64) (Supplementary Fig 2A).
Transfer to the ICU, intubation and mechanical ventilation
Findings from included reviews
A total of 7 reviews investigated the risk of transfer to ICU, need for intubation and mechanical ventilation. Only one meta-analysis (47) of 2 studies with a total of 308 participants was done and found a non-significant 2-fold increase risk of intubation in individuals on HCQ (OR 2.11, 95%CI 0.31-14.03), with significantly high heterogeneity. The remaining 6 reviews (37, 40, 43, 50, 51, 53) which did a narrative synthesis did not find a difference in the risk of transfer to the ICU, intubation or need for mechanical ventilation (Supplementary Table S6).
Updated meta-analysis of experimental studies
Two RCTs (29, 79), and the quasi experimental study (27), reported data on this outcome with a total of 4982 participants, of whom 1704 were in the intervention group. There was no significant difference in risk of ICU transfer, need for mechanical ventilation or intubation in participants who received HCQ with or without Azithromycin, compared to those on standard care (RR 1.10, 95%CI 0.89 – 1.37; Fig. 5A), consistent across the all the studies (I2 = 0%). The Doi plot showed major asymmetry which was not resolved after several sensitivity analyses (Supplementary Fig. 3B).
Virological cure
Findings from included reviews
Twelve reviews (16, 35, 37, 40, 43, 46, 47, 50-53) assessed the outcome of virological cure with 5 of the reviews being meta-analyses. All the 5 meta-analyses (16, 35, 47, 50) found no significant differences between either HCQ alone or HCQ+ Azithromycin and control, in virological cure (Fig. 6).
However, the remaining seven reviews (36, 37, 40, 43, 46, 51, 53) which narratively synthesized this outcome concluded that either HCQ alone or HCQ+ Azithromycin were effective to some extent in the cure of the virus (Supplementary Table S6).
Updated meta-analysis of experimental studies
Five experimental studies, 4 RCTs (70-72, 78) and the single quasi-experimental study (27), assessed virological cure, with a total of 277 participants, of which 147 were in the intervention group. There was no significant difference in viral cure between participants who received HCQ with or without Azithromycin and those on standard care (RR 1.04, 95%CI 0.88 – 1.21; Fig 5), with some heterogeneity (I2 =51%). There was consistency across studies after removal of the quasi-experimental study, Gautret et al., although this did not alter the results of the pooled analysis (RR 1.02 95%CI 0.91-1.14, I2 =0%) (Supplementary Fig. 4). The Doi plot showed major asymmetry which was not resolved after several sensitivity analyses (Supplementary Fig. 3C).
Disease exacerbation
Findings from reviews
Four meta-analyses (16, 35, 47, 52), all with AMSTAR score above 9 found no significant effect of either HCQ or HCQ + Azithromycin on disease exacerbation (Fig 8). All the reviews without meta-analysis on this outcome concluded that either HCQ or HCQ+ Azithromycin reduced the severity of illness (Supplementary Table S6). Three reviews (50, 51, 53) concluded that pneumonia was improved in the HCQ / HCQ+ Azithromycin arm. Singh et al. (50) and Hernandez et al. (40) concluded that both fever and coughing were improved in the HCQ/ HCQ+ Azithromycin group.
Updated meta-analysis of experimental studies
Three RCTs (26, 72, 79) assessed disease exacerbation, with a total of 385 participants, of which 182 were in the intervention group. There was no significant difference in the risk of disease exacerbation between participants who received HCQ and those on standard care (RR 1.18, 95%CI 0.28 – 4.96; Fig. 5C), with low heterogeneity across studies (I2 =28.6%). The Doi plot showed major asymmetry which was not resolved after several sensitivity analyses (Supplementary Fig. 3D).
Adverse events
Findings from reviews
A total of ten reviews investigated the risk of adverse events between HCQ or HCQ+ Azithromycin groups and control. Two of these carried out meta-analyses and found pooled Odds Ratios of 3.9 (95%CI 1.8-8.1, n= 4 studies, 304 participants, I2 = 7.7%) (47) and 4.1 (95%CI 1.4-11.9, n= 4 studies, 1714 participants, I2 = 81%) (35). All the remaining 8 reviews (16, 28, 36, 37, 43, 51, 53) except one (40), found an increased risk of adverse events in the HCQ/ HCQ+ Azithromycin group. The most commonly reported adverse events were QTc interval prolongation,(26, 70-72, 78, 79) diarrhea, arrythmia and first-degree AV block (Supplementary Table S6).
Updated meta-analysis of experimental studies
Five RCTs (26, 70-72, 79) assessed adverse events, with a total of 557 participants, of which 262 were in the intervention group. The most commonly reported adverse events due to HCQ and CQ were gastrointestinal (nausea, vomiting, diarrhea and abdominal pain), reported in five trials (70-72, 78, 79), headache (26, 78, 79) and itchiness and rash (26, 70). Serious adverse events were very rare and reported in only three participants in three (29, 71, 72) of the included trials. These included a single case of torsades de pointes reported in the RECOVERY trial (29), two cases, one with blurred vision and the other with thirst, reported by Tang et al. (71) and a case described as “severe” but with no clear details, Chen J et al. (72).The pooled risk ratio showed a significantly higher risk of adverse events in participants who received HCQ compared to those on standard care (RR 5.71, 95%CI 2.38 – 13.73; Fig. 5D), with substantial and unexplained heterogeneity across studies (I2 =55.0%). The use of random effects model did not result in change in either the effect size or the significance (RR 4.43, 95%CI 2.03 – 9.67, I2 = 55%) (Supplementary Fig. 2B). The Doi plot showed major asymmetry which was not resolved after several sensitivity analyses (Supplementary Fig. 3E)..
Conclusions from reviews
Nine of the reviews (35-37, 40, 46, 47, 50, 51, 53) all concluded that there was insufficient evidence to support the use of either CQ or HCQ with or without Azithromycin in the treatment of people with COVID19 disease. Two reviews concluded that there was some benefit in using HCQ or HCQ+ Azithromycin. Yang et al. concluded that, although they were associated with higher mortality, HCQ with or without Azithromycin were beneficial, based on their effect on COVID19 viral clearance. Sarma et al. concluded that HCQ was beneficial based on its efficacy in reducing radiological progression and that the drug was safe. The remaining 2 reviews concluded that either CQ or HCQ were unsafe based on higher mortality (43) and a higher risk of adverse events (28, 43).
Discussion
In this meta-review, we summarized the findings of seven systematic reviews and five meta-analyses and carried out an updated meta-analysis to investigate the efficacy and safety of CQ and HCQ, with or without a second generation macrolide antibiotic (Azithromycin) in individuals infected with COVID-19, limiting our analysis to eight experimental studies which met a strict inclusion criteria. Findings on the effect of CQ and HCQ on mortality were conflicting; two studies suggesting a 2 to 3-fold increase in mortality associated with HCQ, while other two reviews found no significant association with mortality. Only two meta-analyses were carried out for HCQ combined with Azithromycin and both reported a 2.5-fold increase in risk of mortality. The updated meta analyses carried out for this review showed no significant difference in all-cause mortality among those taking either HCQ alone or in combination with Azithromycin compared to standard care.
This meta-review also found the risk of transfer to ICU is doubled with either HCQ or HCQ and Azithromycin and two meta-analyses conducted on adverse events showed a 4-fold increase in the risk associated with HCQ or HCQ and Azithromycin. The reviews also indicated there were no significant effect of HCQ or HCQ and Azithromycin on both disease exacerbation and virological cure. The updated meta-analysis of eight experimental studies, however, showed that individuals on HCQ or HCQ and Azithromycin had a 6-fold increase in risk of any adverse events. The updated meta-analysis also showed that there was no significant effect of CQ/HCQ with or without Azithromycin on the risk of transfer to the ICU, intubation or need for mechanical ventilation, virological cure and disease exacerbation.
An important consideration in this meta-review is the impact of methodological limitations on the results of both the primary studies and the systematic reviews and meta-analyses included in this umbrella review. These limitations could primarily have resulted from the urgency of the need to find a cure, at short notice, for a pandemic that seems to be worsening in many countries to date. The limitations include, but are not limited to, small study sample sizes, the scarcity of randomized controlled trials, and the lack of methodological rigueur in the primary studies. All the reviews, except one (36), included observational studies, which tend to have confounding and may lead to biased estimates of effects. An additional weakness of these observational studies is that patients and clinicians will most likely choose an experimental drug, compared to standard of care which comprises of symptom management during a pandemic with a perceived high risk of death and no cure. Faced with a life-threatening illness, patients with severe illness will likely choose the experimental drug in the absence of proven alternatives, while those with mild to moderate disease may not want the experimental drug. The inclusion of observational studies in these meta-analyses seems to have been driven by the lack of good quality experimental studies and the need to use as much of the available information as possible. The observational studies were bigger than the RCTs and therefore tended to influence the pooled estimates. The effect of inclusion of the observational studies could have been reduced using quality-adjusted meta-analysis synthesis, which decreases the weight of the observational studies in the overall estimate. However, none of the existing meta-analyses adjusted for the quality of the included studies in their syntheses. An exception is Shamshirian et al. (47), who carried out a sensitivity analysis with experimental studies only. Further, the poor quality of studies included in the reviews is one of the limitations frequently cited by the review authors. Some major limitations in the experimental studies include; participants not having confirmed COVID19 disease in one study (76), controlled clinical trials which had a high risk of selection bias (27, 77), and studies where the veracity of data presented could not be verified (33, 84). An example of the later is a letter about 15 clinical trials in China claiming that CQ was effective (84) but without any data presented. This letter was included in two reviews (35, 53) and contributed to the perceived efficacy of CQ and HCQ, which resulted in regulatory approvals in many countries. The risk of bias associated with a lack of either proper randomization or protection of the allocation sequence in the controlled trials is particularly serious in the case of COVID19. This is because without an effective cure, and based on the hype about CQ/HCQ, severely ill patients were more likely to be given the experimental treatment, and consequently have worse outcomes if the treatment was not efficacious. Finally, it would be remiss to not mention that the biggest observational study (33) included in two of the meta-analyses (43, 47) was subsequently retracted (34). This impacts the findings of these reviews although one of the reviews carried out a sensitivity analysis without this study. The collective impact of these limitations is that even though there are multiple reviews published, there is still uncertainty about the efficacy and safety of CQ or HCQ with or without second-generation macrolide antibiotics such as Azithromycin.
The findings of our updated meta-analysis of experimental studies showed no benefit of HCQ, with or without Azithromycin in reducing the risk of all-cause mortality. Our findings are in agreement with findings of two of the included meta-analyses (35, 47) but contrast with the two others (50, 52). The two meta-analyses which found a higher risk of mortality had fewer studies and participants than the latter two reviews. Singh et al (50) included three studies with 474 participants from two observational studies (85, 86) and the one quasi-experimental study (77) had a weight of 4.6%, in their meta-analysis. Yang et al. (52) included data from two small trials (27, 72) and one observational study, the US veterans study (86) which had a weight of 95% in the meta-analysis, and is misclassified as an RCT in the meta-analysis. Further, all the deaths, except one, occurred in the U.S. veterans’ observational study, and therefore the meta-analysis effectively drew its conclusions from the one big observational study where clinical severity may have influenced allocation of treatment. Similar results were also observed by Yang et al. who also included the same three studies in their meta-analysis for HCQ+ Azithromycin. Although Shamshirian et al. (47) found no significant difference in mortality when HCQ was used alone, they found a 2.5 fold increase in risk of mortality when the drug was combined with Azithromycin, a result similar to that of Yang et al. Shamshirian et al.’s meta-analyses included several observational studies and data from the retracted study by Mehra et al. (33), although sensitivity analyses without this study did not alter their conclusions. Charko et al. (35) included data from five observational studies and only one quasi experimental study which had a weight of 5.8% in the meta-analysis. It is apparent that most of the conclusions from the reviews were driven by data from observational studies. Despite the aforesaid, the evidence from our updated meta-analysis and these existing reviews suggests that, at the very least, CQ, HCQ or CQ/HCQ+ Azithromycin do not have a protective effect against mortality in individuals with COVID19 disease and may be harmful, in the worst-case scenario.
From a meta-analysis of one large RCT (29), one small RCT (79) and one quasi-experimental studies (27), we found no difference in risk of transfer to the ICU, intubation or the need for mechanical ventilation. This finding was in agreement with seven of the included reviews (37, 40, 43, 50, 51, 53) (47). In agreement with the included reviews, we did not find any significant effect of HCQ, with or without Azithromycin, on either virological cure or disease exacerbation but found a 6-fold increase in risk of adverse events which was expected. It should be noted however, that the occurrence of serious adverse events in the included experimental studies was rare, in agreement with the known safety profile of both CQ and HCQ.
Our updated meta-analysis had limitations which include small sample sizes in seven of the eight included trials and high risk of selection bias in the included quasi-experimental study. Some of the strengths of this updated meta-analysis include the inclusion of data from experimental studies only and the use of quality effects models to adjust for the weight of the studies in the meta-analysis.
Conclusion
Our findings from the updated meta-analysis suggest that use of HCQ (with or without Azithromycin) did not result in decreased mortality nor were they any reduction in severe sequela of COVID 19, including transfer to the ICU, intubation, mechanical ventilation, virological cure or disease exacerbation. Rather use of these drugs are associated with a higher risk of adverse events, mainly gastrointestinal such as vomiting, diarrhea, and nausea. However, serious adverse events are rarely reported. Although the available evidence is of heterogeneous quality and, apart from the RECOVERY trial, there are only a few clinical trials with small sample sizes, our findings do not support any further use of either CQ or HCQ, with or without Azithromycin, for the treatment of COVID19.
Author contributions
TC – Conception and design of study, data curation (search for studies, data extraction and assessment of quality) formal analysis, software, writing original draft and revisions and final approval of submitted draft
OAHM - Data curation (search for studies, data extraction and assessment of quality), formal analysis, software, revisions and final approval of submitted draft
GH - Revisions and final approval of submitted draft
NW - Data curation (data extraction and assessment of quality), revisions and final approval of submitted draft
SB - Revisions and final approval of submitted draft
NS - Data curation (data extraction and assessment of quality), formal analysis, software, revisions and final approval of submitted draft
ATMA - Data curation (data extraction and assessment of quality), formal analysis, software, revisions and final approval of submitted draft
JTM - Data curation (search for studies), revisions and final approval of submitted draft MME - Revisions and final approval of submitted draft
LT - Conception and design of study, formal analysis, revisions and final approval of submitted draft
SARD - Conception and design of study, data curation, formal analysis, software, revisions and final approval of submitted draft
Conflict of interest
All the authors declare no conflict of interest
Funding
No funding to report
Abbreviations
COVID19, SARS-COV2, HCQ, CQ,
References
- 1.↵
- 2.↵
- 3.↵
- 4.
- 5.↵
- 6.↵
- 7.↵
- 8.↵
- 9.↵
- 10.↵
- 11.↵
- 12.↵
- 13.↵
- 14.↵
- 15.↵
- 16.↵
- 17.↵
- 18.↵
- 19.↵
- 20.↵
- 21.↵
- 22.↵
- 23.↵
- 24.↵
- 25.↵
- 26.↵
- 27.↵
- 28.↵
- 29.↵
- 30.↵
- 31.↵
- 32.↵
- 33.↵
- 34.↵
- 35.↵
- 36.↵
- 37.↵
- 38.
- 39.
- 40.↵
- 41.
- 42.
- 43.↵
- 44.
- 45.
- 46.↵
- 47.↵
- 48.
- 49.
- 50.↵
- 51.↵
- 52.↵
- 53.↵
- 54.↵
- 55.↵
- 56.↵
- 57.↵
- 58.↵
- 59.↵
- 60.↵
- 61.↵
- 62.↵
- 63.↵
- 64.↵
- 65.↵
- 66.↵
- 67.↵
- 68.↵
- 69.↵
- 70.↵
- 71.↵
- 72.↵
- 73.↵
- 74.↵
- 75.↵
- 76.↵
- 77.↵
- 78.↵
- 79.↵
- 80.↵
- 81.↵
- 82.↵
- 83.↵
- 84.↵
- 85.↵
- 86.↵