Abstract
Importance This systematic review and meta-analysis not only indicated no clinical benefits regarding hydroxychloroquine (HCQ) treatment with/without azithromycin for COVID-19 patients, but the higher mortality rate and frequency of known HCQ adverse effects were observed for the HCQ/HCQ+AZM regimen group.
Objective To overcome the controversies regarding the effectiveness of hydroxychloroquine in the treatment of COVID-19 through meta-analysis.
Data Sources A systematic search was performed in PubMed, Scopus, Embase, Cochrane Library, Web of Science, Google Scholar and medRxiv pre-print database using all available MeSH terms for COVID-19 and hydroxychloroquine up to May 14, 2020.
Study Selection Studies focused on the effectiveness of HCQ with/without azithromycin (AZM) in confirmed COVID-19 patients were entered into the study including: comparative or non-comparative clinical studies including observational/interventional studies with retrospective/prospective nature with/without control group as well as Randomized Clinical Trials (RCTs).
Data Extraction and Synthesis Two researchers have independently evaluated quality assessment of the studies and abstracted data for data extraction, which has reviewed by the supervisor for accuracy. Extracted data were analyzed using CMA v. 2.2.064. Heterogeneity was assessed using the I-squared (I2) test, and fixed/random-effects model was used when appropriate for pooling of studies.
Main Outcome(s) and Measure(s) 1) Clinical effectiveness of HCQ with/without AZM in the treatment of COVID-19; 2) Mortality rate; 3) Disease exacerbation; 4) Frequency of known HCQ adverse effects occurred during treatment; 5) Intubation need; 6) Radiological improvement.
Results Out of 21 studies entered into our systematic review, 18 studies including 11 comparative studies with control group and seven observational studies containing 5,561 participants have entered into the study. The results of the meta-analysis on comparative studies indicated no significant clinical effectiveness (negative in RT-PCR evaluation) for HCQ regimen in the treatment of COVID-19 in comparison to control group (RR: 0.96, 95% CI, 0.76-1.22). The same result was observed for the combination of HCQ+azithromycin (RR: 2.15, 95% CI, 0.31-14.77). Approximately 1.5 times higher mortality rate was observed among the HCQ regimen group (RR: 1.54, 95% CI, 1.04-2.29) and 3.5 times higher in HCQ+AZM (RR: 3.65, 95% CI, 1.10-12.10) group in comparison to control group, which was affected by age differences according to meta-regression analysis (P=0.0003). No substantial difference was observed for disease exacerbation (RR: 1.60, 95% CI, 0.54-4.71) between HCQ group and controls. Also, radiological findings significantly improved in the HCQ group (OR: 0.32, 95% CI, 0.11-0.98). Odds of known HCQ adverse effects (diarrhea, vomiting, blurred vision, rash, headache, etc.) occurred in the HCQ regimen group was approximately 3.5 times of control group (OR: 3.55, 95% CI, 1.61-7.82), but no substantial differences were found regarding intubation odds between HCQ group and control group (OR: 2.11, 95% CI, 0.31-14.03).
Conclusions and Relevance This systematic review and meta-analysis not only showed no clinical benefits regarding HCQ treatment with/without azithromycin for COVID-19 patients, but according to multiple sensitivity analysis, the higher mortality rates were observed for both HCQ and HCQ+AZM regimen groups, especially in the latter. Also, frequency of adverse effects was higher in HCQ regimen group. However, due to that most of the studies were non-randomized and results were not homogenous, selection bias was unavoidable and further large randomized clinical trials following comprehensive meta-analysis should be taken into account in order to achieve more reliable findings.
Question Is hydroxychloroquine effective and safe against COVID-19?!
Findings This systematic review and meta-analysis on 18 studies including 5,561 participants indicated no significant clinical effectiveness for hydroxychloroquine with/without azithromycin against COVID-19 patients. Also, the higher mortality rate and adverse effects were observed in the HCQ/HCQ+AZM regimen group.
Meaning It seems that hydroxychloroquine does not works as expected and attracted attentions as a potential agent against COVID-19, so, we should be cautious about the treatment decision.
Introduction
A novel coronavirus emerged from Wuhan, China, in December 2019 has named respiratory syndrome coronavirus 2 (SARS-CoV-2) and declared as a pandemic by World Health Organization (WHO) on March 26, 20201. According to Worldometer metrics, this novel virus has been responsible for approximately 4,507,689 infections and 302,315 death worldwide up to May 14, 2020.
Although a few months have passed since the onset of the new challenging disease, there is still no specific preventive and therapeutic approach in this regard. Therefore, the quarantine approach, personal hygiene, and social distancing are the basic protective measures against Coronavirus Disease 2019 (COVID-19) according to WHO advice for the public 2.
Moreover, according to a large amount of ongoing research regarding this pandemic issue, many controversies are arising daily among different fields of sciences, which has confronted a “pandemic” with an “infodemic” (e.g. Is coronavirus an airborne? Is COVID-19 transmitted vertically in pregnancy? Should everyone wear a mask? How long can the virus survive on surfaces? Is it possible to get COVID-19 for a second time? etc.).
In this regard, one of the hottest controversies is the hydroxychloroquine (HCQ) efficacy with/without azithromycin (AZM) for COVID-19 patients. While, several studies are talking about promising effects of HCQ regimen against SARS-CoV-2 infection for both prevention and treatment 3-5, others try to come up with neutral or even harmful effects of this drug for such patients when there is no ample evidence 6. It is unavoidable that all these controversies affect the patient’s outcome significantly.
Although there are some systematic reviews and meta-analysis in this regard 7,8, they suffer from several limitations and flaws that should be addressed with a more comprehensive study. Hence, due to the importance of the subject we carried out a rapid systematic review and meta-analysis, which will be updated periodically to report stronger evidence, which might help to overcome the controversies about the effectiveness of HCQ against COVID-19.
Method
Study design & information
The current study has designed by The Tabari Systematic Review Group (TSRG) with an updating approach named “Periodic Systematic Review and Meta-analysis”, which will be updated time to time whenever a new substantial study is published. Also, different versions of this study will be released in our newly under design platform at https://rtp.mazums.ac.ir/Page/9/Tabari-Systematic-Review-Group.
Search Strategy
The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guideline was followed for study design, search protocol, screening, and reporting. A systematic search was performed via databases of PubMed, Scopus, Embase, Cochrane Library, Web of Science and Google Scholar (intitle) as well as preprint database of medRxiv up to May 6, 2020. Moreover, gray literature and references of eligible papers were considered for more available data in this case. The search strategy included all MeSH terms and free keywords found for COVID-19, SARS-CoV-2, and hydroxychloroquine. There was no time/location/ language limitation in this search.
Criteria study selection
Two researchers (A.H and K.H) have screened and selected the papers independently and discussed to solve the disagreements with the third-party (R.A/D.Sh). Studies met the following criteria included into meta-analysis: 1) comparative or non-comparative clinical studies including observational/interventional studies with retrospective/prospective nature with/without control group as well as Randomized Clinical Trials (RCTs); and 2) studies reported the effect of HCQ with/without AZM in confirmed COVID-19 patients. Studies were excluded if they were: 1) animal studies, reviews, case reports, and in vitro studies; 2) duplicate publications; and 3) insufficient for calculating of desired parameters.
Data extraction & quality assessment
Two researchers (A.Sh and A.H) have independently evaluated quality assessment of the studies and extracted data from selected papers. The supervisors (D.Sh/MA.E) resolved any disagreements in this step. Data extraction checklist included the name of the first author, publication year, region of study, number of patients, number of controls, mean age, treatment option, medication dosage, treatment duration, adverse effects, radiological results, nasopharyngeal culture status through Reverse Transcription-Polymerase Chain Reaction (RT-PCR) and mortality.
The Jadad scale, ROBINS-I tool and Newcastle-Ottawa Scale (NOS) checklists were used to value the selected randomized controlled trials, non-randomized controlled trials and observational studies, respectively concerning various aspects of the methodology and study process. Risk-of-bias plots have been created through the robvis online tool 9.
Targeted outcomes
1) Clinical effectiveness of HCQ with/without AZM in the treatment of COVID-19; 2) Mortality rate; 3) Disease exacerbation; 4) Frequency of known HCQ adverse effects occurred during treatment; 5) Intubation need; 6) Radiological improvement.
Comparisons
1) HCQ in comparison to control group with standard treatment; 2) HCQ+AZM in comparison to control group with standard treatment.
Definitions
Clinical effectiveness: nasopharyngeal swab culture resulted negative in RT-PCR evaluation.
Disease exacerbation: clinical symptoms of the disease are worsened.
Adverse effects: occurrence of known symptoms related to HCQ such as diarrhea, vomiting, blurred vision, rash, headache, etc.
Group-A in forest plots: the case groups who receive HCQ with/without AZM regimen.
Group-B in forest plots: the control groups without HCQ/HCQ+AZM regimen.
Heterogeneity assessment
I-square (I2) statistic was used for heterogeneity evaluation. Following Cochrane Handbook for Systematic Reviews of Interventions 10, the I2 was interpreted as follows: “0% to 40%: might not be important; 30% to 60%: may represent moderate heterogeneity; 50% to 90%: may represent substantial heterogeneity; 75% to 100%: considerable heterogeneity. The importance of the observed value of I2 depends on (i) magnitude and direction of effects and (ii) strength of evidence for heterogeneity (e.g. P-value from the chi-squared test, or a confidence interval for I2).”
In case of heterogeneity, DerSimonian and Laird random-effects model was applied to pool the outcomes; otherwise, the inverse variance fixed-effect model was used. Forest plots were presented to visualize the degree of variation between studies.
Data analysis
Statistical analysis was performed using Comprehensive Meta-Analysis (CMA) v. 2.2.064 software. Risk Ratio (RR) or Odds Ratio (OR) were used for outcome estimation whenever appropriate with 95% Confident Interval (CI). Fixed/random-effects model was used according to heterogeneities. In case of zero frequency, the correction value of 0.1 was used. Meta-regression analysis was done to examine the impact of age difference on HCQ regimen group mortality RR. However, due to insufficient data we could not apply the meta-regression analysis on the other moderator variables such as sex, underlying disease, etc.
Publication bias & sensitivity analysis
Begg’s and Egger’s tests as well as funnel plot was used for publication bias evaluation. P-value less than 0.05 was considered as statistically significant.
According to that the sensitivity analysis known as an essential part in systematic reviews with meta-analysis to determine the robustness of the obtained outcomes to the assumptions made in the data analysis 11, we conducted a sensitivity analysis to examine the effect of studies that greatly influence the result, especially by their weight through excluding them from the meta-analysis.
Results
Study selection process
The databases search resulted in 352 papers. Eighty-nine duplicated papers have been excluded and after first step screening, fifteen papers were assessed for eligibility. Finally, 21 papers entered into qualitative synthesis, of which 18 papers entered into the meta-analysis. PRISMA flow diagram for the study selection process presented in Figure 1.
Study characteristics
Out of 18 studies entered into the meta-analysis, 11 studies were comparative randomized and non-randomized (only two studies were randomized) and seven were observational, of which, HCQ arms of the comparative studies has been combined with observational studies for effect size meta-analysis. The studies’ sample size ranged from 10 to 1438 including 5,561 participants. Characteristics of studies entered into the systematic review presented in Table 1.
Quality assessment
Results of quality assessment for studies entered into meta-analysis using Jadad, ROBINS-I and NOS checklists were reported in Table 1 and summary of risk of bias has presented in Fig. 2.
Publication bias
Results of Begg’s and Egger’s tests for every performed analysis were insignificant as follows: hydroxychloroquine regimen effectiveness (PB=0.42; PE=0.55), association between hydroxychloroquine regimen and mortality rate in controlled randomized and non-randomized studies (PB= 1.00; PE=0.81) and overall mortality in all studies (PB=0.09; PE= 0.01). The funnel plot for publication bias of studies presented in Supplementary Fig. 1.
Meta-analysis findings
Treatment outcome
Hydroxychloroquine regimen effectiveness
The meta-analysis of risk ratios (RR) for HCQ effectiveness in all of the comparative randomized and non-randomized studies showed that there were no significant differences between case group, who received the standard treatment with HCQ regimen and the control group that received the standard treatment without HCQ (RR: 0.96, 95% CI, 0.76-1.22). We also found no significant risk difference (RD) between two groups regarding effectiveness of HCQ in COVID-19 patients (RD: 0.00, 95% CI, -0.18-0.18) (Fig. 3). Also, no substantial effectiveness for HCQ was found by meta-analysis of only controlled randomized studies as well (RR: 1.19, 95% CI, 0.87-1.63; RD: 0.12, 95% CI, -0.07-0.33) (Fig. 4).
Sensitivity analysis for hydroxychloroquine regimen effectiveness
Regardless of separate analysis only on controlled randomized studies (Fig. 4), to evaluate the impact of inverse RRs as well as studies’ weight on the meta-analysis results, we conducted several sensitivity analyses as follows: 1) according to the substantial relative weight of three studies on the meta-analysis, by excluding these studies as follows no significant changes was observed: Jun (B) et al. (RR: 0.98, 95% CI, 0.73-1.32), Magagnoli et al. (RR: 1.02, 95% CI, 0.75-1.39), and Tang et al. study (RR: 0.95, 95% CI, 0.69-1.32) (Supplementary Fig. 2-4); 2) Exclusion of above three studies was also resulted in no significant differences (RR: 1.28, 95% CI, 0.49-3.33) (Supplementary Fig. 5); 3) considering four studies with P-value less than 0.05; two have a P<0.05 in favour of Group-A and two has a P<0.05 in favour of Group-B; these are the Magagnoli et al. and Mallat et al. studies, for which the 95% CI of RR has no intersection with the Chen et al. and Gautret (B) et al., thus, the new sensitivity analysis by excluding studies of Magagnoli et al. Mallat et al., resulted in no difference as well (RR: 1.15, 95% CI, 0.88-1.49) (Supplementary Fig. 6); and 4) by excluding both studies in favour of Group-A, meta-analysis showed no significant difference similarly (RR: 0.86, 95% CI, 0.70-1.05) (Supplementary Fig. 7).
Hydroxychloroquine + azithromycin regimen
No significant difference was found for effectiveness of HCQ+AZM combination regimen in comparison to control group in comparative studies entered into meta-analysis (RR: 2.15, 95% CI, 0.31-14.77; RD: 0.37, 95% CI, -0.58-1.33) (Fig. 5).
Hydroxychloroquine regimen & mortality rate
The meta-analysis of death outcome in comparative randomized and non-randomized studies showed no significant differences for mortality rate between HCQ regimen group and standard treatment group (RR: 1.03, 95% CI, 0.50-2.09; RD: -0.02, 95% CI, -0.12-0.08) (Fig. 6).
Sensitivity analysis for hydroxychloroquine regimen mortality
Due to the surprising mortality rate in the control group of the Yu et al. and Membrillo et al. studies, which led to the inverse RR in comparison to other studies and high weight in final estimation in our analysis, we have conducted a sensitivity analysis by excluding these studies. The sensitivity analysis resulted in approximately two times higher mortality rate in HCQ regimen arm in comparison to control group (RR: 1.81, 95% CI, 1.31-2.49) (Supplementary Fig. 8). In addition, concerning significant influence of Magagnoli et al. study on mortality rate (P=0.001), we have performed another sensitivity analysis by excluding above three studies, which indicated about 1.5 higher mortality rate in the HCQ regimen group than the control group in randomized and non-randomized studies (RR: 1.54, 95% CI, 1.04-2.29) (Supplementary Fig. 9).
Meta-regression analysis on effects of age differences on mortality
Meta-regression findings indicated that age differences of studies had substantial effects on risk ratios related to the HCQ regimen group mortality rate (P=0.0003) (Fig. 7).
Hydroxychloroquine + azithromycin regimen & mortality rate
The meta-analysis of mortality outcome in comparative randomized and non-randomized studies indicated approximately 3.5 times higher mortality rate in HCQ+AZM regimen group in comparison to control group (RR: 3.65, 95% CI, 1.10-12.10; RD: 0.40, 95% CI, -0.18-1.00) (Fig. 8).
Overall mortality
For overall mortality we considered the treatment arms of the all comparative studies as observational studies and combined all types together as observational. Pooling mortality rate of studies resulted in overall mortality of 10% (95% CI, 6%-16%) for both HCQ and HCQ+AZM regimen (Fig. 9).
Disease exacerbation
Meta-analysis of all comparative studies showed that the disease exacerbation was not significantly different between HCQ group and control group (RR: 1.60, 95% CI, 0.54-4.71; RD: 0.04, 95% CI, -0.05-0.14) (Fig. 10). Also, doing meta-analysis only on controlled randomized studies indicated no considerable disease exacerbation differences between two groups (RR: 0.59, 95% CI, 0.04-7.79; RD: -0.08, 95% CI, -0.32-0.14) (Fig. 11).
Intubation
Meta-analysis of comparative randomized and non-randomized studies indicated that there were no significant differences between HCQ group and control group about odds of intubation during treatment (OR: 2.11, 95% CI, 0.31-14.03) (Fig. 12).
Adverse effects
The meta-analysis of comparative randomized and non-randomized studies showed that the odds of adverse effects occurrence in patients who received HCQ regimen was approximately 3.5 times higher than control group without HCQ regimen (OR: 3.55, 95% CI, 1.61-7.82) (Fig. 13). Likewise, doing meta-analysis only on controlled randomized studies indicated 4.5 times higher odds of experiencing adverse effects in patients who receive HCQ regimen in comparison to control group (OR: 4.53, 95% CI, 1.86-11.03) (Fig. 14). However, due to high relative weight of Tang et al. study, the sensitivity analysis by excluding this study resulted in no significant differences for odds of adverse effects between two groups (OR: 1.97, 95% CI, 0. 44-8.77) (Supplementary Fig. 10).
Radiological improvement
A considerable Computed Tomography-Scan (CT-Scan) improvement was observed in HCQ group with odds ratio of 0.32 (95% CI, 0.11-0.98) (Fig. 15).
Meta-analysis of observational studies (Fig. 16)
We considered the treatment arms of the comparative studies as observational studies for this section. Hence, the meta-analysis of the other events rate showed the following results: 23% of patients suffered from known HCQ adverse effects (95% CI, 14%-36%); CT-Scan improvement has been observed in 16% of COVID-19 patients (95% CI, 2%-57%); 71% (95% CI, 58%-81%) of patients were discharged from the hospital or their nasopharyngeal culture resulted negative in RT-PCR evaluation, whereas 12% (95% CI, 2%-38%) of patients have been exacerbated, 7% (95% CI, 2%-17%) were admitted to the intensive care unit (ICU) and 23% (95% CI, 6%-57%) underwent intubation.
Discussion
More than three months after the closure of the Hubei region, there is still very little high-quality data on every treatment’s regimen, which raises questions about gaps in scientific works. In this context, one can only be surprised to see observational data without a control group covering a large number of patients when there is an essential need for comparative randomized data. In fact, Analysis of current data is difficult because at least 90% of spontaneous recovery are observed at the base. Thus, it takes well over 90% of recovery to consider that any treatment really brings something. The HCQ was no exception to this rule with a result greater than 90%. On the other hand, in a few percent, the disease becomes serious with Acute Respiratory Distress Syndrome (ARDS) and multi-organ failure. This is where we would like to know what the treatments bring. Concerning all of the limitations and analyze difficulties, we have conducted this systematic review and meta-analysis with great caution and sensitivity in performing analyzes in order to try for overcoming the current controversies regarding effectiveness of HCQ in the treatment of COVID-19, at least at the base.
To be honest, we know that all the studies taken into account should not be so if we wanted to respect the gold standard of performing a meta-analysis. However, few studies are currently available and the idea of this work is to maximize the use of everything that exists. Especially since we know, in principle, that observational studies give an overestimation of the value sought. Thus, whatever the result with the HCQ we can say that the result, by our approach, is also an overestimation of reality; which is very interesting in itself to get now.
Considering the matter, recent investigations indicated that a high concentration of cytokines in the plasma called cytokine storm would be related to severe COVID-19 patients. In this situation, medications transposition is a critical need to find effective anti-inflammatory agents to decrease the cytokines and pro-inflammatory factors production 32 In this regard, HCQ has been known as an effective anti-inflammatory agent for a long time (since the 1950s), especially in autoimmune disorders 33. Besides, the outcome of a new experimental study conducted by Liu et al. has been mentioned in the title of their publication as follows: “Hydroxychloroquine, a less toxic derivative of chloroquine, is effective in inhibiting SARS-CoV-2 infection in vitro”. This also has been investigated and resulted same as the in vitro study of Yao et al. 34
In addition, Pagliano et al. in their letter to the editor of Clinical Infectious Diseases (CID) journal, have been recommended the use of HCQ as pre/post-exposure prophylaxis against SARS-CoV-2 infection for health care staffs exposed to the virus in contaminated environments35.
In contrast, Maurizio Guastalegname and Alfredo Vallone are claiming about the uselessness and even harmful effects of HCQ against COVID-19 in their letter to the editor in the above journal 6. They believe that, while the pathogenicity of the SARS-CoV-2 is still unknown, we should be cautious about the treatment decision, which has been proved through in vitro base studies in order to avoid dire paradoxical consequences like what has happened in treatment of Chikungunya Virus infection with chloroquine 36. Moreover, Molina et al. have followed 11 patients with HCQ + azithromycin regimen and concluded no clinical benefit and reasonable anti-viral activity 19 In addition, the pre-print of a Quasi-Randomized Comparative Study conducted in Detroit, Michigan, has been indicated not only any clinical benefits for HCQ but even increased need for urgent respiratory support (p=0.013) 21.
Also, H.J. Kim et al. in their opinion publication for the COVID-19 Global Rheumatology Alliance pointed at the shortage of HCQ following a sudden high demand after Gautret and colleagues’ publication on 20 March 2020 17 They also referred to that HCQ is a crucial treatment choice for patients with systemic lupus erythematosus and rheumatoid arthritis disorders, who get into trouble in finding HCQ in this critical time 37 Authors recommend that scientific communities have to be very cautious and do not rush in the decision when there is no ample evidence for the subject, especially in such critical situations, which can lead to irreparable consequences. In fact, even if the efficacy of HCQ is confirmed, the world will be facing a new issue for both COVID-19 and rheumatic disorders patients: “Shortage of Hydroxychloroquine".
More recently, the preprinted study of Magagnoli et al. 18 on 368 United States veterans reported not only any clinical benefits for HCQ/HCQ+AZM regimen in COVID-19 patients but even association between higher mortality rate and HCQ group (hazard ratio: 2.61, 95% CI, 1.10-6.17; P=0.03). Also, the target trial emulation conducted by Mahévas et al. 14, did not support the effectiveness of the HCQ regimen, which has performed on 181 patients with SARS-CoV-2 hypoxic pneumonia.
Observational Study of Geleris et al. 30 in New York City on 1376 COVID-19 patients concluded that “the study results should not be taken to rule out either benefit or harm of hydroxychloroquine treatment, given the observational design and the 95% confidence interval”. However, due to that investigators have combined the results of intubation and death together, we could not enter the results into meta-analysis. Another study conducted in New York State by Rosenberg et al. 28 on “Association of Treatment With HCQ/AZM With In-Hospital Mortality in COVID-19 Patients” has also similarly concluded no association for lower in-hospital mortality for both medications.
In this case, we carried out the present systematic review in order to reach a clear result regarding taking or not-taking HCQ. In this study, meta-analysis revealed no significant differences between HCQ arm who received 400-600 mg HCQ daily regimen and standard treatment arm, which was the same for the HCQ+AZM regimen as well. Surprisingly, not only no clinical benefits were found regarding HCQ treatment with/without AZM for COVID-19 patients, but based on multiple sensitivity analysis the higher mortality rates were observed for both HCQ and HCQ+AZM regimen groups in comparison to control groups. For mortality, considering meta-regression analysis, age differences were significantly affected obtained risk ratios. In addition, more cases in the HCQ group presented the improvement in CT-Scan results in comparison to the control group. Moreover, the considerable higher frequency of known HCQ adverse effects such as diarrhea, vomiting, blurred vision, rash, headache, etc. was observed in HCQ groups.
It is remarkable that, Sarma et al. 8 conducted a meta-analysis on three studies and has been concluded the promising effects for HCQ clinical cure, which is inconsistent with our overall conclusion. Additionally, more recently, Million et al. 7 carried out “a meta-analysis based on the first available reports” released in IHU Méditerranée Infection (COVID-IHU #10), in which they have claimed a promising trend toward the chloroquine derivatives benefits against COVID-19 and suggested it to prescribe as a grade I recommendation. Going deeper into the content, the study suffers from some flaws such as ignoring heterogeneity as well as the pattern of dispersion in the results, combining different outcomes in an unusual way, using only odds ratios, when the risk ratios are the priority and preferred in such analysis, talking about four RCTs, whereas some are non-randomized, etc. Also, concerning the importance of mortality, Magagnoli et al. 18 findings, which go against the expected by authors, is ignored without any sensitivity analysis.
As a matter of fact, “statistics are only a tool for understanding and not an end in itself. If it is easy to say anything with statistics, it is also easy to say that you can make them say anything. However, using it well, with a strong consideration of the clinical sciences and an understanding of the possible sense of bias can go very far”, Dr. Rauss says.
While our analysis did not indicate a considerable effectiveness for the HCQ/HCQ+AZM regimen, due to the small number of studies with a small sample size, it is too early to reach a reliable decision. In fact, the possible influence of biases on the results in analysis of such data is inevitable, but what is more important is to know in which direction, the biases can play the role on the results. Thus, as we know, observational studies give a vision that overestimates the real result. Under these conditions, when we incorporate such data, and the result is not good, it means that we have a minimized vision of the result. Hence, the question arises whether the result might be even worse than what is observed?! If yes, then it would be understood that “impact of this meta-analysis finings with strong consideration of the clinic and an understanding of the possible sense of bias is much greater than what is observed and presented”; Dr. Rauss says. Hence, it seems that we are very far from an effective treatment and the urgent need arises for comprehensive randomized controlled trials in order to investigate the efficacy and safety of HCQ consumption and other treatment regimen and approaches in COVID-19 patients. In point of fact, the studies to come should completely reverse what has already been done and just this point is already an important information because the probability is, in fact, extremely low.
In this regard, searching clinical trials registry databases such as WHO International Clinical Trials Registry Platform (ICTRP), Clinicaltrials.gov, Center Watch, Chinese Clinical Trials Registry, International Standard Randomised Controlled Trial Number (ISRCTN), EU Clinical Trials Register, OpenTrials and Iranian Registry of Clinical Trials, resulted in hundreds registered trials on the subject of efficacy, safety and prophylaxis potency of HCQ in COVID-19 patients. Hence, in the near future, the results of these trials will help the medical communities to reach a general opinion regarding the utilization of HCQ as a pre/post-exposure as well as a treatment option for patients infected with SARS-CoV-2.
To our knowledge, this is the most updated systematic review that carried out a meta-analysis for investigating the role of HCQ with/without AZM in COVID-19 patients. However, after releasing outcomes of underway clinical trials, especially “Solidarity” clinical trial for COVID-19 treatment by WHO, an updated systematic review and meta-analysis on this subject could be more conclusive and reliable.
It is worth noting that the current meta-analysis includes the following limitations: 1) due to that most of the studies were non-randomized and results were not homogenous, selection bias was unavoidable; 2) various treatment plan regarding medication dosage and treatment duration; 3) insufficient moderator variables distribution by group such as sex, underlying disease, etc.; and 4) inclusion of studies with small sample size, which leads to type II statistical errors. To overcome the limitations and bias, results of the study should be confirmed by robustly randomized studies.
Conclusion
This systematic review and meta-analysis indicated no clinical benefits regarding hydroxychloroquine treatment with/without azithromycin for COVID-19 patients. Surprisingly, according to multiple sensitivity analysis the higher mortality rates were observed for both HCQ and HCQ+AZM regimen groups, especially in the latter, which was affected by age factor. Also, frequency of known HCQ adverse effects was higher in this regimen group in comparison to the controls. However, due to that most of the studies were non-randomized and results were not homogenous, selection bias was unavoidable, further large clinical trials following comprehensive meta-analysis should be taken into account in order to achieve more reliable findings.
Conflict of interests
The authors declare that they have no conflict of interests.
Acknowledgments
We would like to express our deepest appreciation to Dr. Alain Rauss, who extended a great amount of assistance in regards to clinical and statistical contents of the study with his insightful advice and comments. Our gratitude should also go to the Student Research Committee of Mazandaran University of Medical Sciences for supporting this student research proposal with the code 7586.