Abstract
Objectives Several drugs are being repurposed for the treatment of the coronavirus disease 2019 (COVID-19) pandemic based on in vitro or early clinical findings. As these drugs are being used in varied regimens and dosages, it is important to enable synthesis of existing safety data from clinical trials. However, availability of safety information is limited by a lack of timely reporting of clinical trial results on public registries or through academic publication. We aimed to analyse the knowledge gap in safety data by quantifying the number of missing clinical trial results for drugs potentially being repurposed for COVID-19.
Design ClinicalTrials.gov was searched for 19 drugs that have been identified as potential treatments for COVID-19. Relevant clinical trials for any prior indication were listed by identifier (NCT number) and checked for timely result reporting (within 395 days of the primary completion date). Additionally, PubMed and Google Scholar were searched using the NCT number to identify publications of results not listed on the registry. A second, blinded search of 10% of trials was conducted to assess reviewer concordance.
Results Of 3754 completed trials, 1516 (40.4%) did not post results on ClinicalTrials.gov or in the academic literature. 1172 (31.2%) completed trials had tabular results on ClinicalTrials.gov. A further 1066 (28.4%) completed trials had results from the literature search, but did not report results on ClinicalTrials.gov. Key drugs missing clinical trial results include hydroxychloroquine (37.0% completed trials unreported), favipiravir (77.8%) and lopinavir (40.5%).
Conclusion There is an important evidence gap for the safety of drugs being repurposed for COVID-19. This uncertainty could cause a large burden of additional morbidity and mortality during the pandemic. We recommend caution in experimental drug use for non-severe disease and urge clinical trial sponsors to report missing results retrospectively.
Introduction
Coronavirus disease 2019 (COVID-19) is a pandemic infection caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Its global spread has been rapid and unprecedented, at the time of writing 5,335,868 confirmed cases have been reported and 341,549 deaths across 188 countries (1). Currently, treatment options for COVID-19 are limited. However, several drugs developed for other indications have shown promising results against SARS-CoV-2 in vitro, in animal models, or in compassionate use trials (2,3). Many of these drugs are now being experimentally repurposed for COVID-19 or are undergoing clinical trials in humans (4). Such candidates include nitazoxanide, remdesivir, favipiravir, lopinavir, darunavir, hydroxychloroquine, chloroquine, and ivermectin amongst others (5).
Investigations into the efficacy of these experimental treatments for COVID-19 are ongoing. Meanwhile, in response to positive media coverage, some speculative rather than evidence-based, governments are stockpiling vast supplies of these treatments in anticipation of their licensing for COVID-19. Furthermore, national regulatory institutions, meant to safeguard against the unsafe use of drugs, are under increasing pressure to relax approval standards to accelerate market-entry for COVID-19 treatments. For example, on April 27th 2020 the U.S. Food and Drug Administration (FDA) approved the antimalarial drug hydroxychloroquine for emergency treatment of COVID-19 with unknown optimal dosage and duration of treatment (6). However, the efficacy of hydroxychloroquine is still under question (7). Furthermore, it is cardiotoxic at the higher doses which may be indicated for COVID-19, causing QT prolongation leading to ventricular tachycardia and death (8). Care must be taken not to lose the rigorous safety standards usually stipulated for pharmaceuticals, even during a pandemic, to avoid unnecessary morbidity and mortality worldwide.
Given the pandemic status of COVID-19, it is more important than ever that safety information for drugs potentially being repurposed is publicly available. Pharmacokinetics for many drugs differ by phenotype. Examples include ACE inhibitors, which are cautioned in people of African descent, and dolutegravir, which causes greater weight gain in women than men (9,10). As pharmaceutical companies often run trials in high income countries, treatment cohorts are often largely white (11). Furthermore, regulatory authorities are often conservative about including pregnant women in trials, discouraging female recruitment (11). As experimental treatments for COVID-19 may be distributed to millions of people worldwide, maximising phenotypic variety as well as the total number of person-years-of-follow-up available in the public realm is vital for ensuring that rare adverse events and contraindications can be anticipated.
As many of these drugs have been used widely for years, there should be substantive information on safety, tolerability and pharmacokinetics available in the public domain, including public trial registries. Yet, whilst the International Committee of Medical Journal Editors (ICMJE) policy requires prospective registration of interventional studies on a WHO primary registry or on ClinicalTrials.gov, the ICMJE does not currently require researchers to report summary results on these registries (12). However, according to the FDA Amendment Act 2007, the responsible party for applicable clinical trials that are registered on ClinicalTrials.gov must report results to a public register within twelve months of the primary completion date, or in some cases risk a fine of $11,000 for every day results are delayed (13,14). Failure to share clinical trial results publicly can additionally have far-reaching consequences for health and public expenditure, as illustrated by the widespread stockpiling and prescription of oseltamivir (Tamiflu) during the swine flu outbreak in 2009, despite a lack of evidence on safety and efficacy (15). Clinical trial transparency is therefore vital in order to maximise and unify the sharing of safety information and data on efficacy, during this pandemic and beyond.
Public clinical trial registries are an important tool for transparent collaborative research. On these registries, safety and efficacy data can be uploaded freely, shortly after completion of the study, and protocol and data collection methods are still quality assessed (16,17). In contrast, academic publication may stall for significant periods of time as can be costly and selective, with time-intensive writing and review processes. Furthermore, clinical trial registry data is available free of charge and can be pooled without concern for silent outcome switching or publication bias(18–20). This is especially important during the COVID-19 pandemic, as interest in potential treatments for coronavirus generates even greater incentive than normal for the publication of studies with positive results only. Without rapid sharing of datasets for drugs that may be repurposed for COVID-19, secondary analyses of safety data will be arduous and often incomplete (16,21). This may slow down the biomedical innovation process and could lead to preventable side effects occurring in vulnerable patients if safety information remains missing. As the pharmaceutical pipeline is accelerated to address the COVID-19 pandemic, enhancing clinical trial transparency is more important than ever.
In this study, we aimed to determine the scale of unpublished clinical trial results which may hinder safety reviews of repurposed drugs for COVID-19. We reviewed the number of completed or terminated trials that have not reported results for an extensive, list of medications being repurposed for COVID-19, looking at all previous indications for these drugs. Specifically, we searched for any trial results that have not been made available to the public, with no results published on either the http://ClinicalTrial.gov registry or in the academic literature.
Methods
The 19 drugs assessed were pirfenidone, hydroxychloroquine, azithromycin, favipiravir, oseltamivir, sarilumab, tocilizumab, remdesivir, leflunomide, interferon-alpha, lopinavir-ritonavir, darunavir-ritonavir, baloxivir marboxil, umifenovir, interferon-beta, sofosbuvir, nitazoxanide, APN01 and ivermectin (Table 1). These drugs were selected based on information found in potential COVID-19 treatment reviews (5,22,23). Synonyms and chemical names for these drugs were taken from http://pubchem.ncbi.nlm.nih.gov (Appendix 1) (24).
Trials registered to NCT for repurposed drugs, of which ongoing and suspended or withdrawn.
The U.S. clinical trials registry (ClinicalTrials.gov) was searched for all trials that listed these drugs as an intervention, results of the search were downloaded on 4th April 2020 (25). Numbers of trials with and without results on the registry were recorded. For trials without results, trial status was determined (completed, ongoing, suspended, terminated or withdrawn). Trials listing ‘primary completion date’ in the future were counted as ongoing, if no primary completion date was available then the study completion date was used. Listed trial status was used to identify ongoing, terminated, suspended and withdrawn trials.
For all trials without results on ClinicalTrials.gov, a three-step process was followed between 4th – 27th April 2020 to determine whether results were reported elsewhere through academic publication (Figure 1).
Publications automatically indexed by NCT number on ClinicalTrials.gov were screened and included based on criteria below. If multiple publications were listed, the earliest dated publication was selected.
If results were not available on the registry, the clinical trial identifier (NCT number) was used to search and screen academic publications in PubMed.
If the PubMed search did not retrieve an academic publication, an additional search was conducted in Google Scholar using the following search terms in succession: clinical trial identifier, listed title, intervention name with primary investigator’s name. For each search, the first twenty results were screened.
Flow chart describing the methodology used to search for and identify relevant publications for each of the trials listed on http://CT.gov
If a publication did not include the clinical trial identifier, it was cross-referenced with the primary investigator, study design, intervention and outcomes listed on ClinicalTrials.gov to assess relevance. We excluded publications that had fewer than 500 words, as well as conference abstracts, posters, presentations and non-English texts.
Publications of results were recorded by PMID, DOI and publication date. Trials with results on ClinicalTrials.gov combined with those with a journal publication gave a total number of trials with results where results were found in the public domain. This allowed approximation of registered trials without results.
Additionally, overdue trials were calculated as any completed trial with no result on the registry and a primary completion date before 18th April 2019, 395 days prior to final analysis (1 year + 30-day grace period). This is the standard outlined in the FDAAA 2007 and used as a reference throughout this study despite not all included trials being covered by the law (13). This is also consistent with international ethical standards for timely results dissemination (26).
A second review was conducted by a different researcher on 10% of trials for each drug to check concordance between reviewers. The protocol during the second review remained unchanged and researchers were blinded to the results of the first review. A random number generator was used to select trials for second review. Concordance was calculated by simple percent agreement on a results judgement; above 80% was deemed acceptable from consulting experts and in line with recommendations in the literature (27). Results published between the dates of first and second review (29th April – 9th May 2020) were not counted in the concordance.
Patient and Public Involvement
Patients and the public were not involved in any way in this research.
Results
Nineteen drugs were screened, encompassing a total of 4553 clinical trials registered on ClinicalTrials.gov (Table 1). We excluded 799, with 661 ongoing (primary completion date not passed) and 138 with a trial status of suspended or withdrawn. Figure 2 shows the number of trials found on ClinicalTrials.gov, those excluded from this analysis, and the final results status of all included trials (n = 3754). All recorded percentages in text are in relation to the 3754 completed trials, seen in Table 2.
Flow diagram displaying the numbers of registered trials identified on http://CT.gov and the proportion of these which ultimately have no results available. The number of trials which had results available from various sourc es or had been withdrawn/suspended or trails for which results were not yet due are also shown.
Number of completed trials registered on ClinicalTrials.gov, of which results have been published on ClinicalTrials.gov or in the academic literature.
In sum, our protocol revealed 2238 (59.6%) completed trials had published results either on the registry or in the academic literature (Table 2). Of these, 1172 (31.2%) completed trials had tabular results on ClinicalTrials.gov (Table 2). A further 1066 (28.4%) completed trials had results from the literature search, but did not report results on ClinicalTrials.gov (Table 2). Across the 19 drugs which may be repurposed for the treatment of COVID-19, 1516 (40.4%) of completed clinical trials listed on ClinicalTrials.gov were missing results. Figure 3 shows the proportions of trial results available on ClinicalTrials.gov, available in the literature, and those with no results available.
Bar chart displaying the proportion of trial results available across all potential COVID- interventions – catagorised into those registered fully on http://CT.gov, those with results available elsewhere and those with no results available.
Of the 3754 completed studies, 2379 (63.4%) were without results on ClinicalTrials.gov outside of the 395-day timeframe mandated by the FDAAA 2007 (although not all trials may have been covered by the FDAAA 2007). 1008 (26.9%) of these had published results in the academic literature (13).
In the blinded second review of 341 (10%) trials, the same results status was found 83.6% of the time meeting our 80–90% threshold for acceptable concordance in searches.
Discussion
40.4% of the completed clinical trials for drugs that may be repurposed for COVID-19 were not found to report results on either ClinicalTrials.gov or through academic publication (Table 2). This shows a large gap in the evidence base for these drugs, limiting attempts to comprehensively review their safety before potential global distribution for the COVID-19 pandemic.
The 2238 (59.6%) completed studies with available results were comprised of 1172 (31.2%) with results on the registry and 1066 (28.4%) without results on the registry, but with results from a standardised search of the literature (Table 2). Furthermore, 2379 (63.4%) studies without registry results were outside of the 395 day timeframe for results publication as mandated by the FDAAA 2007 (13), although 1008 of these had results published in academic literature (26.9%). Not all trials include in this study are applicable under the FDAAA 2007, but this remains a benchmark for good scientific practice (26). With 40.4% of clinical trial results unavailable for potential COVID-19 treatments, the data for clinical decision making regarding the safety of these therapeutics are limited. If any drug with an incomplete evidence base is used during the pandemic, even in compassionate use programmes, there is a risk of avoidable harm being done because of missing adverse safety data. A knowledge gap was revealed for drugs which have had extensive media coverage such as hydroxychloroquine (37.0% without results), favipiravir (77.8%) and lopinavir (40.5%) (28–30). These drugs are currently being used in COVID-19 patients or trials across the globe, sometimes in novel regimens and doses (7,31,32). Clinicians have few treatment options available, but with greater transparency and proactiveness from clinical trial sponsors regarding the posting of trial results, there would be less risk of unforeseen adverse outcomes, especially in the treatment of mild-moderate COVID-19 as in the PIONEER trial (31).
Public health decision-makers, guideline developers, clinicians and patients rely on clinical trials, systematic reviews and meta-analyses to inform treatment. Evidence gaps and publication bias therefore influence clinical practice and drug usage worldwide, particularly in a treatment landscape as changeable as the COVID-19 pandemic. If clinical decisions are based on incomplete evidence, this can result in avoidable morbidity and mortality if unsafe drugs or ineffective treatments are given on a large scale. Sponsors and researchers alike also carry an ethical responsibility towards clinical trial participants, who consent to participate in research in order to contribute to scientific understanding and improved clinical practice, to make results publicly available (33). Clinical trial results that are not made available publicly do not fulfil this expectation, betraying the trust of participants who may have given up time and health for the benefit of science. Missing evidence also impacts the direction of future research, which is informed by the existing available body of literature.
Our study reveals an important knowledge gap regarding pharmaceuticals potentially being repurposed for COVID-19. However, the proportion of studies with results available in the academic literature is an approximation and there are several limitations to our study. Our trial population was limited only to those registered on ClinicalTrials.gov. While ClinicalTrials.gov is the largest registry in the world, with over 340,000 registrations as of writing, additional trials on these therapies may have been registered elsewhere. However, it is unclear if these would be expected to report at a different rate than those registered to ClinicalTrials.gov. Our strategy for locating publications included only those listed on ClinicalTrials.gov and identified through searches on PubMed and Google Scholar, open resources that should cover a majority of published clinical research. While including proprietary databases like Scopus or Ovid may have located some additional publications, we do not believe this would have substantially impacted our overall results (34). Additionally, trials that were not registered at all or published in non-English language journals without inclusion of the NCT number would not have been captured by our methodology. Searcher heterogeneity and difficulty identifying results publication in the academic literature limits accuracy in any manual publication search, however, our search strategy was standardised and produced a high level of agreement between assessors in a check of a 10% random sample (83.6%). However, the discordance present between reviewers only illuminates the inherent difficulties in finding results for the drugs in question, especially if the trial ID number was not included in line with CONSORT standards (35).
Our findings add to the existing evidence of the dearth of clinical trial reporting on public registries. This analysis investigated clinical trials of drugs currently being considered for use for COVID-19. However, given the diversity of drug classes included in this report, findings are likely to be representative of many pharmaceuticals. This presents a major problem for researchers attempting to summarise safety and efficacy by pooling trial data (36). Since academic publications often summarise key findings only, secondary research efforts are impinged. ClinicalTrials.gov provides a forum to share complete safety and efficacy data reports, as well as facilitating consistent data reporting in a timely manner (16,17). Prior research has shown that results reported to http://ClincialTrials.gov were often more complete, especially for safety data, when compared to matched journal publications (18–20). However, this depends on researchers registering trials and uploading results in a timely manner, within twelve months of the primary completion date.
The International Committee of Medical Journal Editors (ICMJE) and the editorial offices of medical journals could play an important role in improving the lack of timely results posting on clinical trial registries by demanding submission of a link to summary results on public registries before academic publication, although this may mean that the publication bias of positive, ‘publishable’ results may trickle down to reporting on public registries as well. Furthermore, public funders and institutional publication funds could demand that trial sponsors post their results before allocating funding for academic publication. These funding bodies could also deny individual sponsors funding if they have in the past violated clinical trial reporting rules (37). At the very least, journals should conform to the CONSORT statement in ensuring that registry IDs are clearly indicated in the abstract, full-text, and meta-data of published clinical trials in order promote easier discoverability and record linkage between registries and publications (35). Finally, clinical trial sponsors, such as universities, hospitals, public research institutions, and pharmaceutical companies, should themselves work towards improving their institutional clinical trial reporting performance by making use of available resources that provide detailed step-bystep instructions as to how to go about this task (14). Especially during the COVID-19 pandemic, it is of great importance that trials sponsors release summary results on these registries retrospectively to inform decision making around existing treatments being re-purposed for COVID-19.
Overall, our findings reveal a significant evidence gap for the safety of drugs being repurposed for COVID-19. This uncertainty could cause a large burden of extra morbidity in the global pandemic. We recommend caution in experimental drug use for non-severe disease and urge trial sponsors to report missing results retrospectively. Medicine during the COVID-19 pandemic cannot be evidence-based if a large proportion of the evidence is missing.
Data Availability
Data is publicly available from the U.S. National Library of Medicine (https://clinicaltrials.gov/ct2/home)
Author Contributions
FR and TP conceptualised the study, devised the methodology, and coordinated the research team. SK recruited the research team. All authors were involved in data collection and analysis, and contributed to the final manuscript.
Data Sharing Statement
Data is publicly available from the U.S. National Library of Medicine (https://clinicaltrials.gov/ct2/home).
Licence Statement
I, the Submitting Author has the right to grant and does grant on behalf of all authors of the Work (as defined in the below author licence), an exclusive licence and/or a non-exclusive licence for contributions from authors who are: i) UK Crown employees; ii) where BMJ has agreed a CC-BY licence shall apply, and/or iii) in accordance with the terms applicable for US Federal Government officers or employees acting as part of their official duties; on a worldwide, perpetual, irrevocable, royalty-free basis to BMJ Publishing Group Ltd (“BMJ”) its licensees and where the relevant Journal is co-owned by BMJ to the coowners of the Journal, to publish the Work in BMJ Open and any other BMJ products and to exploit all rights, as set out in our licence.
The Submitting Author accepts and understands that any supply made under these terms is made by BMJ to the Submitting Author unless you are acting as an employee on behalf of your employer or a postgraduate student of an affiliated institution which is paying any applicable article publishing charge (“APC”) for Open Access articles. Where the Submitting Author wishes to make the Work available on an Open Access basis (and intends to pay the relevant APC), the terms of reuse of such Open Access shall be governed by a Creative Commons licence – details of these licences and which Creative Commons licence will apply to this Work are set out in our licence referred to above.
Acknowledgements
We are very grateful for the help of 18 research assistants that helped with compiling the data for this study: Helen Woodward, Joshua Card-Gowers, Joshua Lucas, Sultan Hussein, Mina Aries, Spatikha Sitaram, Holly Melvin, Lauren Hargreaves, Stefano Santorini, Frances Kenworthy, Khalifa Saif Elyazal Ali, Maymunah Malik, Shiron Rajendran, Oliver Wright, Catherine Dominic, Holly Beckett, Tricia Tay and Daphne Lenz.
We are also thankful for the comments of Till Bruckner, Peter Grabitz, and Nicholas DeVito on the draft protocol.
APPENDIX
Footnotes
Funding source: This research received no specific grant from any funding agency in the public, commercial or not-forprofit sectors.
Conflicts of interest: Several of the co-authors on this paper are part of Universities Allied for Essential Medicines U.K. However, views expressed in this paper are not necessarily that of Universities Allied for Essential Medicines Europe.
References
