Abstract
Objectives To assess how often clinical trials exhibit primary outcome discrepancies within registry records that would not be caught by comparing results publications to the latest registry entry, but would require analysing the registration history.
Design Meta-research study.
Setting All 1746 randomised clinical trials with published results, registered in ClinicalTrials.gov or Deutsches Register Klinischer Studien (DRKS), completed at German University Medical Centres between 2009 and 2017. We analysed registry entries for all trials and publications for a random sample of 292 trials.
Participants Not applicable.
Interventions Not applicable.
Main outcome measures [1] Primary outcome discrepancies between registry entries at key study milestones and [2] the first results publication. [3] ‘Hidden’ discrepancies, i.e., only reported in the registry before the last entry, meaning they would only be detected by assessing the full registry change history. We considered discrepancies major if primary outcomes were newly added, dropped, or changed to or from secondary outcomes. [4] Proportion of publications transparently reporting discrepancies. [5] Characteristics associated with ‘open’ and ‘hidden’ discrepancies.
Results Of all 1746 trials, 23% (n=393) had primary outcome discrepancies between trial start and latest registry entry, with 8% (n=142) being major. Primary outcomes in publications were different from the latest registry entry in 41% of trials (120 of the 292 sampled trials; 95% CI [35%, 47%]), with major discrepancies in 18% (54 of 292; 95% CI [14%, 23%]). ‘Hidden’ discrepancies were observed in 14% of trials (41/292; 95% CI [10%, 19%]). Only 1% of discrepancies were reported in the publications (2/161, 95% CI [0%, 4%]). Trials were more likely to have a within-registry discrepancy if they had an earlier registration year (OR 0.74; 95% CI [0.69, 0.80]), were registered on ClinicalTrials.gov (OR 0.41; 95% CI [0.23, 0.70]), or had been industry-sponsored (OR 0.29; 95% CI [0.21, 0.41]).
Conclusions Changes to primary trial outcomes are common, often have major relevance, are rarely transparently reported and typically not detectable with an inspection of the latest registry entry. Authors need to be more transparent and registry entries of published trials need to require more in-depth analysis to reveal potentially misleading reporting practices.
Protocol registration Open Science Framework (https://osf.io/t3qva; amendment in https://osf.io/qtd2b).
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
This work was funded under a grant from the Federal Ministry of Education and Research of Germany (Bundesministerium fuer Bildung und Forschung, BMBF) [01PW18012]. The funder was not involved in the study design, data collection, analysis, or interpretation, writing of the manuscript, or the decision to submit for publication.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.
Yes
Footnotes
martin.haslberger{at}bih-charite.de
samruddhi.yerunkar{at}charite.de
daniel.strech{at}bih-charite.de
lars.hemkens{at}usb.ch
benjamin.carlisle{at}bih-charite.de
Previous versions: Preliminary data of this study were presented at the 9th International Conference on Peer Review and Scientific Publication, Chicago, 9 September 2022.
Code and data availability: Analysis code and original datasets are available on Github (https://github.com/Martin-R-H/HiddenOutcomeChanges). The final dataset has been posted to our OSF page (https://osf.io/e2uct/).
Ethics approval: Not applicable.
Transparency statement: The lead authors affirm that the manuscript is an honest, accurate, and transparent account of the study being reported. No important aspects of the study have been omitted. Any discrepancies from the study as registered (https://osf.io/t3qva) are explained in a registration amendment (https://osf.io/qtd2b). Most importantly, due to the complexity of the project and the large amount of manual screening involved, we decided to split the preregistered project in two standalone projects (outcome discrepancies and spin), which we report in separate publications. The authors declare no conflicts of interest.
Role of the funding source: This work was funded under a grant from the Federal Ministry of Education and Research of Germany (Bundesministerium fuer Bildung und Forschung – BMBF) [01PW18012]. The funder was not involved in the study design, data collection, analysis, or interpretation, writing of the manuscript, or the decision to submit for publication.
Patient and Public Involvement statement: No patients were involved in this study. There was no public involvement for this study.
Dissemination to participants and related patient and public communities: Not applicable.
Data Availability
Analysis code and original datasets are available on Github (https://github.com/Martin-R-H/HiddenOutcomeChanges). The final dataset has been posted to our OSF page (https://osf.io/e2uct/).
