Abstract
Optimising statistical power in early-stage trials and observational studies accelerates discovery and improves the reliability of results. Ideally, intermediate outcomes should be continuously distributed and lie on the causal pathway between an intervention and a definitive outcome such as mortality. In order to optimise power for an intermediate outcome in the RECOVERY trial, we devised and evaluated a modification to a simple, pragmatic measure of oxygenation function - the SaO2/FIO2 (S/F) ratio.
We demonstrate that, because of the ceiling effect in oxyhaemoglobin saturation, S/F ceases to reflect pulmonary oxygenation function at high values of SaO2. Using synthetic and real data, we found that the correlation of S/F with a gold standard (PaO2/FIO2, P/F ratio) improved substantially when measurements with SaO2 ≥ 0.94 are excluded (Spearman r, synthetic data: S/F : 0.31; S/F94: 0.85). We refer to this measure as S/F94.
In order to test the underlying assumptions and validity of S/F94 as a predictor of a definitive outcome (mortality), we collected an observational dataset including over 39,000 hospitalised patients with COVID-19 in the ISARIC4C study. We first demonstrated that S/F94 is predictive of mortality in COVID-19. We then compared the sample sizes required for trials using different outcome measures (S/F94, the WHO ordinal scale, sustained improvement at day 28 and mortality at day 28) ensuring comparable effect sizes. The smallest sample size was needed when S/F94 on day 5 was used as an outcome measure.
To facilitate future study design, we provide an online user interface to quantify real-world power for a range of outcomes and inclusion criteria, using a synthetic dataset retaining the population-level clinical associations in real data accrued in ISARIC4C https://isaric4c.net/endpoints.
We demonstrated that S/F94 is superior to S/F as a measure of pulmonary oxygenation function and is an effective intermediate outcome measure in COVID-19. It is a simple and non-invasive measurement, representative of disease severity and provides greater statistical power to detect treatment differences than other intermediate endpoints.
Competing Interest Statement
JKB and ABD report grants from the UK Department of Health and Social Care (DHSC) during the conduct of the study and grants from Wellcome Trust. PJMO reports personal fees from consultancies (GlaxoSmithKline; Janssen; Bavarian Nordic; Pfizer; and Cepheid) and from the European Respiratory Society; grants from MRC; MRC Global Challenge Research Fund; the EU; NIHR BRC; MRC GlaxoSmithKline; Wellcome Trust; NIHR (HPRU in Respiratory Infection); and is an NIHR senior investigator outside the submitted work. PJMO his role as President of the British Society for Immunology was unpaid but travel and accommodation at some meetings was provided by the Society. JKB reports grants from MRC. MGS reports grants from DHSC; NIHR UK; MRC; HPRU in Emerging and Zoonotic Infections and University of Liverpool during the conduct of the study and is chair of the scientific advisory board and a minority shareholder at Integrum Scientific, outside the submitted work. JSNVT was seconded to the Department of Health and Social Care England (DHSC) October 2017 to March 2022. The views expressed in this manuscript are those of its authors and not necessarily those of DHSC. JSNV-T reports personal fees and travel and accommodation from Astra Zeneca. NS reports grants from Boehringer Ingleheim and Novo Nordisk outside the submitted work.
Funding Statement
This work was supported by the National Institute for Health Research (NIHR) [CO-CIN-01], the Medical Research Council [MC_PC_19059] and by the NIHR Health Protection Research Unit (HPRU) in Emerging and Zoonotic Infections at University of Liverpool in partnership with Public Health England (PHE), in collaboration with Liverpool School of Tropical Medicine and the University of Oxford [200907], NIHR HPRU in Respiratory Infections at Imperial College London with PHE [200927], Wellcome Trust and Department for International Development [215091/Z/18/Z], and the Bill and Melinda Gates Foundation [OPP1209135], and Liverpool Experimental Cancer Medicine Centre (C18616/A25153), NIHR Biomedical Research Centre at Imperial College London [IS-BRC-1215-20013], EU Platform for European Preparedness Against (Re-) emerging Epidemics (PREPARE) [FP7 project 602525] and NIHR Clinical Research Network for providing infrastructure support for this research. JKB gratefully acknowledges funding support from a Wellcome Trust Senior Research Fellowship (223164/Z/21/Z), BBSRC Institute Strategic Programme Grant to the Roslin Institute (BB/P013732/1, BB/P013759/1), UKRI (MC_PC_20004, MC_PC_19025, MC_PC_1905, MRNO2995X/1), and the UK Intensive Care Society. ADB would like to acknowledge funding from the Wellcome PhD training fellowship for clinicians (204979/Z/16/Z), the Edinburgh Clinical Academic Track (ECAT) programme
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
Ethical approval was given by the South Central-Oxford C Research Ethics Committee in England (13/SC/0149), the Scotland A Research Ethics Committee (20/SS/0028), and the WHO Ethics Review Committee (RPC571 and RPC572, April 2013).
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.
Yes
Data Availability
The dataset used and analysed in this study is not available, but a synthetically generated dataset, containing the same key properties as the original dataset is available for sample size calculations on https://isaric4c.net/endpoints
