Abstract
Background Measurement of post-exertion oxygen saturation has been proposed to assess illness severity in suspected COVID-19 infection. We aimed to determine the accuracy of post-exertional oxygen saturation for predicting adverse outcome in suspected COVID-19.
Methods We undertook an observational cohort study across 70 emergency departments during first wave of the COVID-19 pandemic in the United Kingdom. We collected data prospectively, using a standardised assessment form, and retrospectively, using hospital records, from patients with suspected COVID-19, and reviewed hospital records at 30 days for adverse outcome (death or receiving organ support). Patients with post-exertion oxygen saturation recorded were selected for this analysis.
Results We analysed data from 817 patients with post-exertion oxygen saturation recorded after excluding 54 in whom measurement appeared unfeasible. The c-statistic for post-exertion change in oxygen saturation was 0.589 (95% confidence interval 0.465 to 0.713), and the positive and negative likelihood ratios of a 3% or more desaturation were respectively 1.78 (1.25 to 2.53) and 0.67 (0.46 to 0.98). Multivariable analysis showed that post-exertion oxygen saturation was not a significant predictor of adverse outcome when baseline clinical assessment was taken into account (p=0.368). Secondary analysis excluding patients in whom post-exertion measurement appeared inappropriate resulted in a c-statistic of 0.699 (0.581 to 0.817), likelihood ratios of 1.98 (1.26 to 3.10) and 0.61 (0.35 to 1.07), and some evidence of additional prognostic value on multivariable analysis (p=0.019).
Conclusions Post-exertion oxygen saturation provides modest prognostic information in the assessment of patients attending the emergency department with suspected COVID-19.
Registration ISRCTN registry, ISRCTN56149622, http://www.isrctn.com/ISRCTN28342533
Key messages What is already known on this subject?
Post exertional decrease in oxygen saturation can be used to predict prognosis in chronic lung diseases
Post exertional desaturation has been proposed as a way of predicting adverse outcome in people with suspected COVID-19
What this study adds:
Post-exertion oxygen saturation provides modest prognostic information in the assessment of patients attending the emergency department with suspected COVID-19
Introduction
Guidelines for assessment of suspected COVID-19 recommend measurement of peripheral oxygen saturation to determine the severity of acute respiratory infection (1-3). Clinicians have noted that patients with suspected COVID-19 and a relatively normal oxygen saturation may desaturate after exertion, but the clinical importance of this finding is uncertain.
Field walking tests are commonly used to evaluate exercise capacity and assess prognosis in chronic respiratory diseases (4). The lowest arterial oxygen saturation recorded during a 6-minute walk test is an important marker of disease severity and prognosis (5). The rapid 1-minute sit-to-stand test correlates with the 6-minute walk test and the severity of lung disease (6). Exertional tests have shown desaturation in chronic obstructive lung disease (7,8) chronic interstitial lung disease (9-11) and pneumocystis carinii pneumonia (12). A modified 6-minute walk test has been proposed for use in suspected COVID-19 infection (13) but not yet evaluated, to our knowledge. A recent review of rapid exercise tests for oxygen desaturation (14) identified a number of studies, as outlined above, but found no published studies in COVID-19. The authors suggested that a 3% drop in oxygen saturation on exercise was a cause for concern, based on evidence from other conditions.
The Pandemic Respiratory Infection Emergency System Triage (PRIEST) study is a multicentre observational cohort study designed to develop and evaluate triage methods for patients with suspected COVID-19 infection. We added evaluation of post-exertion oxygen saturation to the aims of the PRIEST study in response to reports of its use in the assessment of suspected COVID-19. Our specific objective was to determine the accuracy of post-exertional oxygen saturation as a prognostic factor for 30-day adverse outcome.
Methods
We collected data from consecutive patients presenting with suspected COVID-19 infection to 70 hospital emergency departments from 53 recruiting sites in the United Kingdom (UK). Hospitals used either prospective data collection, through a standardised assessment form for suspected COVID-19, or retrospective data collection, through research staff extracting data from hospital records onto the standardised form.
Patients were included if the assessing clinician used the standardised assessment form or recorded that the patient had suspected COVID-19 infection. The clinical diagnostic criteria used for suspected COVID-19 during the study were (1) fever (pyrexia ≥38°C) or a history of fever and (2) influenza-like illness (two or more of cough, sore throat, rhinorrhoea, limb or joint pain, headache, vomiting or diarrhoea) or severe and/or life-threatening illness suggestive of an infectious process. We did not seek consent to collect data but information about the study was provided in the ED and patients could withdraw their data at their request. Patients with multiple presentations to hospital were only included once, using data from the first presentation identified by research staff.
The population for this analysis was patients who had post-exertion oxygen saturation recorded as part of routine care. The assessing clinician made the decision to measure post-exertion oxygen saturation and determined the approach to achieving exertion. The study did not influence clinical care, so we were unable to standardise the selection of patients or the approach to measuring post-exertion oxygen saturation. Measurement could have been undertaken deliberately, by asking the patient to exercise in a specified way, or opportunistically, by recording oxygen saturation after the patient had exerted themselves for another purpose.
Research staff reviewed hospital records to identify outcomes up to 30 days after initial presentation. We defined patients who died or required respiratory, cardiovascular, or renal support as having an adverse outcome. We defined respiratory support as any intervention to protect the patient’s airway or assist their ventilation, including mechanical ventilation, non-invasive ventilation, or continuous positive airway pressure, but not supplemental oxygen alone or nebulised bronchodilators. We defined cardiovascular support as any intervention to maintain organ perfusion, including extra-corporeal membrane oxygenation, inotropic drugs, or invasive cardiovascular monitoring, but not peripheral intravenous cannulation and/or fluid administration. We defined renal support as any intervention to assist renal function, including haemofiltration, haemodialysis, or peritoneal dialysis, but not intravenous fluid administration or urinary catheterisation.
We undertook an initial descriptive analysis of the patients with post-exertion oxygen saturation recorded. This identified a number of patients for whom post-exertion oxygen saturation measurement appeared unfeasible, based on age (less than three years), performance status bed/chair bound, baseline oxygen saturation below 85%, post exertion oxygen saturation below 50%, receiving supplemental oxygen or Glasgow Coma Score less than 14. We excluded these patients from the analysis.
We examined baseline oxygen saturation, post-exertion oxygen saturation and post-exertion change in oxygen saturation (i.e. baseline minus post-exertion oxygen saturation). Analysis focused on the latter, because this indicates the additional value achieved by measuring oxygen saturation after exertion. We estimated the accuracy of each index test in terms of the sensitivity, specificity and likelihood ratios of each test across a range of thresholds for positivity, for predicting adverse outcome up to 30 days. Confidence intervals for likelihood ratios were calculated using the methods outlined in Koopman et al. (15) Receiving Operator Characteristic (ROC) Curves were constructed and the c-statistic (area under the ROC curve) was calculated for each index test. We did not attempt to determine an optimal threshold for positivity, because that depends upon the relative importance of sensitivity and specificity in the decision that post-exertion oxygen saturation is intended to inform. However, we decided a priori to highlight the performance of a 3% desaturation, as suggested by Greenhalgh et al. (14) Analysis was performed on patients with post-exertion oxygen saturation recorded and available 30 day outcome data, as such missing data was not imputed.
To determine whether measurement of post-exertion oxygen saturation adds prognostic information to standard respiratory assessment, we fitted a multivariable model with age, baseline oxygen saturation, respiratory rate, heart rate, asthma, other chronic respiratory illness and post-exertional oxygen saturation as covariates.
We undertook a secondary analysis that excluded patients for whom post-exertion oxygen saturation measurement appeared less appropriate, based on age (less than 16 years), performance status of limited self-care, baseline oxygen saturation less than 94%, or heart rate, respiratory rate or systolic blood pressure scoring three points on the National Early Warning Score (NEWS2). The rationale for this analysis was that local guidelines (3) recommend admission for patients with oxygen saturation less than 94% or a score of three points or more on any NEWS2 parameter. It has also been suggested that post-exertional assessment is only undertaken in a patient able to stand safely unaided and whose resting saturation is 96% or above. (14)
We planned for the PRIEST study to recruit a sample size of 20,000. The analysis presented here is a secondary analysis, so no sample size was pre-specified.
Patient and public involvement
The Sheffield Emergency Care Forum (SECF) is a public representative group interested in emergency care research. [15] Members of SECF advised on the development of the PRIEST study and two members joined the Study Steering Committee. Patients were not involved in the recruitment to and conduct of the study. We are unable to disseminate the findings to study participants directly.
Results
The PRIEST study recruited 22485 patients across 70 hospitals between 26 March 2020 and 28 May 2020, of whom 39 requested withdrawal of their data. We identified 874 patients who had post-exertion oxygen saturation recorded and excluded 57 in whom measurement appeared unfeasible, leaving 817 for analysis. Adverse outcome occurred in 30 participants (3.7%), of these nine died, 22 had respiratory support, five had cardiovascular support and four renal support.
Supplemental Figure S1 shows the flow of patients through the study, and Supplemental Table S1 shows the characteristics of the whole PRIEST cohort and the characteristics of those included in this analysis. Participants in this analysis were younger, more likely to have unrestricted performance status, less likely to have any comorbidities, tended to have more normal baseline physiology and had a much lower rate of adverse outcome.
Table 1 compares the baseline oxygen saturation, post-exertion oxygen saturation and post-exertion change between those with and without an adverse outcome. Post-exertion oxygen saturation tended to be lower than baseline oxygen saturation and show a greater decrease in those who suffered adverse outcome (2.9% versus 1.9% mean decrease). However, Figure 1 show that oxygen saturations increased post-exertion in a proportion of cases and there was considerable overlap between those with and without adverse outcome. Supplemental Figures S2 and S3 show overlayed histograms for baseline and post-exertion oxygen saturation.
Figure 2 shows the ROC curves for baseline oxygen saturation, post-exertion oxygen saturation and post-exertion change in oxygen saturation. The c-statistic of 0.589 for post-exertion change indicates poor discriminant value, partly due to post-exertion increases in oxygen saturation showing some association with adverse outcome. Table 2 reports sensitivity, specificity and likelihood ratios for thresholds of post-exertion decrease in oxygen saturation (i.e. change less than zero). The positive and negative likelihood ratios of a post-exertional desaturation of 3% or more were 1.78 and 0.67 respectively, suggesting that this finding provides a small amount of additional information in prognostic assessment. Supplemental tables S2 and S3 show the diagnostic parameters for baseline and post-exertion oxygen saturation respectively.
Figure 3 and Table 3 show comparable results for the secondary analysis excluding cases where post-exertion oxygen saturation measurement appeared less appropriate. The c-statistic of 0.699 for post-exertion change in oxygen saturation indicates better discriminant value in this group. This may be explained by exclusion of patients with lower baseline oxygen saturations who had more potential to show a post-exertion change. The positive and negative likelihood ratios of a post-exertional decrease in oxygen saturation of 3% or more were 1.98 and 0.61 respectively. Supplemental tables S4 and S5 show the diagnostic parameters for baseline and post-exertion oxygen saturation respectively for the secondary analysis.
In the multivariable model on the primary analysis cohort, post-exertional oxygen saturation did not add prognostic value over other factors in the model (p-value for model coefficient 0.368, likelihood ratio test for model with and without post-exertion oxygen saturation 0.78, p=0.376). For the secondary analysis, post-exertional oxygen saturation added prognostic value over other factors (p-value for model coefficient 0.019, likelihood ratio test for model with and without post-exertion oxygen saturation 4.82, p=0.078).
Discussion
Our findings suggest that measurement of post-exertion oxygen saturation adds little to clinical assessment of suspected COVID-19 in the emergency department. The likelihood ratios suggest that a desaturation of 3% or more provides a small amount of prognostic information, the c-statistic of 0.589 for post-exertion change suggests little discriminant value, and multivariate analysis suggest that post-exertion oxygen saturation measurement does not add prognostic value once baseline measurements are taken into account. Secondary analysis suggested better discriminant value and some additional prognostic value when testing was limited to more appropriate cases, but likelihood ratios still suggested that a desaturation of 3% or more still provided only modest prognostic information.
The observation that oxygen saturation increased post-exertion, and that some people with adverse outcome showed an increase, may seem surprising, but is probably explained by random variation. Oxygen saturation varies randomly from one measurement to the next and this variation is likely to be greater in sicker patients with baseline hypoxia. Thus we might expect greater variation in oxygen saturation to show some association with adverse outcome.
Our findings suggest that measurement of post-exertion oxygen saturation provides little additional prognostic information in selected patients with suspected COVID-19. Greenhalgh et al suggested using a desaturation of at least 3% to identify cause for concern in selected patients who are well enough for out of hospital management. Our findings suggest that a 3% desaturation indicates small increase in the likelihood of adverse outcome. Further research could determine whether a more systematic and rigorously controlled approach to post-exertion oxygen saturation measurement can result in more useful prognostic information. The feasibility of such research may be limited by low event rates in people who are able to undertake formal post-exertion measurement of oxygen saturation.
Our study consisted of a clinically relevant population and was recruited across a wide range of settings, but evaluation of post-exertion oxygen saturation was a post hoc secondary analysis and the study was not designed specifically for this purpose. We are unable to say how patients were selected for measurement of post-exertion oxygen saturation, and the method for undertaking exertion was not standardised or recorded. We excluded 57 patients from analysis for whom post-exertion oxygen saturation measurement appeared unfeasible, and excluded a further 162 from secondary analysis for whom measurement appeared less appropriate. These cases may reflect opportunistic oxygen saturation measurement after exertion, such as on attempting to mobilise, but we cannot exclude the possibility of data recording errors. Only 874 out of 22446 patients had post-exertion oxygen saturation recorded. This may reflect limited awareness and use of post-exertion oxygen saturation, but may also reflect severity of illness in the emergency department population. Measurement of post-exertion oxygen saturation is only likely to be feasible and clinically indicated in those with milder illness. The relatively small number of adverse outcomes (N=30) limited the precision of our estimates of sensitivity and power to undertake multivariable analysis.
In summary, measuring post-exertion oxygen saturation provides little prognostic information in the assessment of patients attending the emergency department with suspected COVID-19.
Data Availability
Anonymised data are available from the corresponding author upon reasonable request (contact details on first page). The Confidentiality Advisory Group of the Health Research Authority will need to consider any requests for data to be used for purposes other than those specified in our application, so a data request should be accompanied by explanation of the purpose of the request and justification of the public benefit. We also recommend including a pre-specified plan of analysis.
Ethical approval
The North West - Haydock Research Ethics Committee gave a favourable opinion on the PAINTED study on 25 June 2012 (reference 12/NW/0303) and on the updated PRIEST study on 23rd March 2020. The Confidentiality Advisory Group of the Health Research Authority granted approval to collect data without patient consent in line with Section 251 of the National Health Service Act 2006.
Funding
The PRIEST study was funded by the United Kingdom National Institute for Health Research Health Technology Assessment (HTA) programme (project reference 11/46/07). The funder played no role in the study design; in the collection, analysis, and interpretation of data; in the writing of the report; and in the decision to submit the article for publication. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care.
Conflicts of interest
All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare: grant funding to their employing institutions from the National Institute for Health Research; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.
Author contributions
SG, AB, KC, CF, TH, FL, ALe, IM and DW conceived and designed the study. BT, KB, ALo, SW, RS, JS, SC, ES, JH and EY acquired the data. EL, LS, SG, BT, KB and CM analysed the data. SG, AB, KC, CF, TH, FL, ALe, IM, DW, EL, LS, SG, BT, KB and CM interpreted the data. All authors contributed to drafting the manuscript. SG takes responsibility for the paper as a whole.
Data sharing
Anonymised data are available from the corresponding author upon reasonable request (contact details on first page). The Confidentiality Advisory Group of the Health Research Authority will need to consider any requests for data to be used for purposes other than those specified in our application, so a data request should be accompanied by explanation of the purpose of the request and justification of the public benefit. We also recommend including a pre-specified plan of analysis.
Acknowledgements
We thank Katie Ridsdale for clerical assistance with the study, Erica Wallis (Sponsor representative, all members of the PRIEST Research Team (Appendix 1), all members of the Study Steering Committee (Appendix 2) and the site research teams who delivered the data for the study (Appendix 3), and the research team at the University of Sheffield past and present (Appendix 4).