Prognostic accuracy of triage tools for adults with suspected COVID-19 in a pre-hospital setting: an observational cohort study =============================================================================================================================== * Carl Marincowitz * Laura Sutton * Tony Stone * Richard Pilbery * Richard Campbell * Benjamin Thomas * Janette Turner * Peter A. Bath * Fiona Bell * Katie Biggs * Madina Hasan * Frank Hopfgartner * Suvodeep Mazumdar * Jennifer Petrie * Steve Goodacre ## Abstract **Study Objective** Tools proposed to triage patient acuity in COVID-19 infection have only been validated in hospital populations. We estimated the accuracy of five risk-stratification tools recommended to predict severe illness and compare accuracy to existing clinical decision-making in a pre-hospital setting. **Methods** An observational cohort study using linked ambulance service data for patients attended by EMS crews in the Yorkshire and Humber region of England between 18th March 2020 and 29th June 2020 was conducted to assess performance of the PRIEST tool, NEWS2, the WHO algorithm, CRB-65 and PMEWS in patients with suspected COVID-19 infection. The primary outcome was death or need for organ support. **Results** Of 7549 patients in our cohort, 17.6% (95% CI:16.8% to 18.5%) experienced the primary outcome. The NEWS2, PMEWS, PRIEST tool and WHO algorithm identified patients at risk of adverse outcomes with a high sensitivity (>0.95) and specificity ranging from 0.3 (NEWS2) to 0.41 (PRIEST tool). The high sensitivity of NEWS2 and PMEWS was achieved by using lower thresholds than previously recommended. On index assessment, 65% of patients were transported to hospital and EMS decision to transfer patients achieved a sensitivity of 0.84 (95% CI 0.83 to 0.85) and specificity of 0.39 (95% CI 0.39 to 0.40). **Conclusion** Use of NEWS2, PMEWS, PRIEST tool and WHO algorithm could improve sensitivity of EMS triage of patients with suspected COVID-19 infection. Use of the PRIEST tool would improve sensitivity of triage without increasing the number of patients conveyed to hospital. ## Background Emergency Medical Service (EMS) and other urgent and emergency care practitioners assessing patients with suspected COVID-19 infection in the community, must rapidly determine whether patients need treatment in hospital or can safely remain at home. The overall risk of mortality in patients with confirmed infection is around 1% and if conveyance is too liberal, hospitals could be overwhelmed by patients who require no specific treatment.1 However, failing to identify a patient at risk of serious deterioration could lead to avoidable harm.2 Prognostic research has almost exclusively been conducted in hospital settings and current national and international guidelines for risk-stratification of patients with suspected COVID-19 in the community are consensus based.1, 3-5 Clinical acuity scores, such as the UK Royal College of Physicians Early Warning Score, version 2 (NEWS2), have been suggested in some guidelines as a way to risk-stratify patients with suspected COVID-19 infection in the community.6 The WHO decision-making algorithm for respiratory infection and CRB-65 are used to risk-stratify patients with bacterial pneumonia and PMEWS for use in patients with influenza.7-9 However, the accuracy of these risk-stratification tools has only been validated in hospitalised or non-COVID populations. NEWS2 has shown good prediction of adverse outcome in patients attending the Emergency Department (ED) with suspected COVID-19.7 The PRIEST tool was derived by adding age, sex, and performance status to NEWS2, and validation showed improved prediction compared to NEWS2 alone.10, 11 Validation of the PRIEST tool, NEWS2 and other clinical risk-stratification tools recommended for use in hospital in a community setting,7, 10, 12-14 could identify the most accurate means to triage need for hospitalisation, thereby reducing unnecessary hospital attendances and improve the identification of those most at risk of serious adverse outcomes. Our study aimed to: 1. Estimate the accuracy of risk-stratification tools recommended to predict severe illness in adults with suspected COVID-19 infection in a pre-hospital setting. 2. Compare the accuracy of risk-stratification tools to existing clinical decision-making around transport to hospital. ## Methods ### Study Design This observational cohort study used linked routinely collected EMS data to assess the accuracy in a community setting of five clinical risk-stratification tools (PRIEST tool, NEWS2, WHO algorithm, CRB-65 and PMEWS) recommended for use in hospitalised patients with COVID-19 or similar respiratory infections (triage tools shown in Supplementary Material 1).7, 10, 12-14 ### Setting Patients with suspected COVID-19 infection attended by emergency medical services provided by Yorkshire Ambulance Service NHS Trust (YAS). Emergency services provided by YAS covers a region in the north of England of approximately 6,000 square miles and with a population of 5.3 million. ### Data Sources and linkage EMS providers complete an electronic patient report form (ePRF) each time they attend an emergency call, which records presenting patient characteristics and clinical care in a standardised manner. YAS provided a dataset of ePRF data for all EMS responses between the 18th March 2020 and 29th June 2020 where the attending ambulance staff recorded a clinical impression of suspected or confirmed COVID-19 infection. The dataset consisted of patient identifiers, demographic data, measured physiological parameters, other available clinical information and the outcome of the assessment (including whether the patient was conveyed to hospital). Health and social care data relating to the population in England within the UK National Health Service (NHS) is managed by NHS Digital. We provided patient identifiers to NHS Digital to trace patients in our cohort and supply additional individual level demographic, co-morbidity and outcome data. NHS Digital identified records in their collections belonging to patients in our cohort, and provided data on patient demographics, limited COVID-related general practice (GP) records, emergency department attendances, hospital inpatient admissions, critical care periods, and death registrations from the UK Office of National Statistics. YAS and NHS Digital removed records where patients indicated that they did not wish their data to be used for research purposes, via the NHS data opt-out service.15 The study team also excluded patients who had opted out of any part of the PRIEST study and those with inconsistent records (e.g. multiple deaths recorded or death before latest activity). Patient identifiers across all datasets were replaced with a consistent pseudo-identifier to enable the identification and linkage of records belonging to the same patient across all datasets but without revealing any patient’s identity. ### Inclusion Criteria Our final cohort consisted of all adult (aged 16 years and over) patients at time of first (index) EMS attendance between 18th March and 29th June 2020, in which the attending ambulance staff recorded a clinical impression of suspected or confirmed COVID-19 infection, and who were successfully traced by NHS Digital. ### Outcome The primary outcome was death, renal, respiratory, or cardiovascular organ support (identified from death registration and critical care data) at 30 days from index attendance. The secondary outcome was death up to 30 days from index contact. ### Patient Characteristics Physiological parameters were extracted from the first (primary) set of clinical observations recorded by the ambulance crew. Consistent with methods used to estimate the Charlson comorbidity index from the available routine data, comorbidities were included if recorded 12 months before the index EMS attendance.16, 17 In a similar way, only immunosuppressant drug prescriptions documented in GP records within 30 days before the index attendance, contributed to the immunosuppression co-morbidity variable. Pregnancy status was based on GP records recorded in the previous 9 months. Frailty in patients older than 65 years was derived from the latest recorded Clinical Frailty Scale (CFS) score (if recorded) in the electronic GP records prior to index attendance.18 Patients under the age of 65 years were not given a CFS score since it is not validated in this age group. However, if a CFS score was required to calculate a triage tool and the patient was under the age of 65, it was assumed to be 1. Performance status was estimated from the CFS. ### Analysis We retrospectively applied the 5 triage tools to our cohort to assess their accuracy for the primary and secondary outcomes.7, 10, 12-14 Supplementary Material 1 provides details of scoring and handling missing data for the triage tools. For each tool we plotted the receiver operating characteristic (ROC) curve and calculated the area under the ROC curve (c-statistic) for discriminating between patients with and without adverse outcome. We calculated sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) at the following pre-specified decision making thresholds based on recommended or usual use: 0 vs 1+ CRB-65; 0–1 vs 2+ NEWS2; 0–2 vs 3+ PMEWS; 0–4 vs 5+ PRIEST; 0 vs 1 WHO score. The NEWS2 and PMEWS thresholds used are lower than previously proposed (0–3 vs 4+ NEWS and 0–3 vs 4+ PMEWS) for triaging patient acuity, and are based on the assessment of their performance in a UK ED population of patients with suspected COVID-19 infection, where higher thresholds gave sub-optimal sensitivity.19 The WHO algorithm and CRB-65 are positive if any criterion is positive. These tools were compared to the sensitivity, specificity, PPV and NPV of EMS clinicians’ decision to transfer patients to hospital. All analyses were based on assessment during the index EMS attendance and completed with SAS v9.4. ## Ethical Approval The North West—Haydock Research Ethics Committee gave a favourable opinion on the PAINTED study on 25th June 2012 (reference 12/NW/0303) and on the updated PRIEST study on 23rd March 2020, including the analysis presented here. The Confidentiality Advisory Group of the NHS Health Research Authority granted approval to collect data without patient consent in line with Section 251 of the National Health Service Act 2006. Access to data collected by NHS Digital was recommended for approval by its Independent Group Advising on the Release of Data (IGARD) on 11th September 2021 having received additional recommendation for approval for access to GP records from the Profession Advisory Group (PAG) on 19th August 2021. ## Patient Public Involvement The Sheffield Emergency Care Forum (SECF) is a public representative group interested in emergency care research.20 Members of SECF advised on the development of the PRIEST study and two members joined the Study Steering Committee. Patients were not involved in the conduct of the study. ## Results All totals presented from NHS Digital derived data sets (sex, number of current medications, comorbidities, clinical frailty scores and outcomes) are rounded to the nearest 5, with small numbers suppressed to comply with NHS Digital data disclosure guidance. ### Study population Figure 1 and Table 1 summarise study cohort derivation and the characteristics of the 7,549 included individual adult patients. In total, 1,330 patients (17.6%, 95% CI:16.8% to 18.5%) experienced the primary outcome (death or organ support) and 1,065 (14.1%, 95% CI: 13.4% to 14.9%), the secondary outcome (death). Of the 7, 549 patients, the decision was made to transport 4,905 (65%) to hospital at index attendance. Of those, 1,120 (22.9%) experienced the primary adverse outcome. Of those not transported to hospital, 210 (7.9%) had an adverse outcome. Within the cohort, 3,925 patients (52%, 95% CI:50.9% to 53.1%) were admitted as inpatients and 2,785 (36.9%, 95% CI: 35.8% to 38%) had a diagnosis of COVID confirmed in hospital (since unrestricted community testing was not available until the 18/05/2020) within 30 days of index EMS attendance. ![Figure 1.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/07/29/2021.07.27.21261031/F1.medium.gif) [Figure 1.](http://medrxiv.org/content/early/2021/07/29/2021.07.27.21261031/F1) Figure 1. STROBE flow diagram of study population selection View this table: [Table 1.](http://medrxiv.org/content/early/2021/07/29/2021.07.27.21261031/T1) Table 1. Patient characteristics by outcome ### Triage tool performance Sensitivity, specificity, positive and negative predictive values for predicting the primary composite outcome using pre-defined score thresholds are provided in Table 2 and the secondary outcome of death in Table 3. Sensitivity and specificity statistics are provided for every score threshold in Supplementary Material 2. The ROC curves for these analyses are shown in Figures 2–3. View this table: [Table 2.](http://medrxiv.org/content/early/2021/07/29/2021.07.27.21261031/T2) Table 2. Triage tool diagnostic accuracy statistics (95% CI) for predicting any adverse outcome View this table: [Table 3.](http://medrxiv.org/content/early/2021/07/29/2021.07.27.21261031/T3) Table 3. Triage tool diagnostic accuracy statistics (95% CI) for predicting death within 30 days ![Figure 2.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/07/29/2021.07.27.21261031/F2.medium.gif) [Figure 2.](http://medrxiv.org/content/early/2021/07/29/2021.07.27.21261031/F2) Figure 2. ROC curves showing triage tool performance for predicting any adverse outcome ![Figure 3.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/07/29/2021.07.27.21261031/F3.medium.gif) [Figure 3.](http://medrxiv.org/content/early/2021/07/29/2021.07.27.21261031/F3) Figure 3. ROC curves showing triage tool performance for predicting death within 30 days The PRIEST tool was robust to the removal of the performance status parameter; when doing so the C-statistic remained at 0.83 (95% CI 0.82 to 0.84). EMS decision to transfer patients to hospital had a sensitivity of 0.84 (95% CI 0.83 to 0.85) and specificity 0.39 (95% CI 0.39 to 0.40) for the primary outcome. The positive predictive value (PPV) was 0.23 (95% CI 0.22 to 0.23) and the negative predictive value (NPV) 0.92 (95% CI 0.92 to 0.92). Hypothetical use of any of the 5 triage tools would have achieved a higher sensitivity than the decision to transfer to hospital by the EMS crews within the cohort, but in the case of NEWS2, the WHO algorithm and PMEWS, this was at a cost of a lower specificity (Table 2). Of the tools assessed at the pre-determined thresholds, CRB-65 achieved the highest specificity but at the cost of sensitivity, and the PRIEST tool achieved a balance between both sensitivity, specificity and C-statistic of 0.83 (95% CI 0.82 to 0.84). The triage tools generally demonstrated better discrimination (except NEWS2) and a higher sensitivity for the secondary outcome, but a lower specificity (Table 3). ## Discussion ### Summary The NEWS2, PMEWS, PRIEST tool and WHO algorithm identified patients at risk of adverse outcome with high sensitivity (>0.95) and specificity ranging between 0.3 (NEWS2) and 0.41 (PRIEST tool). They are therefore potentially suitable for use as triage tools to select patients for transfer to hospital. The high sensitivity of NEWS2 and PMEWS was achieved by using lower thresholds (NEWS2; 0–1 vs 2+ and PMEWS; 0–2 vs 3+) than previously recommended, based upon performance in an ED population of patients with suspected COVID-19 infection.19 At index attendance, 65% of patients were transported to hospital. Although a useful comparator for triage tool performance, the observed accuracy of EMS decision-making to transfer patients to hospital does not account for clinical best interest decisions not to covey patients to hospital who subsequently deteriorated, or patient wishes not to be conveyed. The sensitivity (0.84, 95% CI 0.83 to 0.85) and specificity (0.39, 95% CI 0.39 to 0.40) achieved by EMS decision making is nonetheless similar to that of tools used to triage undifferentiated patient acuity in the ED.21 To be clinically useful to EMS crews, the use of a triage tool would need to improve upon the existing sensitivity of clinical decision making, thereby reducing the risk of not transporting a patient to hospital who subsequently deteriorates, without leading to a disproportionately large increase in hospital conveyance. Use of any of the five triage tools at the pre-specified thresholds would potentially improve upon the sensitivity of existing EMS decision making. However, use of PMEWS, the WHO algorithm or NEWS2 would lead to up to a 10% increase in ED conveyances (Table 2). Use of both CRB65 and the PRIEST score would lead to improvements in sensitivity without sacrificing specificity. CRB65 achieved the highest specificity of any of the tools (0.54 95% CI: 0.53 to 0.54) and its use would reduce the number of patients conveyed to hospital by around 10%. However, patients not conveyed to hospital would have around a 4% risk of subsequently deteriorating. The PRIEST tool achieved a sensitivity of 0.97 (95% CI: 0.97 to 0.97) without increasing the number of patients transported to hospital. Using the PRIEST tool, patients who were not conveyed to hospital would have a 2% risk of subsequent deterioration (compared to an estimated 8% on EMS decision making in this cohort). ### Strengths and limitations Previous evaluations of triage tool accuracy and prognostic COVID-19 prognostic research in the pre-hospital setting, are limited by only including patients who were subsequently admitted to hospital.22-25 This is the first evaluation to use a large cohort of patients identified from routinely collected EMS records and linked to nationally collected, patient-level, healthcare data to provide robust outcome data for all patients including those not conveyed to hospital. We had low rates of missing data in the variables used in the triage tools assessed (Table 1). We also assessed the performance of triage tools in a cohort of patients with suspected infection which, in the absence of accurate universally available rapid COVID-19 diagnostic tests, reflects the population which EMS staff must clinically triage. Most existing research either aimed to determine if patients with suspected infection have COVID-19, or to risk stratify patients with confirmed infection in a hospital setting.26 Our evaluation of triage tool accuracy is limited to a single ambulance service, albeit one covering a large population across the North of England, so the results may not be generalisable to other healthcare settings. Other ambulance services may serve populations with a different risk-profile, provide different types of EMS response or have different thresholds and guidelines regarding when to convey patients to hospital. The population used is likely to be have similar baseline characteristics to that used to derive and validate the PRIEST score in an ED population, as it was conducted at the same time at hospitals in the region (and elsewhere in the UK).10 A sensitivity analysis in which patients recruited to the ED-based PRIEST study were removed from analysis, did not affect estimates of triage tool performance. The PRIEST tool may perform less well if applied to a different, especially non-UK, health care setting. We assumed that if co-morbidities were not recorded in routine data within the previous 12-months of the index event, they were not present. Our cohort is based on the clinical impression of likely COVID infection as determined by EMS crews. This is partly determined by prevalence of COVID-19 infection which varied during the study period however YAS guidance stated possible COVID-19 infection should be considered in all patients with shortness of breath, cough or fever and in patients with a history of close contact with someone with these symptoms. ### Implications Clinical tools should be used in conjunction with clinical decision making when determining whether a patient needs to be conveyed to hospital by EMS crews. As previously highlighted, there may be good clinical reasons why patients who subsequently deteriorated were not conveyed to hospital in this cohort. Within this limitation, our study provides evidence that use of existing clinical triage tools may improve clinical decision making in a prehospital setting where the prevalence of serious adverse outcomes is similar to the ED. In health care contexts where minimising risk of adverse outcomes in those not conveyed to hospital is the priority, use of PMEWS or the WHO criteria may be recommended, as they appear to optimise sensitivity. Use of the COVID-specific PRIEST tool would achieve almost the same gains in sensitivity (0.97 versus 0.98) without leading to a corresponding increase in patients being unnecessarily conveyed to hospital. The use of CRB65 would maximise specificity over gains in sensitivity, with a 4% risk of adverse outcomes in patients left to self-care in the community. This may be appropriate in resource constrained health care contexts and, as oxygen saturations do not form part of the assessment tool, it can be practically applied to a large range of health care settings. Further research assessing triage tool performance alongside clinical judgement in the prehospital setting would be helpful to determine whether triage tools would improve accuracy of decisions to transfer patients to hospital in practice. Given the high prevalence of adverse outcomes in this cohort, the findings may not be applicable to other lower risk community settings (e.g. patients being assessed by general practitioners) and therefore similar research is needed for these populations. ## Conclusion The NEWS2, PMEWS, PRIEST tool and WHO algorithm achieved high estimated sensitivities (>0.95) with respect to death or organ support, and specificities ranging between 0.3 (NEWS2) and 0.41 (PRIEST tool). EMS decision to transfer patients to hospital achieved a sensitivity of 0.84 (95% CI 0.83 to 0.85). Although, there may be good clinical reasons why patients who deteriorated were not conveyed to hospital, use of these triage tools would potentially improve EMS triage of patients with suspected COVID-19 infection. Use of the PRIEST tool would lead to significant gains in sensitivity without increasing the number of patients conveyed to hospital. ## Supporting information Supplementary Material [[supplements/261031_file03.docx]](pending:yes) ## Data Availability The data used for this study are subject to data sharing agreements with NHS Digital and Yorkshire Ambulance Service which prohibits further sharing of individual level data. The data sets used are obtainable from these organisations subject to necessary authorisations and approvals. ## Footnotes * Fax: +44 (0)114 222 0749 Tel: (+44) (0)114 222 4345, Email: C.Marincowitz{at}Sheffield.ac.uk * Fax: +44 (0)114 222 0749 Tel: (+44) (0)114 222 4345 * Received July 27, 2021. * Revision received July 27, 2021. * Accepted July 29, 2021. * © 2021, Posted by Cold Spring Harbor Laboratory This pre-print is available under a Creative Commons License (Attribution 4.0 International), CC BY 4.0, as described at [http://creativecommons.org/licenses/by/4.0/](http://creativecommons.org/licenses/by/4.0/) ## References 1. 1.Bentivegna M, Hulme C, Ebell MH. Primary Care Relevant Risk Factors for Adverse Outcomes in Patients With COVID-19 Infection: A Systematic Review. The Journal of the American Board of Family Medicine. 2021;34:S113. 2. 2.Gandhi RT, Lynch JB, del Rio C. Mild or Moderate Covid-19. New England Journal of Medicine. 2020;383:1757–1766. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1056/NEJMcp2009249&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F07%2F29%2F2021.07.27.21261031.atom) 3. 3.[https://www.who.int/publications/i/item/WHO-2019-nCoV-clinical-2021-1.4](https://www.who.int/publications/i/item/WHO-2019-nCoV-clinical-2021-1.4). 4. 4.[https://www.nice.org.uk/guidance/ng191](https://www.nice.org.uk/guidance/ng191). 5. 5.[https://www.cdc.gov/coronavirus/2019-ncov/hcp/clinical-guidance-management-patients.html](https://www.cdc.gov/coronavirus/2019-ncov/hcp/clinical-guidance-management-patients.html). 6. 6.Greenhalgh T, Thompson P, Weiringa S, et al. What items should be included in an early warning score for remote assessment of suspected COVID-19? qualitative and Delphi study. BMJ Open. 2020;10:e042626. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYm1qb3BlbiI7czo1OiJyZXNpZCI7czoxMzoiMTAvMTEvZTA0MjYyNiI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIxLzA3LzI5LzIwMjEuMDcuMjcuMjEyNjEwMzEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 7. 7.“Clinical care of severe acute respiratory infections – Tool kit,” 11 April 2020. [Online]. Available: [https://www.who.int/publications-detail/clinical-care-of-severe-acute-respiratory-infections-tool-kit](https://www.who.int/publications-detail/clinical-care-of-severe-acute-respiratory-infections-tool-kit). In: WHO, ed. 8. 8.[https://bestpractice.bmj.com/topics/en-gb/3000108/management-recommendations](https://bestpractice.bmj.com/topics/en-gb/3000108/management-recommendations). 9. 9.Goodacre S, Challen K, Wilson R, Campbell M. Evaluation of triage methods used to select patients with suspected pandemic influenza for hospital admission: cohort study. Health Technol Assess. 2010;14:173–236. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3310/hta14460-03&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=20923612&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F07%2F29%2F2021.07.27.21261031.atom) 10. 10.Goodacre S, Thomas B, Sutton L, et al. Derivation and validation of a clinical severity score for acutely ill adults with suspected COVID-19: The PRIEST observational cohort study. PLoS One. 2021;16:e0245840. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1371/journal.pone.0245840&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F07%2F29%2F2021.07.27.21261031.atom) 11. 11.Thomas B, Goodacre S, Lee E, et al. Prognostic accuracy of emergency department triage tools for adults with suspected COVID-19: The PRIEST observational cohort study. medRxiv. 2020:2020.2009.2002.20185892. 12. 12.National Early Warning Score (NEWS) 2: Standardising the assessment of acute-illness severity in the NHS. Updated report of a working party. London: RCP. In: Physicians RCo, ed2017. 13. 13.Lim WS, van der Eerden MM, Laing R, et al. Defining community acquired pneumonia severity on presentation to hospital: an international derivation and validation study. Thorax. 2003;58:377–382. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToidGhvcmF4am5sIjtzOjU6InJlc2lkIjtzOjg6IjU4LzUvMzc3IjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjEvMDcvMjkvMjAyMS4wNy4yNy4yMTI2MTAzMS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 14. 14.Challen K, Bright J, Bentley A, Walter D. Physiological-social score (PMEWS) vs. CURB-65 to triage pandemic influenza: a comparative validation study using community-acquired pneumonia as a proxy. BMC Health Serv Res. 2007;7:33. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/1472-6963-7-33&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=17328822&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F07%2F29%2F2021.07.27.21261031.atom) 15. 15.[https://digital.nhs.uk/services/national-data-opt-out](https://digital.nhs.uk/services/national-data-opt-out). 16. 16.Campbell MJ, Jacques RM, Fotheringham J, Maheswaran R, Nicholl J. Developing a summary hospital mortality index: retrospective analysis in English hospitals over five years. BMJ. 2012;344:e1001. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE3OiIzNDQvbWFyMDFfMS9lMTAwMSI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIxLzA3LzI5LzIwMjEuMDcuMjcuMjEyNjEwMzEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 17. 17.Street A, Maynou L, Gilbert T, Stone T, Mason S, Conroy S. The use of linked routine data to optimise calculation of the Hospital Frailty Risk Score on the basis of previous hospital admissions: a retrospective observational cohort study. The Lancet Healthy Longevity. 2021;2:e154–e162. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S2666-7568(21)00004-0&link_type=DOI) 18. 18.Rockwood K, Song X, MacKnight C, et al. A global clinical measure of fitness and frailty in elderly people. CMAJ. 2005;173:489–495. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czo5OiIxNzMvNS80ODkiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMS8wNy8yOS8yMDIxLjA3LjI3LjIxMjYxMDMxLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 19. 19.Thomas B, Goodacre S, Lee E, et al. Prognostic accuracy of emergency department triage tools for adults with suspected COVID-19: the PRIEST observational cohort study. Emergency Medicine Journal. 2021:emermed-2020-210783. 20. 20.Hirst E, Irving A, Goodacre S. Patient and public involvement in emergency care research. Emergency Medicine Journal. 2016;33:665–670. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiZW1lcm1lZCI7czo1OiJyZXNpZCI7czo4OiIzMy85LzY2NSI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIxLzA3LzI5LzIwMjEuMDcuMjcuMjEyNjEwMzEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 21. 21.Zachariasse JM, van der Hagen V, Seiger N, Mackway-Jones K, van Veen M, Moll HA. Performance of triage systems in emergency care: a systematic review and meta-analysis. BMJ Open. 2019;9:e026471. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYm1qb3BlbiI7czo1OiJyZXNpZCI7czoxMToiOS81L2UwMjY0NzEiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMS8wNy8yOS8yMDIxLjA3LjI3LjIxMjYxMDMxLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 22. 22.Lancet EA, Gonzalez D, Alexandrou NA, et al. Prehospital hypoxemia, measured by pulse oximetry, predicts hospital outcomes during the New York City COVID-19 pandemic. Journal of the American College of Emergency Physicians Open. 2021;2:e12407. 23. 23.Dillon K, Hook C, Coupland Z, Avery P, Taylor H, Lockyer A. Pre-hospital lowest recorded oxygen saturation independently predicts death in patients with COVID-19. Br Paramed J. 2020;5:59–65. 24. 24.Saberian P, Tavakoli N, Hasani-Sharamin P, Modabber M, Jamshididana M, Baratloo A. Accuracy of the pre-hospital triage tools (qSOFA, NEWS, and PRESEP) in predicting probable COVID-19 patients’ outcomes transferred by Emergency Medical Services. babol-caspjim. 2020;11:536–543. 25. 25.Inada-Kim M, Chmiel FP, Boniface MJ, Pocock H, Black JJM, Deakin CD. Validation of home oxygen saturations as a marker of clinical deterioration in patients with suspected COVID-19. medRxiv. 2020:2020.2011.2006.20225938. 26. 26.Wynants L, Van Calster B, Collins GS, et al. Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal. BMJ. 2020;369:m1328. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE3OiIzNjkvYXByMDdfMi9tMTMyOCI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIxLzA3LzI5LzIwMjEuMDcuMjcuMjEyNjEwMzEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9)