LMIC-PRIEST: Derivation and validation of a clinical severity score for acutely ill adults with suspected COVID-19 in a middle-income setting ============================================================================================================================================= * Carl Marincowitz * Peter Hodkinson * David McAlpine * Gordon Fuller * Steve Goodacre * Peter A. Bath * Laura Sbaffi * Madina Hasan * Yasein Omer * Lee Wallis ## Abstract **Background** Uneven vaccination and less resilient health care systems mean hospitals in LMICs are at risk of being overwhelmed during periods of increased COVID-19 infection. Risk-scores proposed for rapid triage of need for admission from the emergency department (ED) have been developed in higher-income settings during initial waves of the pandemic. **Methods** Routinely collected data for public hospitals in the Western Cape, South Africa from the 27th August 2020 to 11th March 2022 were used to derive a cohort of 446,084 ED patients with suspected COVID-19. The primary outcome was death or ICU admission at 30 days. The cohort was divided into derivation and Omicron variant validation sets. We developed the LMIC-PRIEST score based on the coefficients from multivariable analysis in the derivation cohort and existing triage practices. We externally validated accuracy in the Omicron period and a UK cohort. **Results** We analysed 305,564, derivation 140,520 Omicron and 12,610 UK validation cases. Over 100 events per predictor parameter were modelled. Multivariable analyses identified eight predictor variables retained across models. We used these findings and clinical judgement to develop a score based on South African Triage Early Warning Scores and also included age, sex, oxygen saturation, inspired oxygen, diabetes and heart disease. The LMIC-PRIEST score achieved C-statistics: 0.82 (95% CI: 0.82 to 0.83) development cohort; 0.79 (95% CI: 0.78 to 0.80) Omicron cohort; and 0.79 (95% CI: 0.79 to 0.80) UK cohort. Differences in prevalence of outcomes led to imperfect calibration in external validation. However, use of the score at thresholds of three or less would allow identification of very low-risk patients (NPV ≥0.99) who could be rapidly discharged using information collected at initial assessment. **Conclusion** The LMIC-PRIEST score shows good discrimination and high sensitivity at lower thresholds and can be used to rapidly identify low-risk patients in LMIC ED settings. **What is already known on this subject** * Uneven vaccination in low- and middle-income countries (LMICs) coupled with less resilient health care provision mean that emergency health care systems in LMICs may still be at risk of being overwhelmed during periods of increased COVID-19 infection. * Risk-stratification scores may help rapidly triage need for hospitalisation. However, those proposed for use in the ED for patients with suspected COVID-19 have been developed and validated in high-income settings. **What this study adds** * The LMIC-PRIEST score has been robustly developed using a large routine dataset from the Western Cape, South Africa and is directly applicable to existing triage practices in LMICs. * External validation across both income settings and COVID-19 variants showed good discrimination and high sensitivity (at lower thresholds) to a composite outcome indicating need for inpatient admission from the ED **How this study might affect research, practice or policy** * Use of the LMIC-PRIEST score at thresholds of three or less would allow identification of very low-risk patients (negative predictive value ≥0.99) across all settings assessed * During periods of increased demand, this could allow the rapid identification and discharge of patients from the ED using information collected at initial assessment. ## Background The severity of illness associated with COVID-19 has been reduced by mass vaccination and emergence of less severe variants. However, emergency health care systems in low- and middle-income countries (LMIC) may still be at risk of being overwhelmed during periods of increased infection, due to uneven vaccination and less resilient health care systems.1,2 Risk-stratification scores including the UK Royal College of Physicians National Early Warning Score, version 2 (NEWS2) and the COVID-specific Pandemic Respiratory Infection Emergency System Triage (PRIEST) score have been proposed to aid clinical decision-making around need for inpatient admission in the Emergency Department (ED) in patients with suspected COVID-19.3–5 Such risk-stratification scores were developed in high-income settings during initial waves of the pandemic.3,4 Validation of these scores in the middle-income setting of the Western Cape, South Africa, demonstrated good discrimination.6 However, the scores did not outperform existing clinical decision-making and used predictors and physiological cut-offs that are not part of routine clinical practice in this setting. In LMICs, disposition decision-making is based on clinician experience and gestalt.7 Use of risk-stratification score to allow rapid triage of need for hospitalisation can help prevent hospitals being overwhelmed and assist less-experienced clinicians during periods of increased COVID infection. To be applicable and easily useable in LMICs, a risk-stratification score must be based upon existing clinical triage practice. In South Africa patient acuity on arrival to the ED is triaged using the South African Triage Scale (SATS).8 The Western Cape of South Africa presented a unique opportunity to use routinely collected linked electronic health-care data, in a setting with a high degree of COVID case ascertainment compared to similar settings,9 to develop a risk-stratification score applicable to LMICs. Our study aimed to: 1. Develop a contextually appropriate clinical severity score for patients with suspected COVID-19 in an Emergency Department setting in the Western Cape. 2. Externally validate the developed score. ## Methods ### Study Design This observational cohort study used routinely collected clinical data from EDs across the Western Cape, from the Hospital Emergency Centre Triage and Information System (HECTIS)10 data repository, to develop a clinical risk-stratification score for ED patients with suspected COVID. The performance of the risk-stratification score was externally validated in patients who presented during the Omicron wave and in a cohort of patients from the UK Pandemic Respiratory Infection Emergency System Triage (PRIEST) study (collected during the first wave).11 The study was conducted and reported in accordance with Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) and Reporting of studies Conducted using Observational Routinely-collected Data (RECORD) guidelines.12,13 ### Setting For the development and Omicron period validation cohorts, data were collected from patients with suspected COVID-19 infection who attended public-sector EDs in the Western Cape Province. This is one of nine provinces in South Africa, and has almost 7 million inhabitants, of whom three quarters use public sector services.14 A convenience sample of seven hospital EDs (based on use of the HECTIS system) was selected, representing predominantly urban, Cape Town metropole district and, a large peri-rural hospital ED. Clinical decision-making was largely based on clinician experience, contextualised to the local status: at times hospitals were overwhelmed and admission thresholds were raised.15,16 No specific prognostic score were applied in the ED beyond routine triage with SATS. The external validation population was derived from the PRIEST mixed prospective and retrospective cohort study that collected data from 70 EDs across 53 sites in the UK between 26th March and 28th May 2020.3 ### Data Sources and linkage In the Western Cape, data on ED clinical presentation are routinely collected by the HECTIS system, including presenting complaint, triage variables and outcome of consultation. Through deterministic matching, based on unique patient hospital numbers (performed by the Western Cape Provincial Health Data Centre (PHDC)),14 linked data were obtained which included COVID test results from the National Health Laboratory Services (NHLS), comorbidities (based on prior health system encounters), data around admissions and movements within the health care system during the index COVID encounter, and death (if within, or reported to, the health care system). For patients with multiple ED attendances, data were extracted for the first ED attendance and outcomes were assessed up to 30 days from index attendance. Data collection for the UK PRIEST cohort study has been described in previous publications.3,17 An anonymised version of the study data was used to derive the external validation cohort.18 ### Inclusion Criteria Our Western Cape study sample consisted of all adults (aged 16 years and over) at the time of first (index) ED attendance between 27th August 2020 and 11th March 2022, where a clinical impression of suspected, or confirmed, COVID-19 infection had been recorded. For those with multiple presentations, analysis was limited to the index presentation. Patients who presented after the emergence of the Omicron variant (November 2021) were included in a validation cohort.19 Patients for whom age or sex were not recorded were excluded from analyses. ### Outcome The primary composite outcome, in the Western Cape population, was intubation or non-invasive ventilation in the ED on index attendance, Intensive Care Unit (ICU) admission or inpatient death up to 30 days from index attendance. This was comparable to the PRIEST study primary outcome of death or organ support (respiratory, cardiovascular, or renal) by record review at 30 days. The secondary outcomes were: 1) death and 2) ICU admission (organ support in UK cohort), up to 30 days. ### Patient characteristics and candidate predictor variables Physiological parameters and presenting complaints at triage at index ED presentation were extracted from the HECTIS database. Where no comorbidities were recorded, they were assumed not to be present. Implausible physiological variables were set as missing, including systolic blood pressure <50 mm HG, temperature >42 or <25 degrees, heart rate < 10/minute, oxygen saturation < 10% and respiratory rate = 0/minute. Candidate predictor variables were selected on the basis of a previous systematic review of factors suitable for use in LMICs, previous research and availability at ED triage in the Western Cape.1,3,20 Variables included: age, sex, presenting symptoms (cough or fever), co-morbidities (heart disease, diabetes, HIV, chronic lung disease, hypertension or pregnancy), physiological parameters and supplemental oxygen. Asthma/COPD was excluded from analysis due to an implausible protective relationship identified in preliminary modelling. ### Prognostic model development The model development cohort was randomly split into derivation and internal validation cohorts. Candidate predictors were combined in a multivariable regression with Least Absolute Shrinkage and Selection Operator (LASSO) using ten-sample cross-validation to select models. The LASSO began with a full model of candidate predictors and simultaneously performed predictor selection and penalisation during model development to avoid overfitting. The LASSO was performed twice: first, when the number of predictors were unrestricted, and a second time, with restriction to ten predictors. Estimates of selected model discrimination and calibration were performed in the split internal validation cohort. Continuous variables were modelled using fractional polynomials to account for non-linear forms and using categories based on TEWS (Supplementary Material 3).8 As TEWS categories are used as part of existing triage, unless alternative modelling methods demonstrated significant increases in accuracy, these categories were planned *a priori* to be used in the clinical severity score. Three multivariable analyses were completed using different approaches to missing predictor variable data in the derivation cohort for comparison: (1) Complete case; (2) Multiple imputation using chained equations (10 imputations); (3) Deterministic imputation with missing variable assumed to be in the normal range using TEWS categorisation. ### Clinical severity score derivation and validation Clinical members of the research team reviewed the models and selected variables for inclusion in the triage score, based on the prognostic value and consistency of selection across models, the clinical credibility of their association with the primary outcome, and their availability in the South African ED setting. Selected variables were categorised and assigned integer values using TEWS and the PRIEST score, if present in these clinical scoring systems, whilst checking that categorisation reflected the relationship between the variable and adverse outcome in the derived models. Additionally, selected variables were assigned integer values to each category of predictor variable, based on the coefficient derived from a multivariable logistic regression model using categorised continuous predictors. This generated a composite clinical score in which risk of adverse outcome increased with the total score. We applied the clinical score to the model development cohort, the Western Cape Omicron period and UK PRIEST external validation cohorts, calculating diagnostic parameters at each threshold of the score, constructing a receiver-operating characteristic (ROC) curve, calculating the area under the ROC curve (C-statistic) and calculating the proportion with an adverse outcome at each level of the score. Calibration plots for the risk-score were estimated in the external validation cohorts. We used deterministic imputation to handle missing data in the validation cohort, assuming missing predictor variable data were within normal physiological categories but excluding cases with fewer than three predictor variables. All analyses were completed in STATA version 17.21 ### Sample Size The sample size was fixed based on a census sample of patients in the Western Cape recorded on the HECTIS during the study period. In the smallest prognostic model development cohort, there were 102, 503 patients with over 100 outcomes per predictor parameter. #### Ethics The study was approved by the University of Cape Town Human Research Ethics Committee (HREC 594/2021), and the Western Cape Health Research Committee (WC_202111_034). All data were de-identified at source before being provided to the research team. #### Patient and Public Involvement (PPI) A community advisory board (CAB) comprising eight community members affected by COVID (infected themselves or immediate family infected/ hospitalised). PPI members were recruited by an experienced community liaison officer with links to key community groups. Members were intentionally sought to be representative of the various population groups and demographics of the population. Through several meetings, the CAB were kept abreast of the study, and given the opportunity to input on the outcomes, particularly the acceptability of the risk-stratification score. ## Results ### Study populations Figures 1 and 2 summarise population selection for the study cohorts. Table 1 summarises the characteristics of the 305,564 patients used for model development. Supplementary Materials 1 and 2 present the characteristics of the 140,520 patients in the Omicron, and 20,698 patients in the UK, validation cohorts. In total, 12,610 patients (4.13%, 95% CI:4.06% to 4.2%) experienced the primary outcome in the development cohort. This compared to, 2,787 patients (1.98%, 95% CI: 1.91% to 2.06%) in the Omicron period, and 4,579 patients (22.12%, 95% CI: 21.56% to 22.69%) in the UK, validation cohorts. In total, 74,580 patients used for model development (24.41%, 95% CI: 24.26 to 24.56%) had a diagnosis of COVID confirmed by PCR testing. View this table: [Table 1.](http://medrxiv.org/content/early/2022/11/07/2022.11.06.22281986/T1) Table 1. Patient characteristics by outcome in the model development cohort. ![Figure 1:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/11/07/2022.11.06.22281986/F1.medium.gif) [Figure 1:](http://medrxiv.org/content/early/2022/11/07/2022.11.06.22281986/F1) Figure 1: Flow diagram of study population selection Western Cape, South Africa ![Figure 2:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/11/07/2022.11.06.22281986/F2.medium.gif) [Figure 2:](http://medrxiv.org/content/early/2022/11/07/2022.11.06.22281986/F2) Figure 2: Derivation of UK PRIEST Validation cohort ### Prognostic models Tables 2, 3 and 4 show the results of multivariable analysis restricted to inclusion of 10 predictor variables using complete case analysis, multiple imputation and deterministic imputation. Supplementary Materials 4, 6 and 8 show the results of the unrestricted multivariable analyses. Supplementary Material 5 and 7 present the corresponding calibration plots from split internal validation. Unrestricted LASSO on multiply imputed data with modelling of continuous variables using multifractional polynomials produced the model with the highest C-statistic (0.87, 95% CI 0.866 to 0.874) and calibration in the large (CITL) of −0.017 (95%CI −0.043 to 0.009). However, restriction of modelling to 10 predictors and categorisation of continuous variables using TEWS only marginally reduced measures of accuracy in internal validation (worst performing restricted models; C-statistic 0.85 (95%CI 0.845 to 0.855) and CITL 0.126 (95% CI: 0.098 to 0.155)). View this table: [Table 2:](http://medrxiv.org/content/early/2022/11/07/2022.11.06.22281986/T2) Table 2: Restricted multivariable analysis complete case analysis (N=102, 402) View this table: [Table 3:](http://medrxiv.org/content/early/2022/11/07/2022.11.06.22281986/T3) Table 3: Restricted Deterministic imputation (N=152, 782) View this table: [Table 4:](http://medrxiv.org/content/early/2022/11/07/2022.11.06.22281986/T4) Table 4: Restricted multivariable analysis, using multiple imputation (10 imputations; N=152,782 each imputation) When restricted, there were eight predictors that were retained in all analyses (age, use of supplemental oxygen, oxygen saturation, diabetes, consciousness level, heart disease, respiratory rate, heart rate). Sex was retained in all but one complete case analysis. ### Clinical severity score derivation and validation Clinical review judged that the eight predictors retained in all models and sex (included in all but one model) were clinically credible and should be included in the clinical severity score. As TEWS categories for physiological parameters are used clinically in the Western Cape and categorisation did not materially reduce measures of model accuracy, these were used in the risk-score. TEWS also routinely includes measurement of systolic blood pressure and temperature, which were retained in unrestricted models and therefore also included. The co-morbidities diabetes and heart disease were assigned scores based on the relative size of their coefficients across models. As age had a similar modelled form and effect size to the original UK PRIEST study, it was assigned categories and scores based on the PRIEST score. The developed score is shown in Table 5. View this table: [Table 5:](http://medrxiv.org/content/early/2022/11/07/2022.11.06.22281986/T5) Table 5: LMIC-PRIEST score (Score 0-27) The LMIC-PRIEST score was applied to the model development cohort, Omicron and UK PRIEST validation cohorts. The estimated ROC curves for the primary outcome of these analyses are presented in Figure 3. Supplementary Material 9 and 10 show the estimated ROC curves when estimating the secondary outcome of 1) death or 2) admission to ICU/organ support. The score achieved better estimated discrimination when predicting death (C-statistic range: 0.79 UK cohort to 0.83 development cohort) compared to organ support/ICU admission (C-statistic range: 0.68 Omicron cohort to 0.74 development cohort). Figure 4 shows the calibration plots for performance of the score in the external validation cohorts. The score overestimated risk in the Omicron cohort as risk increased and systematically underestimated risk in the UK cohort. ![](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/11/07/2022.11.06.22281986/F3/graphic-12.medium.gif) [](http://medrxiv.org/content/early/2022/11/07/2022.11.06.22281986/F3/graphic-12) ![](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/11/07/2022.11.06.22281986/F3/graphic-13.medium.gif) [](http://medrxiv.org/content/early/2022/11/07/2022.11.06.22281986/F3/graphic-13) Figure 3: ROC curves for predicting primary outcome for modified PRIEST Risk Stratification tool ![](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/11/07/2022.11.06.22281986/F4/graphic-14.medium.gif) [](http://medrxiv.org/content/early/2022/11/07/2022.11.06.22281986/F4/graphic-14) ![](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/11/07/2022.11.06.22281986/F4/graphic-15.medium.gif) [](http://medrxiv.org/content/early/2022/11/07/2022.11.06.22281986/F4/graphic-15) Figure 4: Calibration curves for triage tool performance in external validation Existing clinical decision-making to admit patients to hospital from the ED in the South African setting had a sensitivity of 0.77 (95% CI 0.76 to 0.78) and specificity of 0.88 (95% CI 0.87 to 0.88) for the primary outcome (prevalence primary outcome 3.45%). The positive predictive value (PPV) was 0.18 (95% CI 0.18 to 0.18) and the negative predictive value was (NPV) 0.99 (95% CI 0.99 to 0.99). Clinicians discharged 85.28% of patients on first presentation. Table 6 presents the estimated sensitivity, specificity, negative and positive predictive values for levels of the score for the primary outcome in the three study populations that could be used clinically to inform admission decisions. Supplementary Materials 11, 12 and 13 shows these values for each level of the score and Supplemental Material 14 shows the risk of the primary outcome for at each level in the 3 study cohorts. View this table: [Table 6:](http://medrxiv.org/content/early/2022/11/07/2022.11.06.22281986/T6) Table 6: Sensitivity, specificity, PPV, NPV and proportion with a positive score at different score thresholds for predicting the primary outcome ## Discussion ### Summary The LMIC-PRIEST score has been developed using a large cohort of patients with suspected COVID and a study period that encompasses Beta, Delta and Omicron waves in the Western Cape. Alongside use of an external validation cohort, we were able to assess accuracy in both different income settings and variants. The LMIC-PRIEST score has shown consistent discrimination across different settings, C-statistics: 0.82 (95% CI: 0.82 to 0.83) development cohort, 0.79 (95% CI: 0.78 to 0.80) Omicron cohort and 0.79 (95% CI: 0.79 to 0.80) UK validation cohort. However, differences in prevalence of adverse outcomes resulted in over- and underestimation of risk in the Omicron and UK external validation cohorts (Figure 4). The LMIC-PRIEST score builds on existing clinical triage practices in South Africa, and in addition to parameters used to calculate SATS, the score includes other routinely collected variables. The score is therefore clinically applicable to the intended setting of use. In existing practice, clinicians admitted 14.7% of patients, and discharged patients had a 1% risk of the primary outcome (NPV 0.99, 95% CI 0.99 to 0.99). Use of the score at score thresholds up to <5 could offer gains in sensitivity, but, in the Western Cape, this would increase the proportion of admitted patients with a very small associated reduced risk of false negative triage. Lower thresholds could be used to rapidly and transparently identify a proportion of very low-risk patients who could be discharged from the ED based on information routinely collected at initial assessment. ### Comparison to previous literature A systematic review found that no risk-stratification scores for patients with suspected COVID had been developed and validated in LMICs.20 The Nutri-CoV score was subsequently developed and validated in Mexico using data from the first wave of the pandemic and has been proposed for use to triage patient acuity in hospital settings.22 In internal validation, the score achieved a C-statistic of 0.797 (95% CI 0.765 to 0.829), sensitivity 0.93 (95% CI 0.88 to 0.98), specificity 0.11 (95% CI 0.02 to 0.21) and NPV 0.6 (95% CI 0.34 to 0.87) when estimating death or ICU admission at a recommended threshold. Although not explicitly requiring inpatient investigations, a key predictor in the Nutri-CoV score is diagnosis of pneumonia, which requires either radiological or clinical diagnosis. Consequently, the Nutri-CoV score may not be suitable for rapid identification of low-risk patients suitable for discharge using information available at triage. Unlike the LMIC-PRIEST score score, the Nutri-COV score has not undergone external validation in different COVID variants or income settings. The variables consistently selected by LASSO modelling and used to inform the LMIC-PRIEST score are consistent with other studies.23 Age, inspired oxygen and oxygen saturations have been found, as in our models, to be highly predictive of adverse outcomes in the ED setting.3,24 Although diabetes and heart disease, amongst other comorbidities, have been found to be prognostic in COVID infection, they have not been found to be as highly predictive of adverse outcomes in the ED setting as in our study.3 However, diabetes has previously been identified as a strong predictor (OR 1.84, 95% CI 1.24 to 2.73) of death in patients with COVID in studies conducted in South Africa.15 ### Strengths and limitations We followed robust statistical model development techniques that have been rated as low risk of bias and have an adequate sample size for model development.3,23 Restricted and unrestricted models derived using different methods for handling missing data, alongside clinical judgement, were used for score development. These measures all reduce the risk of over-optimistic prediction and inclusion of variables by chance that are not truly predictive. The use of cohorts of patients with suspected infection means that the score has been developed and validated in the population of intended use, the population whom ED staff must clinically triage.1 Our available datasets also comprise multiple COVID waves and different income settings allowing external validation and assessment of generalisability.19 The cohort used for model development was collected from selected government hospitals using the recently implemented HECTIS system. Use of the HECTIS system, electronic records and linking of data from various sources is in its infancy in this context and is dependent on many data entry points across facilities and institutions. Data collection is not primarily intended for research purposes and may be subject error and missingness. However, the HECTIS system is used clinically to collect and record the physiological and other variables used to calculate SATS in the ED as part of clinical practice.25 Deaths are recorded if they occurred in, or were notified to, a health facility. Our cohort is formed by patients who were tested and diagnosed with COVID and clinical staff performing the initial assessment in the ED had a strong clinical impression of likely infection. This was partly determined by prevalence of infection and clinical guidance, which varied during the study period. Although use of PRIEST study data allowed external validation of our developed score, the intended setting of use is in other LMICs during current waves of the pandemic. ### Implications During periods of increased COVID prevalence, patients in South Africa with suspected infection were found to bypass primary care and self-present to hospitals.15 This was associated with excess attendances for patients who required no specific treatment. This partly explains the lower prevalence of the primary outcome in the Western Cape. In the UK, telephone triage was used effectively to reduce ED attendances of lower-risk patients.26 Disposition decision-making in LMICs is based on clinician experience and gestalt.7 Existing clinical decision-making was found to perform well with only 14.7% of patients admitted as inpatients and a risk of false negative triage of around 1%. Although clinical-decision rules have been found to rarely out-perform clinician gestalt,27 exercising clinical judgement requires time and experience, which may be limited during periods of increased demand. Despite, imperfect calibration in external validation, use of the LMIC-PRIEST score at thresholds of three or less would allow identification of very low-risk patients (NPV ≥0.99) across different settings using information routinely collected during initial triage. During periods of increased COVID prevalence and corresponding ED attendances, the score could potentially be used by practitioners with basic training to identify very low-risk patients for discharge without full clinical assessment, thereby reducing the risk of hospitals being overwhelmed. A conservative, high sensitivity (0.96) threshold of >2, in the Omicron validation cohort would allow the theoretical discharge of 30% of assessed patients. The wide variation in the prevalence of the primary outcome between settings resulted in miscalibration in external validation. The LMIC-score will need calibration in settings of intended use. This may simply involve selecting the most appropriate threshold based on the population risk and clinical context. However, the emergence of new variants may require different weightings of predictor variables. A primary outcome of death and ICU admission/organ support was used to encompass need for hospital admission.3 However, across all development and validation settings, the score predicted death better than ICU admission/organ support (Supplementary 9 and 10). The accuracy of the LMIC-PRIEST score for the composite outcome should not be used to guide treatment decisions beyond need for admission, such as potential benefit from invasive treatments, as differences in the prediction of death and interventions are likely to mean that the estimation of benefit is inaccurate.28 ## Conclusion The LMIC-PRIEST score has been developed using robust methods and the score shows generalisable discrimination across a range of COVID variants and income settings. It is specifically designed to be used as part of existing triage practices in South Africa and other LMICs. The score could be used to identify very low-risk patients with suspected COVID infection rapidly and transparently during periods in which health care systems experience increased demand due to a high prevalence of infection. Further external validation may be necessary if the score is used in different settings or novel COVID variants. ## Data Availability The data used for this study are subject to a data sharing agreement with the Western Cape Government Department of Health and Wellness, which prohibits further sharing of patient- level data. Access to these and related data should be requested directly from this organisation and is subject to the necessary ethical and organisational approval processes. ## Author Disclosure Statement No competing financial interests exist. ## Funding CM is a National Institute for Health Research (NIHR) Clinical Lecturer in Emergency Medicine (Grant Number Not Applicable/NA). This work is part of the Grand Challenges ICODA pilot initiative, delivered by Health Data Research UK and funded by the Bill & Melinda Gates Foundation and the Minderoo Foundation. The Provincial Health Data Centre (PHDC), Health Intelligence Directorate, Western Cape Government Health and Wellness acknowledges funding from the United States National Institutes of Health (R01HD080465, U01AI069911), Bill and Melinda Gates Foundation (1164272; 1191327; INV-004657, INV-017293), the Wellcome Trust (203135/Z/16/Z), the United States Agency for International Development (72067418CA00023) ## Authors’ contributions The idea for the study was conceived by SG, LW, PH, PB, GF and CM. Data processing and linkage was supervised and completed DM. The analyses were completed by CM with additional expert advice from SG, and PB. All authors contributed to interpretation of results, read, and approved the final manuscript. ## Data sharing The data used for this study are subject to a data sharing agreement with the Western Cape Government Department of Health and Wellness, which prohibits further sharing of patient-level data. Access to these and related data should be requested directly from this organisation and is subject to the necessary ethical and organisational approval processes. View this table: [Supplementary Material 1:](http://medrxiv.org/content/early/2022/11/07/2022.11.06.22281986/T7) Supplementary Material 1: Population characteristics Omicron period validation cohort View this table: [Supplementary Material 2:](http://medrxiv.org/content/early/2022/11/07/2022.11.06.22281986/T8) Supplementary Material 2: Population characteristics UK PRIEST validation cohort View this table: [Supplementary Material 3:](http://medrxiv.org/content/early/2022/11/07/2022.11.06.22281986/T9) Supplementary Material 3: Categorisation of continuous variables using TEWS View this table: [Supplementary Material 4:](http://medrxiv.org/content/early/2022/11/07/2022.11.06.22281986/T10) Supplementary Material 4: Multivariable Analysis complete case analysis (N=102, 402) ![Supplementary Material 5:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/11/07/2022.11.06.22281986/F5.medium.gif) [Supplementary Material 5:](http://medrxiv.org/content/early/2022/11/07/2022.11.06.22281986/F5) Supplementary Material 5: Calibration plots split internal validation complete case analysis View this table: [Supplementary Material 6:](http://medrxiv.org/content/early/2022/11/07/2022.11.06.22281986/T11) Supplementary Material 6: Unrestricted Deterministic Imputation (N=152,782) ![Supplementary Material 7:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/11/07/2022.11.06.22281986/F6.medium.gif) [Supplementary Material 7:](http://medrxiv.org/content/early/2022/11/07/2022.11.06.22281986/F6) Supplementary Material 7: Calibration plots split internal validation deterministic imputation View this table: [Supplementary Material 8:](http://medrxiv.org/content/early/2022/11/07/2022.11.06.22281986/T12) Supplementary Material 8: Multivariable analysis, using multiple imputation (10 imputations; N=152,782) ![Supplementary Material 9:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/11/07/2022.11.06.22281986/F7.medium.gif) [Supplementary Material 9:](http://medrxiv.org/content/early/2022/11/07/2022.11.06.22281986/F7) Supplementary Material 9: ROC curves for predicting secondary outcome death for LMIC-PRIEST score ![Supplementary Material 10:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/11/07/2022.11.06.22281986/F8.medium.gif) [Supplementary Material 10:](http://medrxiv.org/content/early/2022/11/07/2022.11.06.22281986/F8) Supplementary Material 10: ROC curves for predicting secondary outcome ICU admission/organ support for LMIC-PRIEST Score View this table: [Supplementary Material 11:](http://medrxiv.org/content/early/2022/11/07/2022.11.06.22281986/T13) Supplementary Material 11: Sensitivity, specificity, PPV, NPV and proportion with a positive score at each LMIC-PRIEST score threshold for predicting the primary outcome model development cohort (N=305,564) View this table: [Supplementary Material 12:](http://medrxiv.org/content/early/2022/11/07/2022.11.06.22281986/T14) Supplementary Material 12: Sensitivity, specificity, PPV, NPV and proportion with a positive score at each LMIC-PRIEST score threshold for predicting the primary outcome model Omicron validation cohort (N= N=140,520) View this table: [Supplementary Material 13:](http://medrxiv.org/content/early/2022/11/07/2022.11.06.22281986/T15) Supplementary Material 13: Sensitivity, specificity, PPV, NPV and proportion with a positive score at each LMIC-PRIEST score threshold for predicting the primary outcome model UK PRIEST validation cohort (N= N=20,698) ![Supplementary Material 13:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/11/07/2022.11.06.22281986/F9.medium.gif) [Supplementary Material 13:](http://medrxiv.org/content/early/2022/11/07/2022.11.06.22281986/F9) Supplementary Material 13: Probability of primary adverse outcome for each value of the LMIC-PRIEST score ## Acknowledgements This work uses data provided by patients as part of their care and support and the authors wish to recognise the Western Cape Government Health and Wellness (WCGHW) for their contribution of the data that made this research possible, specifically Nesbert Zinyakatira and the team from the Provincial Health Data Centre, Health Impact Assessment Directorate, Western Cape Government Health; and Dr Moosa Parak and the HECTIS team. We further acknowledge and thank the National Health Laboratory Service of South Africa, for their contribution to the study through the provision of the digitised laboratory results accessed through the Provincial Health Data Centre. Dr Laura Sutton (University of Sheffield) conducted the statistical analysis for the UK PRIEST study and provided a template for the statistical analyses. ## Footnotes * Fax: +44 (0)114 222 0749 Tel: (+44) (0)114 222 4345, Email: C.Marincowitz{at}Sheffield.ac.uk * Received November 6, 2022. * Revision received November 6, 2022. * Accepted November 7, 2022. * © 2022, Posted by Cold Spring Harbor Laboratory This pre-print is available under a Creative Commons License (Attribution 4.0 International), CC BY 4.0, as described at [http://creativecommons.org/licenses/by/4.0/](http://creativecommons.org/licenses/by/4.0/) ## References 1. 1.Barros LM, Pigoga JL, Chea S, et al. Pragmatic Recommendations for Identification and Triage of Patients with COVID-19 in Low- and Middle-Income Countries.Am J Trop Med Hyg 2021;104(3_Suppl):3–11. doi: 10.4269/ajtmh.20-1064 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.4269/ajtmh.20-1064&link_type=DOI) 2. 2.Polašek O, Wazny K, Adeloye D, et al. Research priorities to reduce the impact of COVID-19 in low- and middle-income countries.J Glob Health 2022;12:09003–03. doi: 10.7189/jogh.12.09003 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.7189/jogh.12.09003&link_type=DOI) 3. 3.Goodacre S, Thomas B, Sutton L, et al. Derivation and validation of a clinical severity score for acutely ill adults with suspected COVID-19: The PRIEST observational cohort study.PLoS One 2021;16(1):e0245840. doi: 10.1371/journal.pone.0245840 [published Online First: 2021/01/23] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1371/journal.pone.0245840&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F11%2F07%2F2022.11.06.22281986.atom) 4. 4.Suh EH, Lang KJ, Zerihun LM. Modified PRIEST score for identification of very low-risk COVID patients.Am J Emerg Med 2021;47:213–16. doi: 10.1016/j.ajem.2021.04.063 [published Online First: 2021/04/28] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.ajem.2021.04.063&link_type=DOI) 5. 5.1. Physicians RCo, ed National Early Warning Score (NEWS) 2: Standardising the assessment of acute-illness severity in the NHS. Updated report of a working party. London: RCP. In: Physicians RCo, ed., 2017. 6. 6.Marincowitz C, Sbaffi L, Hodkinson P, et al. Prognostic accuracy of triage tools for adults with suspected COVID-19 in a middle-income setting: an observational cohort study.medRxiv 2022:2022.08.23.22279112. doi: 10.1101/2022.08.23.22279112 [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoibWVkcnhpdiI7czo1OiJyZXNpZCI7czoyMToiMjAyMi4wOC4yMy4yMjI3OTExMnYxIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjIvMTEvMDcvMjAyMi4xMS4wNi4yMjI4MTk4Ni5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 7. 7.Soto-Mota A, Marfil-Garza BA, Castiello-de Obeso S, et al. Prospective predictive performance comparison between clinical gestalt and validated COVID-19 mortality scores.J Investig Med 2022;70(2):415–20. doi: 10.1136/jim-2021-002037 [published Online First: 2021/10/09] [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiamltIjtzOjU6InJlc2lkIjtzOjg6IjcwLzIvNDE1IjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjIvMTEvMDcvMjAyMi4xMS4wNi4yMjI4MTk4Ni5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 8. 8.[https://emssa.org.za/wp-content/uploads/2011/04/SATS-Manual-A5-LR-spreads.pdf](https://emssa.org.za/wp-content/uploads/2011/04/SATS-Manual-A5-LR-spreads.pdf). 9. 9.Wang H, Paulson KR, Pease SA, et al. Estimating excess mortality due to the COVID-19 pandemic: a systematic analysis of COVID-19-related mortality, 2020–21.The Lancet 2022;399(10334):1513–36. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/s0140-6736(21)02796-3&link_type=DOI) 10. 10.[https://www.gov.za/speeches/western-cape-government-launches-safety-dashboard-providing-real-time-data-provincial](https://www.gov.za/speeches/western-cape-government-launches-safety-dashboard-providing-real-time-data-provincial). 11. 11.Goodacre S, Thomas B, Lee E, et al. Characterisation of 22445 patients attending UK emergency departments with suspected COVID-19 infection: Observational cohort study.PLOS ONE 2020;15(11):e0240206. doi: 10.1371/journal.pone.0240206 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1371/journal.pone.0240206&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F11%2F07%2F2022.11.06.22281986.atom) 12. 12.Benchimol EI, Smeeth L, Guttmann A, et al. The REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) Statement.PLOS Medicine 2015;12(10):e1001885. doi: 10.1371/journal.pmed.1001885 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1371/journal.pmed.1001885&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=26440803&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F11%2F07%2F2022.11.06.22281986.atom) 13. 13.Moons KG, Altman DG, Reitsma JB, et al. New Guideline for the Reporting of Studies Developing, Validating, or Updating a Multivariable Clinical Prediction Model: The TRIPOD Statement.Adv Anat Pathol 2015;22(5):303–5. doi: 10.1097/PAP.0000000000000072 [published Online First: 2015/08/12] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1097/PAP.0000000000000072&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F11%2F07%2F2022.11.06.22281986.atom) 14. 14.Boulle A, Heekes A, Tiffin N, et al. Data centre profile: the provincial health data centre of the Western Cape Province, South Africa.International Journal of Population Data Science 2019;4(2) 15. 15.Mash RJ, Presence-Vollenhoven M, Adeniji A, et al. Evaluation of patient characteristics, management and outcomes for COVID-19 at district hospitals in the Western Cape, South Africa: descriptive observational study.BMJ Open 2021;11(1):e047016. doi: 10.1136/bmjopen-2020-047016 [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYm1qb3BlbiI7czo1OiJyZXNpZCI7czoxMjoiMTEvMS9lMDQ3MDE2IjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjIvMTEvMDcvMjAyMi4xMS4wNi4yMjI4MTk4Ni5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 16. 16.Vallabhjee K, Gilson L, Davies M-A, et al. Reflections on the health system response to COVID-19 in the Western Cape Province.South African Health Review 2021;2021(1):173–87. 17. 17.Thomas B, Goodacre S, Lee E, et al. Prognostic accuracy of emergency department triage tools for adults with suspected COVID-19: the PRIEST observational cohort study.Emergency Medicine Journal 2021:emermed-2020-210783. doi: 10.1136/emermed-2020-210783 [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiZW1lcm1lZCI7czo1OiJyZXNpZCI7czo4OiIzOC84LzU4NyI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIyLzExLzA3LzIwMjIuMTEuMDYuMjIyODE5ODYuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 18. 18.Thomas, Benjamin; Sutton, Laura; Goodacre, Steve; Biggs, Katie; Loban, Amanda (2020): PRIEST study anonymised dataset. The University of Sheffield. Dataset. [https://doi.org/10.15131/shef.data.13194845.v1](https://doi.org/10.15131/shef.data.13194845.v1) 19. 19.Mendelsohn AS, D. Sá A, Morden E, et al. COVID-19 wave 4 in Western Cape Province, South Africa: Fewer hospitalisations, but new challenges for a depleted workforce.S Afr Med J 2022;112(2):13496. [published Online First: 2022/02/11] 20. 20.Hirner S, Pigoga JL, Naidoo AV, et al. Potential solutions for screening, triage, and severity scoring of suspected COVID-19 positive patients in low-resource settings: a scoping review.BMJ open 2021;11(9):e046130–e30. doi: 10.1136/bmjopen-2020-046130 [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYm1qb3BlbiI7czo1OiJyZXNpZCI7czoxMjoiMTEvOS9lMDQ2MTMwIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjIvMTEvMDcvMjAyMi4xMS4wNi4yMjI4MTk4Ni5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 21. 21.StataCorp. 2021. Stata Statistical Software: Release 17. College Station, TX: StataCorp LLC. 22. 22.Bello-Chavolla OY, Antonio-Villa NE, Ortiz-Brizuela E, et al. Validation and repurposing of the MSL-COVID-19 score for prediction of severe COVID-19 using simple clinical predictors in a triage setting: the Nutri-CoV score.PLoS One 2020;15(12):e0244051. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1371/journal.pone.0244051&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F11%2F07%2F2022.11.06.22281986.atom) 23. 23.Update to living systematic review on prediction models for diagnosis and prognosis of covid-19.BMJ 2021;372:n236. doi: 10.1136/bmj.n236 [FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE2OiIzNzIvZmViMDNfMy9uMjM2IjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjIvMTEvMDcvMjAyMi4xMS4wNi4yMjI4MTk4Ni5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 24. 24.Haimovich AD, Ravindra NG, Stoytchev S, et al. Development and Validation of the Quick COVID-19 Severity Index: A Prognostic Tool for Early Clinical Decompensation.Ann Emerg Med 2020;76(4):442–53. doi: 10.1016/j.annemergmed.2020.07.022 [published Online First: 2020/07/21] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.annemergmed.2020.07.022&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F11%2F07%2F2022.11.06.22281986.atom) 25. 25.Dixon J, Burkholder T, Pigoga J, et al. Using the South African Triage Scale for prehospital triage: a qualitative study.BMC Emergency Medicine 2021;21(1):125. doi: 10.1186/s12873-021-00522-3 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/s12873-021-00522-3&link_type=DOI) 26. 26.Marincowitz C, Stone T, Bath P, et al. Accuracy of telephone triage for predicting adverse outcomes in suspected COVID-19: an observational cohort study.BMJ Qual Saf 2022 doi: 10.1136/bmjqs-2021-014382 [published Online First: 2022/04/01] [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjE5OiJibWpxcy0yMDIxLTAxNDM4MnYxIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjIvMTEvMDcvMjAyMi4xMS4wNi4yMjI4MTk4Ni5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 27. 27.Sanders S, Doust J, Glasziou P. A systematic review of studies comparing diagnostic clinical prediction rules with clinical judgment.PLoS One 2015;10(6):e0128233. doi: 10.1371/journal.pone.0128233 [published Online First: 2015/06/04] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1371/journal.pone.0128233&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=26039538&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F11%2F07%2F2022.11.06.22281986.atom) 28. 28.Dash K, Goodacre S, Sutton L. Composite Outcomes in Clinical Prediction Modeling: Are We Trying to Predict Apples and Oranges? Ann Emerg Med 2022