Abstract
Aim To manage patients with suspected coronavirus disease (COVID-19) when they arrive at the hospital emergency department (ED), a clinical severity score is required to quickly identify patients requiring immediate hospital admission and close monitoring. The aim of this study was to evaluate, within the context of the pandemic, the performance of National Early Warning Score 2 (NEWS-2) to anticipate the admission of patients with suspected COVID-19 to a specialised emergency care unit.
Methods This retrospective study was conducted on patients presenting at the COVID-19 entrance of the ED of the Vert-Galant private hospital (Paris, France) during the first national pandemic peak from March 20 to April 20, 2020. All patients completed a questionnaire and clinical data and vital signs were recorded. Statistical analysis and modelling were used to estimate the ability of different scores (NEWS-2, qSOFA, CRB-65) to predict hospital emergency admission and/or early COVID-19 diagnosis.
Results NEWS-2, with a cut off value of 5, predicted hospital admission with 82% sensitivity, 98% specificity and an area under the curve (AUC) of 96%. NEWS-2 was superior to qSOFA and CRB-65 scores for predicting hospital admission of COVID-19 patients. Multilinear or logistic regression analysis of clinical data did not improve this result.
Conclusion NEWS-2 is an excellent score to predict hospital admission of COVID-19 patients.
Introduction
The COVID-19 pandemic started in early December 2019 in Wuhan, China, and spread worldwide as a severe acute respiratory syndrome caused by the novel coronavirus SARS-CoV-2.1 On January 30, 2020, the World Health Organisation (WHO) declared COVID-19 as a Public Health Emergency of International Concern (PHEIC) and designated it as a pandemic on March 11, 2020.2 In France, the first case of COVID-19 was identified on January 24, 2020, and “level three” emergency of the COVID epidemic was officially declared by the French authorities on March 14, 2020.
In many countries, the pandemic resulted in a massive influx of patients to hospital emergency departments (ED). The rapid spread of the disease, the increasing number of severe cases requiring hospital admission and the lack of preparation for this unexpected patient influx by the existing healthcare system meant that hospitals had to quickly adapt and modify their strategies for efficient hospital admission.
COVID-19 is associated with a great variety and severity of clinical symptoms; 20% of infected patients who go to hospital will require immediate admission and 5% will be admitted to an intensive care unit (ICU), mainly due to respiratory problems.3-6 Although the hospital system in France offers an average of 621 hospital beds per 100 000 inhabitants, in regions of high incidence during the study period most hospitals were overwhelmed by the sudden influx of patients.7 In this context, the Vert-Galant Hospital (VGH), a 100-bed private hospital located in Tremblay-en-France, Ile-de-France, 33 km North-East of the centre of Paris, underwent a major organisational change of its ED in order to cope with such an unexpected situation. This reorganisation of the ED encompassed a specific rapid patient pathway that included triage, an initial questionnaire, recording of vital signs with a telemedicine booth (TMB) and a medical examination.
In times of health crisis and a large influx of patients to the ED, clinical scores are often used to rapidly and accurately refer patients. In an evaluation of the changes to be implemented during the epidemic crisis, the use of a standardised and rapid method to assess the severity of COVID-19 patients presenting at the ED was recommended to effectively address the need for admission with rapid hospital triage.8-10 Thus, the National Early Warning Score 2 (NEWS-2) that was originally developed as a tool for patient evaluation and monitoring by the Royal College of Physicians in 2012 was identified.11-13 All data needed for its calculation are recorded routinely by medical staff. The score aims to improve communication between nurses and physicians, to standardise basic triage practices based on the interpretation of vital signs and, ultimately, to identify patients requiring hospitalisation or intensive care at an early stage.14-16 This is reinforced by the instability of vital signs, particularly respiratory, which are a predictive factor for cardiorespiratory aggravation of COVID-19.17
The aim of this observational, retrospective, single centre study was to evaluate the performance of NEWS-2 in the disposition of patients with suspected COVID-19 presenting at the ED: either hospital admission (HOS) or discharge home (DH). The secondary aims were to compare NEWS-2 with the qSOFA (quick Sequential Sepsis-related Organ Failure Assessment)18 and CRB-65 (Mental Confusion, Respiratory rate, Blood pressure, Age)19 scores, and to evaluate the concordance between NEWS-2 and pulmonary computed tomography (CT) scans as a marker of disease severity. The final aim was to determine whether NEWS-2 can be considered as an aid to COVID-19 diagnosis.
Methods
Study design and population
All patients included in this study were adults (≥18-years-old) and presented for triage at the ED of the COVID sector of VGH between March 19 and April 29, 2020. The exclusion criteria included: more than one presentation during the recruitment period; hospital admission despite the lack of clinical severity due to social reasons or if discharging the patient back home was not possible; protected patient including: adult under guardianship or other legal protection, deprived of liberty by judicial or administrative decree; pregnancy or breastfeeding; hospital admission without consent; and missing data preventing calculation of NEWS-2. All patients presenting at the ED with at least one of the respiratory and gastrointestinal signs and symptoms of COVID-19, as described in the literature, were included in the study. They represent 41% of the total ED population during the study period.20
Ethics and data protection
All patients were informed of the data collection by a specific poster in the waiting room and gave their informed consent prior to the recording of vital signs by the TMB. All patients received a letter informing them of the study and gave their informed consent, according to MR004 methodology.
The project, (referenced under the number COS-RGDS-2021-05-001-FAURE-V) has been submitted to the members of the Scientific Committee of the GCS Ramsay Santé for Education and Research. The committee has reviewed the following documents: application form for ethical advice on a project research and the synopsis of the study. This study has received a favorable opinion on 05/04/2021 (Numéro IORG de Ramsay santé Recherche & Enseignement: IORG0009085; Numéro IRB du Conseil d’Orientation Scientifique: IRB00010835).
Data collection
Demographic and clinical data, vital signs and details of patient disposition were collected retrospectively from the electronic medical records. During the day, a Class 2 certified TMB was used, allowing the autonomous measurement and automatic recording of vital signs (pulse rate, oxygenation, systolic pressure (SBP), diastolic pressure (DBP), pulse pressure, mean arterial pressure (MAP), temperature and respiratory rate). The TMB equipment (ConsultStation®) was generously provided by H4D Inc., Paris, France. The data collected by the TMB were copied into the electronic record.
Data for mobile patients were collected by the TMB, while data for bed-bound patients were collected by the nursing staff. The clinical signs recorded included: changes in consciousness; breathlessness; altered general condition (defined as a clinical condition where the vital signs were not significantly altered, but other clinical symptoms such as intense asthenia, vomiting and anorexia (difficulty eating and drinking during the previous 24 h) pointed to hospital admission. By convention, when a figure for the respiratory rate was not available and the physician reported the respiration of the patient as “normal” or “eupnoeic” on the report, it was quantified as 16 breaths min-1. All data, including that from the medical and paramedical examination and data collected by the TMB, were analysed anonymously.
Scores and complementary examinations
Three clinical scores were calculated and compared: NEWS-2, qSOFA and CRB-65.
NEWS-2 is based on the measurement of six physiological parameters: respiratory rate, oxygen saturation, SBP, pulse rate, level of consciousness and body temperature. Each parameter is allocated a sub-score as it is measured, with the magnitude reflecting how much the parameter varies from the norm. The sub-scores are then added together as indicated in the latest version of NEWS-2.13 NEWS-2 score can range from 0 to 20, with a score of ≥7 indicating the need for a rapid clinical assessment and potential transfer of the patient to an ICU.
The qSOFA score is used to identify a possible infectious syndrome. Total score ranges from 0 to 3 with one point each allocated for low blood pressure (SBP ≤100 mmHg), high respiratory rate (≥22 breaths min-1) and altered consciousness (Glasgow coma scale <15).
Finally, the CBR-65 score is used to assess the severity of community-acquired pneumonia and to determine whether a patient requires inpatient treatment or not. Total CRB-65 score ranges from 0 to 4, with one point each being allocated for: mental confusion, respiratory rate ≥30 breaths min-1, SBP <90 mmHg or DBP ≤60 mmHg, and age ≥65 years.
Other tests included RT-PCR for the diagnosis of COVID-19 and a pulmonary CT scan interpreted according to the Society for Thoracic Imaging (STI).21
Evaluation criteria
The primary evaluation criterion was the concordance between NEWS-2 and real disposition of the patient from the ED: HOS or DH. The secondary objectives were to compare NEWS-2 with the qSOFA and CRB-65 scores for patient disposition and to determine whether NEWS-2 can be considered as an aid to COVID-19 diagnosis.
Statistical analysis
The ability of NEWS-2 to predict HOS or DH was first assessed by descriptive and exploratory analyses. Sensitivity/specificity analysis and receiver operating characteristic (ROC) curves were used to explore NEWS-2 performance and to detect the best cut-off value for NEWS-2 as a predictor of patient disposition and COVID-19 pre-diagnosis severity. To avoid problems with multiple testing, P values were adjusted using Bonferroni correction to keep the false discovery rate (α-risk) at 5%. Discriminant analysis or statistical models based on multilinear or logistic regression were used to determine whether adding weights to sub-scores or adding new parameters (e.g., comorbidities, anorexia, age, general condition of the patient) could improve the performance of NEWS-2 for predicting hospital disposition or severity for COVID-19 positive patients.
All statistical calculations were performed with XLSTAT software (version 2020.5.1, Addinsoft) and SavGIS software (version 9.07.015, Institut de Recherche pour le Développement, IRD).
Results
Study population
A total of 853 patients, chosen after initial triage, were included in the study population. Thirty-nine declined to take part in the study, thus 814 patients were included in the final analysis. Mean age was 46.8 years, with a M/F sex ratio of 0.5. Almost two-thirds (60%) of the patients visited the TMB to record their vital signs. The comorbidities of the patients included, arterial hypertension (HA; 18.9% of all patients), cardiovascular disease (Cardio; 6.8%), diabetes (10.0%) and chronic obstructive pulmonary disease (COPD; 1.6%). Nearly one-quarter (23.7%) of patients had at least one comorbidity (14.2% had a single comorbidity, 6.8% had two comorbidities, 2.6% had three comorbidities and 0.2% had four comorbidities).
COVID-19 pre-diagnosis
Within the context of the epidemic, COVID-19 was diagnosed clinically by the emergency doctor in two-thirds (n=543, 67%) of the 814 patients. A total of 428/814 patients were tested by RT-PCR, a pulmonary CT scan, or both. A positive diagnosis of COVID-19 was made in 246/428 (57.5%) tested patients. The demographic and clinical data for the study participants are summarised in Table 1.
Demographic and clinical data for the study population (TMB: telemedicine booth; COPD: chronic obstructive pulmonary disease; HA: arterial hypertension; Cardio: cardiovascular disease; SBP: systolic blood pressure; DBP: diastolic blood pressure; MAP: mean arterial blood pressure; BMI: body mass index).
NEWS-2 score ranged from 0 to 13 with a mean of 2.36, a bimodal distribution and a natural cut-off between 4 and 5 (Fig. 1).
Patient disposition and scores
After the initial triage, a second triage was carried out in the ED, assessing the vital signs and general condition of the patient, to determine the disposition of the patient: HOS or DH. The patient disposition chosen by the ED team served as the reference to evaluate the three scores (NEWS-2, qSOFA, CRB-65). Among the 814 patients, 114 (14.0%) were HOS and 700 (86.0%) were DH. Most of the variables examined differed significantly between the two groups (Table 2). The greatest difference between the two groups was “altered general condition” (p-value <10−6). Age and sex were also significantly different between the two groups but these variables are often considered as confounding factors (age nevertheless remains a risk factor for severity and death in the context of COVID-19 even after removing the comorbidities associated with it). Odds ratios were also calculated after adjusting for sex and age (>65-years).22,23
Comparison of patients admitted to hospital (HOS) and those who returned home (RH) (CI: Confidence Interval; TMB: Telemedicine Booth; COPD: chronic obstructive pulmonary disease; HA: arterial hypertension; Cardio: cardiovascular disease; SBP: systolic blood pressure; DBP: diastolic blood pressure; MAP: mean arterial blood pressure; BMI: body mass index; ED: emergency department. N/A: not applicable)
To evaluate NEWS-2 as an aid for the disposition of COVID-19 patients, the discriminatory power of the score for hospital admission was verified, evaluating the best cut-off value and analysing the patients who were misclassified by the score (Fig. 2). NEWS-2 with its optimum cut-off was then considered as a test for hospital admission, to measure its sensitivity and specificity. Among the 620 patients for whom we could calculate NEWS-2, 98 (15.8%) were HOS and 522 (84.2%) were DH. Even though it did not take into account comorbidities or ‘altered general condition’, NEWS-2 was significantly different between the two groups (mean 7.36 for HOS vs. 1.42 for DH; p-value <10−6). The sensitivity, specificity and predictive values for the different cut-off values of NEWS-2 are shown in Table 3. The cut-off value of 5 offered the best sensitivity (82%) and specificity (98%). This value minimised the number of patients that were misclassified (n=28) (17 false negatives: NEWS-2 score ≤5 but patients HOS; 11 false positives: NEWS-2 score >5 but patients DH).
Sensitivity, specificity, and predictive values for the different cut-off values for NEWS-2 in relation to Hospital admission (TP: true positive; FP: false positive; TN: true negative; FN: false negative; PVP: predictive value positive; PVN: predictive value negative; LR+: positive likelihood ratio = sensitivity / (1 − specificity); LR-: negative likelihood ratio = (1 − sensitivity) / (specificity)).
ROC curves of the three scores showed that NEWS-2 was the most predictive for hospital admission (AUC = 0.96) (Fig. 3).
In order to understand why 17 patients with NEWS-2 ≤5 were admitted to hospital by the ED, we investigated the differences between the HOS group with NEWS-2 = 4 or 5 (false negatives) and DH with NEWS-2 = 4 or 5 (true negatives). Age, anorexia and ‘altered general condition’ were the most discriminating factors for HOS that were not directly included in NEWS-2. In order to understand why 11 patients with a score >5 were not admitted to hospital, we investigated the differences between the DH group with NEWS-2 = 6 or 7 (false negatives) and the HOS group with NEWS-2 = 6 or 7 (true positives). Altogether, age and sex were significant factors for DH (in the absence of comorbidities, principally HA and diabetes). The presence of vomiting was not a criterion for HOS (Table 4).
Differences in variables between true and false negatives, NEWS-2 score of 4 or 5 and, between true and false positives, NEWS-2 score of 6 or 7 (COPD: chronic obstructive pulmonary disease; HA: arterial hypertension; Cardio: cardiovascular disease; SBP: systolic blood pressure; DBP: diastolic blood pressure; MAP: mean arterial blood pressure; BMI: body mass index; ED: emergency department; TP: true positive; FP: false positive; TN: true negative; FN: false negative; RH returned home; HOS: hospital admission).
When the different scores were compared, qSOFA did not give any benefit when compared to NEWS-2. Likewise, CRB-65 did not result in any improvement in the false negatives, but clearly differentiated the false positives and true positives for NEWS-2 (6 and 7) values (close to the best cut-off value of 5 for NEWS-2). For these two scores, the reliability of COVID-19 diagnosis seemed to play a role since the diagnosis (confirmed by PCR or CT scan) supported hospital admission of the patient when there was a contradiction between NEWS-2 and disposition (Table 4).
COVID-19 diagnosis
After the initial triage, almost all patients received a clinical diagnosis from the emergency doctors (799/814 patients) and some were tested by RT-PCR (222/814) or pulmonary CT scan (370/814). A COVID-19 diagnosis was therefore established: positive = RT-PCR positive, or if not, CT scan positive (confirmed diagnosis); negative = CT scan negative (confirmed diagnosis), and if no scan performed, diagnosis based on clinical signs (clinical diagnosis). We therefore obtained 491 positive and 314 negative patients. Few vital signs were discriminatory for COVID-19 diagnosis (apart from SpO2, pulse rate and ‘altered general condition’) (Table 5). Nearly 16% of positive clinical diagnoses were contradicted by a negative CT scan.
Comparison of clinical characteristics between the positive and negative groups for COVID-19 diagnostic (CI: confidence interval; TMB = telemedicine booth; COPD: chronic obstructive pulmonary disease; HA: arterial hypertension; Cardio: cardiovascular disease; SBP: systolic blood pressure; DBP: diastolic blood pressure; MAP: mean arterial blood pressure; BMI: body mass index).
COVID-19 diagnosis and NEWS-2
Although NEWS-2 was higher in COVID+ patients than in COVID-patients (mean 2.71 vs. 1.72, respectively), it was not a good predictive test for diagnosis, whatever the chosen cut-off value (Fig. 4). For example, out of 414 patients with NEWS-2 ≤2, 248 were diagnosed as COVID+ (60%) and of 528 patients with NEWS-2 ≤5, 327 were diagnosed as COVID+ (62%). The ROC curve of sensitivity vs. specificity of NEWS-2 had an AUC of 0.607 (Fig. 5).
Of 620 patients for whom we could calculate NEWS-2, 277 underwent a pulmonary CT scan. NEWS-2 reflected the degree of pulmonary involvement (Bravais-Pearson correlation coefficient r = 0.48). NEWS-2 was a good indicator of disease severity in COVID+ patients (Table 6).
NEWS-2 score means as a function of pulmonary Computed Tomography Scan result (Numbers in parenthesis indicate group sizes; COVID+ = positive with either RT-PCR, or CT scan positive; COVID-= negative with RT-PCR negative, CT scan negative, or negative diagnosis on clinical signs).
If we consider patients with NEWS-2 >5 (optimum cut-off value for disposition), only a few parameters were significantly different between COVID+ and COVID-patients: pulse pressure (p-value =0.02), temperature (p-value =0.00016) and degree of pulmonary involvement (p-value <10−6).
COVID diagnosis and disposition
Although HOS patients more often had a confirmed diagnosis than DH patients (78% vs. 60%, respectively), a diagnosis of COVID-19 was not a discriminating factor for disposition.
Improvement of NEWS-2
The question was how can NEWS-2 be improved in order to use it for either patient disposition or for the diagnosis of COVID-19 severity without losing its simplicity (rapid to perform and calculate, and easy to interpret) and without changing the nature of the score (i.e. a score based on a patient’s vital signs only). Using a cut-off of 5, NEWS-2 is already an excellent predictor of HOS since it concurred with 96% of dispositions made from our ED. However, it is a less good predictor of COVID-19 diagnosis, whatever the cut-off chosen. A Principal Component Analysis (PCA) with the six variables included in NEWS-2 (Table 7) indicates a first axis that explains 33% of the variance and gives the influence of the different components in this variance. All components are standardised for ease of interpretation. O2, SpO2 and respiratory rate were the most discriminating factors (Table 7).
Contribution of NEWS-2 score variables to patient orientation.
It may be possible to improve NEWS-2 by adding coefficients to the sub-scores, by eliminating non-discriminating components, or by adding new variables.
Multilinear or logistic modelling of ED disposition according to NEWS-2 sub-scores as regressors gave normalised coefficients indicating the major influence of the first three factors (O2, SpO2, respiratory rate sub-scores) (Table 8).
Normalized coefficients for linear and logistic models for NEWS-2 score.
The result of the linear model gave a highly discriminatory distribution. With a cut-off of 0.5, the model included 26 patients who were misclassified with respect to real disposition: 9 false positives (DH classified as HOS by regression) and 17 false negatives (HOS classified as DH by regression): compared to the initial NEWS-2 (with a cut-off of 5) the model did not show any significant improvement to predict real disposition from the ED.
Discriminant factor analysis of disposition according to the parameters used in NEWS-2 did not yield a better prediction than NEWS-2 itself, even when age and sex were added (42 misclassified: 30 false negative, 12 false positive) (Table 9).
Discriminant factor analysis: results.
Adding new variables
Some potential predictors not present in NEWS-2, including age, comorbidities (e.g. hypertension, coronary disease, stroke, diabetes) were considered in a multilinear regression model to supplement and improve NEWS-2 for the current situation, but the results were not conclusive. As observed, ‘altered condition’ was highly discriminatory, but did not really fit into the philosophy of a score that must be calculated from vital measurements only (recorded by the nursing staff or by a TMB) and not from an assessment of the patient’s condition that requires a physician’s judgment. In addition, biological parameters were not taken into consideration for NEWS-2 improvement as it was considered that the simplicity of NEWS-2 and its independence from laboratories and limited resources had to be preserved.
Discussion
The main finding of this study was that NEWS-2, used in the ED, was effective at predicting the risk of hospital admission in patients with suspected COVID-19. The study population was selected after an initial triage based on COVID-19 symptoms. NEWS-2 calculated at admission to the ED quickly and easily identified suspected COVID-19 patients at risk of clinical deterioration without medical intervention, proving its usefulness at increasing patient safety, management and monitoring. NEWS-2 >5 predicted hospital admission with 82% sensitivity and 98% specificity. Our data are consistent with recent publications evaluating NEWS-2 in COVID populations that demonstrate its effectiveness at predicting severe disease, in-hospital mortality and admission to an ICU.24-28 In our study, NEWS-2 appeared to be superior to the qSOFA and CRB-65 scores at predicting hospital admission of COVID-19 patients, as reported previously.27,29-32
Of the 814 patients included in this study, 13.7% were hospitalised, which is lower than the figures found in the literature (20%)33 and is one of the major limitations of the study. This can be explained by the fact that the patients self-referred and were generally younger (mean age 46.8-years). Thus, our conclusions on the interpretation of NEWS-2 refer to this population and are not representative of the overall hospitalised population.
Using NEWS-2 >5 as the criterion for disposition misclassified 25 patients. For patients with dyspnoea or acute pain, NEWS-2 was positively affected by pulse rate and respiratory rate. The number of DH patients with a high NEWS-2 could have been reduced by repeating the score after analgesic treatment or reassurance. Elderly patients were more often admitted to hospital despite a low NEWS-2, as their general condition (e.g. anorexia, vomiting, dehydration), degree of pulmonary damage and associated risk factors (e.g. diabetes, high blood pressure, stroke, immunosuppression, myocardial infarction history and chronic lung disease) seem to be important elements for the ED team deciding on hospital admission. Social isolation was another risk factor taken into account by ED practitioners. The addition of these parameters to NEWS-2 could have an influence on decisions for hospital admission, but would add complexity to data recording and processing.
The study population was selected using a clinical questionnaire at the ED entrance to reflect the day-to-day organisation and advanced triage of the emergency services in France during the pandemic. Hospitalising potentially severe COVID-19 cases using a standardised NEWS-2 rather than a COVID biological diagnosis has the advantage of taking into consideration other diseases as well. NEWS-2 could be used as a general tool for admitting seriously ill patients but also for monitoring COVID-19 patients at home. This score has been shown to be effective in both the primary and secondary healthcare settings in several studies.14, 35-36
NEWS-2, as its name illustrates, was conceived to detect signs of physiological failure and not as a specific diagnostic tool. Future research might take into consideration the association of NEWS-2 with other scores. To associate a generic score for physiological failure with other tools that are disease-aetiology specific might be extremely helpful. Such a two-step procedure would allow the early detection of patients requiring urgent medical attention while accurately orienting a patient pre-diagnosis.
An interesting perspective is the automatic calculation of NEWS-2 by a TMB integrating autonomous recording of vital signs and the collection of personal history (telemedical record). This allows early self-evaluation of patients and appropriate dispensation. It could be a powerful tool in hospital settings, but even more efficient if deployed at the pre-hospital level as part of a telemedical network. During an epidemic, it would allow massive, rapid, easy and partially health-personnel-free screening and follow-up of the general population (data recording) while reducing patients’ mobility. Minimal risk patients could be excluded from a hospital-based evaluation and high-risk patients could be detected promptly and transferred to the appropriate facilities.16, 36-41 This could help regulate the flow of patients to hospitals, predict hospital admission rates and prevent or reduce ED overcrowding. An additional benefit could come from improvements in data collection for epidemiological studies and public health planning.
Conclusion
NEWS-2 was a useful tool for the evaluation of patients with suspected COVID-19, helping medical practitioners to predict and make decisions on hospital admission. NEWS-2 as a criterion for hospital admission of COVID-19 patients was not improved by the addition of other patient variables, such as comorbidities or age. However, its association with other scores might improve and facilitate intra-hospital evaluation by adding sickness-specificity. The automatic calculation of NEWS-2 by a TMB provided with the appropriate devices and algorithms could be a major improvement in patient management, follow-up and screening, particularly in epidemic situations.
Data Availability
Database is available on request if confidentiality is ensured.
