Rapid assessment of COVID-19 mortality risk with GASS classifiers ================================================================= * Salvatore Greco * Alessandro Salatiello * Nicolò Fabbri * Fabrizio Riguzzi * Emanuele Locorotondo * Riccardo Spaggiari * Alfredo De Giorgi * Angelina Passaro ## ABSTRACT **BACKGROUND** Risk prediction scores and classification models are fundamental tools to effectively triage incoming COVID-19 patients. However, current triaging methods often have poor predictive performance, are based on variables that are expensive to measure, and lead to decisions that are sometimes hard to interpret. **OBJECTIVE** We introduce two new classification methods that are able to predict COVID-19 mortality risk from the automatic analysis of routine clinical variables with high accuracy and interpretability. The classifiers, denominated SVM22-GASS and Clinical-GASS, leverage machine learning methods and clinical expertise, respectively. **METHODS** Both classifiers were developed using a derivation cohort of 499 patients and were validated with an independent validation cohort of 250 patients. The cohorts included COVID-19 positive patients admitted to two hospitals in the Italian Province of Ferrara between March 2020 and June 2020 (derivation cohort) and between September 2020 and March 2021 (validation cohort). The potential predictive variables analyzed in this study included demographic, anamnestic, and laboratory data, retrieved with the patients’ consents from their electronic health records. The SVM22-GASS classifier is based on a Support Vector Machine model (SVM) with Radial Basis Function kernel (RBF). Importantly, the model uses only a subset of predictive variables that were automatically selected with the Least Absolute Shrinkage and Selection Operator (LASSO), while the RBF kernel is approximated with random feature expansions to reduce the computational requirements. The Clinical-GASS classifier is a threshold-based classifier that leverages the General Assessment of SARS-CoV-2 Severity (GASS) score: a highly interpretable COVID-19-specific clinical score that has been recently shown to be more effective at predicting the COVID-19 mortality risk than standard clinical scores. **RESULTS** The SVM22-GASS model was able to predict the mortality risk of the validation cohort with an Area Under the Receiver Operating Characteristic Curve (AUC) of 0.87 and an accuracy of 0.88 — performing on par with influential classification methods that exploit variables derived from expensive analyses such as medical imaging. Furthermore, variable importance analyses showed that the model relies primarily on eight variables for its predictions: White Blood Cell Count, Lymphocyte Count, Brain Natriuretic Peptide, Creatine Phosphokinase, Lactate Dehydrogenase, Fibrinogen, PaO2/FiO2 Ratio, and High-Sensitivity Troponin I. Similarly, the Clinical-GASS classifier predicted the mortality risk of the validation cohort with an AUC of 0.77 and an accuracy of 0.78 — on par with other established and emerging machine-learning-based methods. **CONCLUSIONS** Our results demonstrate that it is possible to accurately predict the COVID-19 mortality risk using only routine clinical variables that can be readily collected in the very early stages of hospital admission. The classifiers have the potential to assist the clinicians in quickly identifying the patients’ mortality risk to optimally allocate both human and financial resources. Keywords * COVID-19 * mortality risk prediction * SVM22-GASS classifier * Clinical-GASS classifier * GASS score * machine learning ## Introduction In late 2019, a new member of the coronavirus family, named Severe Acute Respiratory Syndrome CoronaVirus-2 (SARS-CoV-2), emerged in the Chinese province of Hubei[1] and rapidly spread worldwide, causing the first pandemic by a coronavirus. As of November 2021, the virus has caused more than 250 million cases of infection, and has led to over 5 million deaths in more than 200 countries [2] 2. COVID-19 (COronaVIrus Disease 19), the clinical manifestation of SARS-CoV-2 infection, is undoubtedly the most important global health concern this century has witnessed, and has been taking a heavy toll in terms of human, financial, and social resources. Moreover, the appearance of new SARS-CoV-2 strains and the lagging vaccination campaigns hinder the objective of reaching herd immunity, implying that such issues will likely persist in the coming months. Hospitals are particularly affected, especially during periods of increased disease transmission, where the number of patients that simultaneously need treatment increases exponentially. Such waves of patients can quickly deplete hospital resources if not dealt with optimally. In these circumstances, it is fundamental to estimate the amount of resources incoming COVID-19 patients will likely require during their hospital stay, ideally, in the early phases of hospital admission. This ensures that high-risk patients will have access to adequate resources and that, in general, hospital resources are not misdirected. General risk scores and comorbidity indices, such as the Charlson Comorbidity Index (CCI)[3] 3, have been shown to be helpful in estimating the prognosis and thus the level of care COVID-19 patients require[4,5] 4,5. Similarly, pneumonia-specific risk scores such as the CURB-6[6] 6 and the CURB-65[7] 7 scores have been shown to have even greater prognostic ability[8,9] 8,9. Nevertheless, both kinds of risk scores tend to underperform clinical risk scores that are specifically targeted to COVID-19 patients[10–12] 10-12. This is in part due to the remarkable variability across patients of the SARS-CoV-2 manifestations. The SARS-CoV-2 infection manifests itself heterogeneously: symptoms can range from being completely absent (up to 33% of total cases[13] 13) to being critical (up to 5% of total cases[14] 14), in which case they include respiratory failure, high fever, hyperinflammation, and multiorgan dysfunction. The percentage of critical cases increases substantially in the cohorts of hospitalized COVID-19 patients, reaching peaks close to 20%[15] 15; such percentages are particularly high in older patients with coexisting morbid conditions and cardiovascular diseases. To move beyond the limitations of general risk scores, in this work, we introduce two COVID-19-specific classifiers: the Clinical-GASS and the SVM22-GASS. Both classifiers identify high-risk patients by predicting their 30-day mortality outcome and were developed and validated using independent derivation and validation cohorts. The SVM22-GASS classifier is based on a Support Vector Machine model (SVM) and is built in a fully data-driven fashion exploiting state-of-the-art machine learning methods. The Clinical-GASS builds on a COVID-19 specific risk score we recently introduced and proved effective at stratifying the population of hospitalized COVID-19 patients: the General Assessment of SARS-CoV-2 Severity (GASS) score[10] 10. ## Materials and methods ### Study design and participant recruitment This is a prospective observational cohort study, developed in the two hospitals of Ferrara’s territory dedicated to COVID inpatients, “Arcispedale S.Anna” in Cona (Fe) and “ Ospedale del Delta” in Lagosanto (Fe). The province of Ferrara is geographically located in the Eastern part of the Emilia-Romagna region of Italy, with a population of approximately 350,000 inhabitants, and it is characterized by a high presence of elderly subjects (∼26% of total population is aged >65 years, and nearly 1% >90 years). This study analyzes the data of two independent cohorts of COVID-19 patients: a *derivation cohort* — used to develop the classifiers — and a *validation cohort* — used to validate them. In the machine learning community, such data are commonly referred to as training and test set, respectively. The derivation cohort comprises data of 499 patients recruited between March 2020 and June 2020, which have been already partially described and analyzed in our previous study[10] 10. The validation cohort comprises data of 450 patients recruited between September 2020 and March 2021; however, the data relative to 200/450 patients were not complete of all the relevant variables and were thus discarded. The collected variables include demographic, anamnestic and laboratory data. These were extracted with the patients consent from their electronic health records. Baseline symptoms and vital signs were added to their records and used to calculate both their CCI score (Charlson Comorbidity Index), and the GASS score. Information about the items considered by CCI and GASS scores can be found in the *Supplementary Table*. SARS-CoV-2 infection was confirmed with the reverse transcriptase-polymerase chain reaction (RT-PCR) test. The exclusion criteria were the negativity of the swab tests to viral detection and age (patients younger than 18 years were excluded). All patients signed an informed consent when possible; in case the patients were unable to sign, the consent was obtained from their legal representatives. ### Descriptive analyses We evaluated the differences between subjects in terms of the three major COVID-19 outcomes (1. IoC=intensification of care, meant as the need for non-invasive mechanical ventilation or for endotracheal intubation; 2. in-hospital death; 3. 30-day death). The observation period was protracted until the 30th day since hospital admission for those patients who survived the hospitalization: this was possible using records of the local registry office linked to the hospital informatic system. Patients needing IoC can be recognized by the acronym “IoCp” while, for those who did not undergo IoC, we chose the acronym “nIoCp”. The groups of patients who survived or died after a period of observation of 30 days are presented with the acronyms “30-ddp” (30-day deceased patients) and “30-dsp” (30-day survived patients), respectively. ### Statistical analysis Data analyses were performed by using SPSS 26.0 software (IBM SPSS Statistics, IBM Corporation) and MATLAB (MATLAB 2020a, The MathWorks, Natick, MA). The normal distribution of the continuous variables was analyzed using Kolmogorov-Smirnov and Shapiro-Wilk tests. Variables not normally distributed were log-transformed before entering the parametric statistical analysis. Categorical variables were summarized by using frequencies and percentages, and continuous data were presented as median (interquartile range, IQR). The Mann-Whitney U test was used for continuous variables, while the χ2 test was used for categorical variables. Variables with a *p* value <0.05 in the univariate analyses were used to perform multivariate logistic regression analyses. All *p* values <0.05 are considered statistically significant. ### The Clinical-GASS classifier In our previous study[10] 10, we discussed the ability of the GASS score to stratify the population of hospitalized COVID-19 patients into groups with significantly different 30-day mortality risk. Such a stratification has the potential to improve patient care by signaling to the hospital personnel the patients who are likely to need a mild (GASS < 6), moderate (6 ≤ GASS ≤ 10) and high (GASS>10) level of care. However, clinical personnel might also benefit from knowing whether the patient at hand has a low or a high risk of dying, and is likely to require specialized treatment such as the admission to the intensive care unit. This can be accomplished by building a *binary classifier* that predicts, for example, the 30-day mortality outcome. This is the primary outcome measure we aim to predict with this and the other classifiers considered in the following sections. To assess the potential of the GASS score to be used for such a binary classification task, we built a simple threshold-based classifier. The classifier — named Clinical-GASS — was designed by simply computing the optimal Receiver Operating Characteristic (ROC) threshold on the training set, in terms of the cost/benefit ratio[16] 16; equal costs were assumed for the misclassification of survivors and non-survivors. Patients with a GASS score below such a threshold are classified as likely survivors while patients with a GASS score above such a threshold are considered as likely non-survivors. A similar approach was used to build the Charlson Comorbidity Index (CCI) classifier from the CCI score[3] 3. The CCI is a clinical score that is often used as a baseline method to estimate the mortality risk in the general population. ### The SVM22-GASS classifier Both the Clinical-GASS and the CCI classifiers are built leveraging medical expertise and thus might have a strong appeal to clinicians: they are likely to have full understanding of how the classifiers work, to trust their decisions and to thus be willing to use them. However, both classifiers only use a limited number of features (Clinical-GASS: n=11, CCI classifier: n=19) and combine them with only sums and other simple operations defined by piecewise constant functions. For these reasons, neither classifier might be able to fully exploit useful features contained in the patients’ health records, and to capture the non-linear interactions between the multitude of variables potentially at play in determining a patient’s survival outcome. To address this issue, we designed a Support-Vector-Machine-based classifier (SVM) with Radial Basis Function (RBF) kernel[17] 17. An RBF-SVM is a binary non-linear classification method that classifies data by non-linearly mapping the feature vectors into an infinite-dimensional space in which data from different groups can be easily separated by a hyperplane. In brief, data *x* are classified by the decision function *f*(*x*), which is given by ![Formula][1] Here, *N* is the number of support vectors, *α**i* are weighting coefficients, and *k*(*y, z*) is the kernel function. In our case, we chose the Gaussian Radial Basis Function: ![Formula][2] The SVM classifier was trained using only the data contained in the *training set*, was tested on the independent *test set*, and was compared against the CCI and GASS classifiers. #### Data preprocessing Before training the SVM classifier, we made sure to have sound training data. This was accomplished in two steps. First, we excluded from the analysis the variables that were measured in less than 50% of the patients. Secondly, we imputed the remaining missing values with the weighted average of the values of the three most similar instances, with weights inversely proportional to the distances from these. As a distance measure, we used the Euclidean distance for continuous features and the Hamming distance for binary ones. #### Data augmentation Learning a classifier from imbalanced training data such as ours (% survivors = 75.7 %) is a challenging task and can lead to poor sensitivity to the minority class. To deal with this issue, we balanced the training set using the Synthetic Minority Over-sampling Technique (SMOTE[18] 18). In brief, this method works by generating synthetic instances of the minority class by perturbing the minority instances in the directions of the difference vectors from their nearest neighbours of the same class. #### Feature selection To remove potentially redundant and uninformative features, while reducing the computational cost of training the SVM model, we performed feature selection; this was accomplished by learning a regularized logistic regression model with LASSO penalty[19,20] 19,20. The regularization parameter λLASSO, which determines the regularization strength and thus the number of selected features, was chosen by minimizing the ten-fold cross-validation deviance. #### Parameter training To learn the model’s parameters, we minimized the hinge loss using the Limited-memory Broyden-Fletcher-Goldfarb-Shanno (LBFGS) solver[21] 21. Importantly, to speed up training and reduce memory requirements, we approximated the Gaussian kernel using the Fastfood[22] 22 random feature expansion method. To deal with potential residual classification biases due to the class imbalance, we adjusted the SVM classification threshold[23] 23. Specifically, we used the SVM classification scores to compute the optimal operating point of the ROC curve and used the resulting threshold to classify the data. #### Hyperparameters optimization Before learning the final model’s parameters, to preselect a class of SVM models suitable for the dataset at hand, we performed hyperparameter optimization[24] 24 using Bayesian optimization[25] 25. Specifically, with this procedure we found the kernel scale s2, the regularization strength λSVM, and the dimensionality of the random features space d, that minimize the five-fold cross-validation hinge loss. #### Model evaluation To evaluate the quality of the model’s predictions, we computed Area Under the ROC Curve (AUC), accuracy, sensitivity, and specificity on the independent test set. The SVM’s performance is then compared to that of the other classifiers of interest, namely, the CCI classifier and the GASS classifier. As a baseline model, we also considered the majority classifier, which assigns the majority class to every instance in the dataset, regardless of its features. #### Model interpretation As it is often the case with most high-dimensional and non-linear machine learning models, also in our case, once the SVM model is fully trained, it is challenging to gain an intuitive understanding of the mechanism underlying the classifier’s decision process. To shed some light on such a mechanism, we estimated the variable importance (VI)[26] 26 of each input feature of the SVM model, by computing the *model reliance*. In brief, such method assigns high reliance to the features that, when perturbed. lead to a strong decrease in classification accuracy. Furthermore, to assess the discriminative power of the most important features selected with this method, we trained a reduced SVM model to classify survivors and non-survivors using only the top-3 features[27] 27 and measured the decrease in classification performance. ### Guidelines and Ethical approval For the compilation of this manuscript, we followed STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) guidelines for reporting observational studies. The local Ethics Committee “Comitato Etico Indipendente di Area Vasta Emilia Centro (CE-AVEC)” approved the protocol of this study; the protocol code is 712/2020/Oss/AOUFe. ## Results ### Descriptive analyses The distribution of males and females in the overall population was quite homogeneous (54.8% males vs. 45.2% females), while in the group of patients who underwent intensification of care (IoCp) there was a higher percentage of males (65.8% vs. 34.2%; *p*=0.021). The median age of the overall population was 72 years (IQR 58-82 years) and in-hospital mortality was recorded for 62 patients (24.8%) The subjects who died within the hospitalization period (deceased) were significantly older than those who were discharged (median age 82 vs. 67 years; *p*<0.001), as well as it happened for the group of patients who died within 30 days (30-ddp) compared with those who survived within the same period (30-dsp) (median age 81 vs. 68 years; *p*<0.001). Not surprisingly, we found differences between the IoCp and NIoCp groups in terms of length of stay (21 vs. 10 days; *p*<0.001); significant differences were also found between from the 30-ddp and 30-dsp groups (9 days vs. 13 days; *p*<0.001). The GASS scores were higher in the IoCp group compared to NIoCp (9 vs. 7 points; p<0.001); as expected, similar differences were found between deceased and discharged patients (11 vs. 6 points; *p*<0.001) and also between 30-ddp and 30-dsp (11 vs. 7 points; *p*<0.001). The patients with worse disease outcomes presented with a greater load of comorbidities (CCI 3.3±2.7 vs. 1.6±2.3 points, *p*<0.001 between deceased and discharged and 3.0±2.6 vs. 1.7±2.4 points, *p*=0.001 between 30-ddp and 30-dsp). As for vital parameters, IoCp and NIoCp groups differed in terms of respiratory rate, RR (22 bpm, IQR 20-30 vs. 22 bpm, IQR 16-22; *p*<0.001) and PaO2/FiO2 ratio at the first visit (300, IQR 230-352 vs. 229, IQR 155-295; *p*<0.001). Deceased and discharged patients differed in terms of systolic blood pressure (SBP), diastolic blood pressure (DBP) and RR; similarly, 30-ddp and 30-dsp differed for SBP, DBP and RR. The laboratory findings were also heterogeneously distributed between groups: IoCp and NIoCp differed for serum C Reactive Protein (CRP), procalcitonin, D-Dimer, isoamylase, ALT, LDH and ferritin. Differences between deceased and discharged patients concerned serum lymphocyte count, creatinine, CRP, procalcitonin, D-Dimer, LDH, BNP and high sensitivity troponin I (HS TnI). Similar differences were also found between 30-ddp and 30-dsp: these groups differed for serum creatinine, CRP, procalcitonin, D-Dimer (1.15 mg/L FEU, IQR 0.76-2.00 vs. 0.88 mg/L FEU, IQR 0.48-2.04; *p*=0.044), LDH, BNP and HS TnI (42 ng/ml, IQR 19-101 vs. 9 ng/ml, IQR 4-22; *p*<0.001). All data concerning the characteristics of the studied population can be found in Table 1; extended results were reported only for those variables confirming their importance in the multivariate analyses later performed. View this table: [Table 1.](http://medrxiv.org/content/early/2021/12/08/2021.12.07.21267425/T1) Table 1. Characteristics of population and differences between groups. Data are reported as median (interquartile range IQR) unless otherwise specified; p, p value between groups; IoCp, patients who underwent intensification of care; nIoCp, patients who did not undergo intensification of care; 30-ddp, patients deceased after 30 days; 30-dsp, patients survived after 30 days; GASS, General Assessment of SARS-CoV-2 patientS; SD, Standard Deviation; CCI, Charlson Comorbidity Index; SBP, Systolic Blood Pressure; DBP, Diastolic Blood Pressure; HR, Hart Rate; RR, Respiratory Rate; WBC, White Blood Cells; CRP, C Reactive Protein; ALT, Alanine Transferase; CPK, Creatine Phosphokinase; LDH, Lactic Dehydrogenase; BNP, Brain Natriuretic Peptide; HS TnI, High Sensitivity Troponin I. Differences between groups were also found in terms of comorbidities, and all the results are summarized in Table 2: patients of the IoCp group were more often smoker; in the group of deceased subjects we found more diagnosis of hypertension, ischemic heart disease, heart failure, chronic kidney disease, history of stroke or transient ischemic attack (TIA), peripheral arterial disease (PAD), chronic obstructive pulmonary disease (COPD), localized or haematological cancer, dementia and diabetes with organ damage. View this table: [Table 2.](http://medrxiv.org/content/early/2021/12/08/2021.12.07.21267425/T2) Table 2. Comorbidities and differences between groups. Data are reported as number of subjects (percentage). p, p value between groups; IoCp, patients who underwent intensification of care; nIoCp, patients who did not undergo intensification of care; 30-ddp, patients deceased after 30 days; 30-dsp, patients survived after 30 days; TIA, Transient Ischemic Attack; PAD, Peripheral Arterial Disease; COPD, Chronic Obstructive Pulmonary DiseaseThe subjects from the 30-ddp group presented in their clinical history more often hypertension, chronic kidney disease, stroke/TIA, COPD, localized or haematological cancer and dementia. Comparison analyses between groups were also performed for each item considered in the Clinical-GASS score and they can be found in the *Supplementary Table*. All variables found to be significantly different between groups in the univariate analyses were entered into multivariate analyses. In the logistic regression analyses concerning the need for intensification of care, the strongest predictive variable was the PaO2/FiO2 ratio: both ratios <100 and 100-199 showed to be somehow protective towards the intensification of care (OR 0.11, 95% CI 0.02-0.72; *p*=0.021 and OR 0.07, 95% CI 0.02-0.27; *p*<0.001, respectively), while the male sex and a higher respiratory rate (>30 breaths per minute) seemed to be predictive for a higher need for intensification of care (OR 3.56, 95% CI 1.20-10.56; *p*=0.022 and OR 4.76, 95% CI 1.00-22.96; *p*=0.050). Concerning the in-hospital mortality, age and PaO2/FiO2 ratio >300 were the only variables able to adequately predict this outcome (OR 1.07, 95% CI 1.02-1.12; *p*=0.007 and OR 0.35, 95% CI 0.13-0.92; *p*=0.034, respectively); moreover, age was particularly able at predicting death within a period of 30 days (OR 1.06, 95% CI 1.01-1.12; *p*=0.027), together with serum D-dimer between 0.5 and 2.0 mg/L FEU (OR 4.22, 95% CI 1.28-13.90; *p*=0.018); serum troponin I (HS TnI) < 20 ng/ml was, instead, protective towards 30-day death (OR 0.23, 95% CI 0.06-0.80; *p*=0.022). The forest plots concerning the regression analyses are reported in figure 1. ![Figure 1.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/12/08/2021.12.07.21267425/F1.medium.gif) [Figure 1.](http://medrxiv.org/content/early/2021/12/08/2021.12.07.21267425/F1) Figure 1. Forest plots of Regression Models. **(A)** Logistic regression modelling for identifying the variables associated with the need for intensification of care. **(B)** Logistic regression modelling for identifying the variables associated with in-hospital mortality. **(C)** Logistic regression modelling for identifying the variables associated with 30-day mortality. The Clinical-GASS score was calculated for each patient, and this allowed us to predict the relative risk of in-hospital and 30-day death based on the analyses of this cohort of subjects. We developed an open access web tool in order to help clinicians identify the patients at higher risk of bad outcomes by COVID-19, simply filling out the form with all the variables needed. This tool can be found at the following link: [https://ml.unife.it/GASS.html](https://ml.unife.it/GASS.html) and it is available for free consultation. Furthermore, we performed Receiver Operating Characteristic (ROC) curve analyses, in order to evaluate the predictive power of the GASS score, compared to the Charlson Comorbidity Index, which has been also recently tested in COVID-19 inpatients[28] 28. The main results of the classification analysis are reported in Figure 2. ![Figure 2.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/12/08/2021.12.07.21267425/F2.medium.gif) [Figure 2.](http://medrxiv.org/content/early/2021/12/08/2021.12.07.21267425/F2) Figure 2. Summary prediction performance. **(A)** Receiver Operating Characteristic (ROC) curves of the considered classifiers, computed on the training set (majority classifier: black; Charlson Comorbidity Index (CCI) classifier: grey; GASS classifier: orange; SVM classifier: blue). Filled circles indicate the optimal operating points. Note that the classification threshold of the majority classifier is fixed; thus, the corresponding ROC curve collapses to a single point. **(B)** Corresponding curves computed on the independent test set. Filled circles represent the optimal operating points of the training ROC curves. **(C)** Considered classification performance measures, computed on the training set; values are rounded to the nearest hundredths; AUC: Area Under the ROC Curve. **(D)** Corresponding measures computed on the test set. Note that the majority class in both the training and test set is the survival class, which was arbitrarily associated to a negative test result; therefore, by always predicting survival, the majority classifier has a sensitivity of 0 and a specificity of 1. ### The CCI classifier The optimal ROC threshold for the Charlson Comorbidity Index classifier was 11. This means that the optimal way to classify COVID-19 patients based on the CCI alone is to consider as high-risk all the patients with a CCI greater than or equal to this value. However, this classifier performed poorly on the test set, with an Area Under the Curve (AUC) of 0.66 and an accuracy of 0.76 (Figure 2D, grey bars). The poor performance of this classifier is attributable to the complete inability to correctly identify non-survivors (sensitivity=0), which makes it effectively equivalent to the naïve majority classifier (black bars). ### The Clinical-GASS classifier The optimal ROC threshold for the Clinical-GASS classifier was 13, which lies, as expected, within the range of values of the high mortality risk class (GASS >10). Overall, the Clinical-GASS displayed satisfying test performance, with an AUC of 0.77 and an accuracy of 0.78 (Figure 2D, orange bars). Importantly, the Clinical-GASS classifier performed better than both the majority and the CCI classifiers, due to its improved ability to identify non-survivors (sensitivity=0.31). ### The SVM22-GASS classifier The minimum-deviance ten-fold cross-validation regularization strength λLASSO was equal to 4.9·10−3. The corresponding LASSO logistic regression model fitted with such a parameter selected a total of 38 features, which are reported in Figure 2A. The best cross-validated hyperparameters of the RBF-SVM classifier trained using such features were s2 = 774.16, λSVM = 1.4·10−6, and d = 7318. The classifier trained with such hyperparameters achieved nearly optimal performance on the training set (AUC ≈ 1, accuracy ≈ 1, Figure 2C – blue bars), which means that the decision hyperplane is able to completely separate survivors from non-survivors in the transformed feature space. Importantly, the SVM classifier achieved very good performance also on the independent test set, with an AUC of 0.87 and an accuracy of 0.88. The ability to classify COVID-19 patients of the SVM classifier is thus superior to that of all the other considered classifiers. Such an improvement is largely attributable to a markedly better ability to correctly identify non-survivors (sensitivity=0.61). #### Model interpretation and reduced models The SVM classifier just described predicts the most likely 30-day mortality outcome of COVID-19 patients by non-linearly processing a selection of 38 features that characterize them. To understand whether some of these features are more important than others, we computed the model reliance[16]. The results of this analysis are reported in Figure 3A, where we plotted the model reliance of the full SVM model, on all the 38 features. The features are sorted from top to bottom in decreasing order of reliance. According to this analysis, 8 of the 38 features appear to be particularly informative to the model, as its predictions change drastically when such features are perturbed. Specifically, the eight most important features to the full SVM classifier are White Blood Cell Count (WBC), Lymphocyte Count (LYM), Brain Natriuretic Peptide (BNP), Creatine Phosphokinase (CPK), Lactate Dehydrogenase (LDH), Fibrinogen (FIBR), PaO2/FiO2 Ratio (PFR), and High-Sensitivity Troponin I (TnI). ![Figure 3.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/12/08/2021.12.07.21267425/F3.medium.gif) [Figure 3.](http://medrxiv.org/content/early/2021/12/08/2021.12.07.21267425/F3) Figure 3. Understanding the full SVM model. **(A)** Model reliance of the full SVM model on all the 38 features its predictions are based on. Features are sorted (from top to bottom) in descending order of importance. **(B)** Medians, 95% bootstrap confidence intervals, and interquartile ranges of the top eight features, for the 30-day survival (green) and death (red) group. Black dots represent medians; boxes heights represent confidence intervals; thin lines represent interquartile ranges. Asterisks indicate significant (Bonferroni-adjusted *p* value ≤0.05) differences between groups (Mann–Whitney U test). To understand how such features, differ between survivors and non-survivors, and whether such differences are statistically significant, we performed Mann–Whitney U tests, using a Bonferroni-adjusted significance value of 0.05 (Figure 2B). Out of the 8 most important features, only 2 do not appear to be significantly different between survivors and non-survivors (CPK and fibrinogen, p value >0.05). Medians, 95% bootstrap confidence intervals, and interquartile ranges of the top eight features, for the 30-day survival (green) and death (red) group can be found in the Figure 3, panel B. The fact that the full SVM model appears to rely mostly on a subset of the full feature set of 38 features, raises the question of whether a reduced model, which has only access to the most informative features, can also accurately predict the COVID-19 outcome. To address this question, we trained a reduced RBF-SVM model to predict the COVID-19 outcome using only the three most informative features, that is White Blood Cell Count (WBC), Lymphocyte Count (LYM), and Brain Natriuretic Peptide (BNP). Choosing exactly three features allows us to visualize the entire feature space in which the classifier operates, and to plot the resulting decision surface. As an additional baseline, we also trained another RBF-SVM model that had access to only the three least informative features [namely, Creatinine (CRE), Procalcitonin (PCT), and D-dimer (XDP)]. The models, which we refer to as SVM-TOP3 and SVM-BOT3 respectively, were trained following the same steps adopted for the full SVM model. The results of this analysis are reported in Figure 4. As expected, both reduced models exhibited worse training and test performance than the full SVM model (Figure 4B, 4C). Nevertheless, while the SVM-BOT3 classifier exhibited overall poor test performance, comparable to that of the CCI classifier (AUC = 0.7, accuracy = 0.7 — Figure 4C), the SVM-TOP3 classifier performed surprisingly well. As a matter of fact, the SVM-TOP3 had better test performance than the Clinical-GASS classifier (AUC = 0.83, accuracy = 0.83 — Figure 4C), despite using only three features. ![Figure 4.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/12/08/2021.12.07.21267425/F4.medium.gif) [Figure 4.](http://medrxiv.org/content/early/2021/12/08/2021.12.07.21267425/F4) Figure 4. Reduced models and decision surface. **(A)** Decision surface of the SVM-TOP3 model, which only uses the top-3 features. The surface divides the feature space into two regions: a survival region (green) and a death region (red). Small green circles represent 30-day survivors, while small red circles represent 30-day non-survivors; misclassified instances are marked with a black dot. The corresponding large circles represent the group medians. Note: to facilitate the visualization of the decision surface, we are only plotting the data within a cuboid delimited by the 20th and 80th percentiles. **(B)** Classification performance measures of the full SVM model and of the reduced SVM models, computed on the training set. The SVM-TOP3 model only uses the top-3 features, while the SVM-BOT3 model only uses the bottom-3 features; values are rounded to the nearest hundredths. **(C)** Corresponding measures computed on the test set. The resulting decision surface (Figure 4A) is, as expected, highly non-linear. The 3D plot of the feature space confirms that, generally, non-survivors seem to have higher serum BNP and lymphocyte count. Interestingly, the white blood cells count appears to become strongly predictive of a non-survival outcome, only for very high values (WBC>10·103 n/mmc), especially when associated with high values of lymphocytes (LYM>9 · 102 n/mmc). ## Discussion Predicting the mortality risk of hospitalized COVID-19 patients could go a long way to optimally allocate hospital resources. Popular clinical scores are potentially useful in this regard, but have proven to be less accurate than COVID-19-specific methods[10–12] 10-12. In this work we have introduced and validated two new methods for predicting 30-day mortality risk: the Clinical-GASS and the SVM22-GASS. Overall, our results show that both methods are reliable and can thus be used to effectively triage incoming COVID-19 patients using only readily available variables. ### Clinical-GASS score and risk factors The Clinical-GASS score is based on 11 variables easily available after the first visit at the Emergency Room, with the execution of both a venous and an arterial blood sample. The need for reliable and easy-to-use tools, able to predict the outcomes in COVID-19 inpatients is day by day more urgent because of the scarceness of human and financial resources, already under strain from the beginning of the pandemic. The retrospective analyses from the first 499 patients hospitalized with COVID-19 to our hospitals in the territory of Ferrara, allowed us to notice a strong relationship between the Clinical-GASS score and mortality by COVID-19 (both in-hospital and 30-day rates); moreover, a slight relationship with the need for intensification of care was observed and confirmed only in the population with a Clinical-GASS score lower than 10 points; in the current study, this was true only for extreme values of the score (<5 or >10 points). In our cohort of patients, the in-hospital mortality rate was 24.8%, while on the 30th day of observation, it was 23.6%. Our findings showed how patients who underwent intensification of care, needing non-invasive mechanical ventilation or endotracheal intubation, were more often males, with a greater Clinical-GASS score (*p*<0.001) and worse respiratory performances (higher respiratory rates and lower PaO2/FiO2 ratios). As for laboratory abnormalities, those patients had higher serum inflammatory markers (CRP and ferritin), higher organ damage markers (isoamylase, ALT, LDH, HS TnI), and marked pro-coagulative status (higher serum D-Dimer). Such findings are consistent with previous studies: male sex has been already linked to poor chances of survival from COVID-19 and with increased risk of admission to the Intensive Care Units (ICUs)[29] 29; similarly, PaO2/FiO2 ratio was already shown to be independently associated with worse COVID-19 outcomes[30] 30. Furthermore, recent studies showed how COVID-19 inpatients present commonly with laboratory abnormalities and the role of inflammatory, and organ-damage markers has been also consistently reported[31,32] 31,32. Our analyses also confirmed the role of age. Early Chinese studies showed how the elderly population was bound to encounter worse COVID-19 outcomes[33] 33, and such findings were later confirmed in studies with European[34,35] 34,35 and American patients[36] 36. Interestingly, the laboratory findings and respiratory performances reported in these studies were similar to those of our IoC group. Additionally, in accordance with previous studies that showed how low levels of blood pressure at hospital admission are often associated with worse COVID-19 outcomes[37] 37, we observed higher blood pressure (both systolic and diastolic) in the survivor group. Finally, the negative role of comorbidities (especially cardiovascular) and smoke[38] 38 is also well established[39,40] 39,40. ### Alternative clinical scores Popular clinical scores of general applicability, such as the Charlson Comorbidity Score (CCI)[4] 41, the modified Elixhauser Index (mEI)[41] 42, the National Early Warning Score 2 (NEWS2)[42] 43 and the CURB models[8,9] 8,9 have been used during the current pandemic to predict COVID-19 outcomes with discrete success. However, they tend to underperform COVID-19-specific clinical scores[10,11]10-12. The first COVID-19-specific scores were developed in China[43,44] 44,45, uring the first wave of the pandemic. Further scores for prediction of COVID outcomes were later developed all over the world[12,45,46] 12,46,47, and data from more heterogeneous datasets were collected in order to reduce potential selection bias stemming from sampling from a restricted population of subjects. A case in point is represented by the 4C score, developed by Knight et al.[12] 12: such a score has quickly become popular due to its vast derivation and validation cohorts and its ease of use. However, the predictive performance of the score (validation AUC = 0.77), appears to lag behind the one of SVM22-GASS introduced in this work (validation AUC = 0.87, accuracy = .88), and to be comparable with the one of the Clinical-GASS (validation AUC = 0.77, accuracy = 0.78). Similar arguments can be made with regard to the recently developed “Piacenza score” (validation AUC = 0.78, accuracy = 0.55). Importantly, the SVM22-GASS bases its prediction on the analysis of clinical variables that can be quickly and inexpensively retrieved early on during hospital admission. This is a significant advantage over other popular multivariate methods that rely on the manual analysis of medical imaging results (e.g.,[11] 11), and yet achieve comparable predictive performance (validation AUC = 0.88). Like most machine-learning-based classifiers, the SVM22-GASS, the 4C and the Piacenza scores are purely data-driven. This makes them carry an intrinsic limitation represented by the fact that often it can be hard to give a “clinical” sense to the weight assigned to each feature considered in the development of the score. For this reason, in our first work, we decided to develop the GASS score, an informatic tool with the ability to accurately predict COVID-19 outcomes while keeping a clinical sense. The 11 features were chosen, in fact, based on both the strength of the associations with the outcomes and the clinical importance established for each of them in the current literature. ### Interpreting the SVM22-GASS classifier The SVM22-GASS predicts the 30-day mortality outcome by nonlinearly processing 38 patient features. Our variable importance analysis isolated eight variables as the most important for the model: White Blood Cell Count (WBC), Lymphocyte Count (LYM), Brain Natriuretic Peptide (BNP), Creatine Phosphokinase (CPK), Lactate Dehydrogenase (LDH), Fibrinogen (FIBR), PaO2/FiO2 Ratio, and (PFR), and High-Sensitivity Troponin I (TnI). Interestingly, half of them, (namely, LYM, BNP, PFR, and TnI) are also part of the Clinical-GASS score, which confirms their strong information content and predictive power. Consistently, serum BNP has been recently found to be significantly elevated in critically ill COVID patients in a recent meta-analysis[47] 48; whether or not this peptide can help discriminate high-risk COVID-19 patients remains unclear and it merits further investigation. Similarly, high WBC values, high LDH values[15] 15, and low LYM values[15] 15, and have also been recently linked to high mortality risk. On the other hand, variables like age, sex, respiratory rate and serum D-Dimer (XDP) do not appear to influence significantly the predictions of the SVM22-GASS classifier. This finding might appear surprising, as their association with poor COVID-19 outcome is well established. For example, age is among the predominant risk factors for developing the severe form of the disease, due to the immuno-senescence and all the physiological modifications related to it[33] 33. Similarly, the serum levels of D-Dimer (XDP) were found to be strictly associated with COVID mortality[48] 49. Furthermore, the respiratory rate is a useful indicator of potential respiratory disfunctions. Nevertheless, the fact that the SVM22-GASS does not rely on such variables, does not mean that they provide no information about the mortality risk; it merely means that other variables, those that carry greater weight, are more informative and/or less noisy. To validate the results of the variable importance analysis, we trained two additional classifiers: the SVM-TOP3 — which had only access to the three most important features White Blood Cell Count (WBC), Lymphocyte Count (LYM), Brain Natriuretic Peptide (BNP) — and the SVM-BOT3 — which had only access to the three least important features. Interestingly, we observed only a moderate performance decrease of the SVM-TOP3 with respect to the full SVM22-GASS model (AUC: 0.83 vs. 0.87, accuracy: 0.83 vs. 0.88). This suggests that SVM-TOP3 is also a competent predictor and can be used in those cases in which one does not have access to all of the 38 variables for a given patient. ### Potential Limitations In this prospective cohort study, we used independent derivation and validation cohorts to develop and validate our COVID-19-specific classifiers and risk score. However, both cohorts were recruited from the same two hospitals in the Italian Province of Ferrara and included only participants of Caucasian ethnicity. This might lead to overestimating the performance of our methods. Additionally, the sample size we used, albeit comparable to that of many other related studies (e.g., see[46] 47), is still too small to allow to draw any final conclusion concerning the generalization ability of our approaches. To deal with these issues, future work will focus on undertaking a larger and multicentered study to validate our approaches on a larger and more heterogeneous cohort. Furthermore, in the current study we did not use information about medical treatments, as this was missing. This implies that we cannot determine whether and how drug prescriptions altered the prognostic trajectories of our patients. ## Conclusions This work introduced two reliable classification approaches for the rapid assessment of mortality risk of COVID-19 patients. Importantly, both approaches rely on the automatic analysis of only routine clinical variables that can be readily acquired during hospital admission. The classifiers were developed and validated with independent derivation and validation cohorts and exhibited classification performance comparable to that of popular approaches that leverage information obtained with expensive medical imaging methods (e.g., chest radiography[11] – 11). Our results prove that the classifiers have the potential to facilitate the triaging of incoming COVID-19 patients, thereby optimizing the allocation of hospital resources. Future work will further validate the proposed methods using larger and more heterogeneous validation cohorts, and will focus on designing suitable strategies to assess them in hospital settings. ## Supporting information Supplementary Table [[supplements/267425_file06.docx]](pending:yes) ## Data Availability All data produced in the present study are available upon reasonable request to the authors ## Conflicts of Interest The authors declare no conflicts of interest ## Acknowledgements Special thanks to the directors of the medical departments where our research took place: Giovanni Zuliani MD, PhD director of the Department of University Internal Medicine in the “Arcispedale S.Anna” in Cona (Fe), Roberto Manfredini MD, PhD, director of the Department of Clinical Medicine in the “Arcispedale S.Anna” in Cona (Fe) and Stefano Parini MD, director of the Department of Internal Medicine in the “Ospedale del Delta” in Lagosanto (Fe). We would like to dedicate our sincere thanks to all the members of the medical staff that is still participating in the fight against the virus, to all the extremely skillful and valuable nursing staff members as well as our assistance personnel, every day working on the frontline. Finally, the authors thank the International Max Planck Research School for Intelligent Systems (IMPRS-IS) for supporting AS. * Received December 7, 2021. * Revision received December 7, 2021. * Accepted December 8, 2021. * © 2021, Posted by Cold Spring Harbor Laboratory The copyright holder for this pre-print is the author. All rights reserved. The material may not be redistributed, re-used or adapted without the author's permission. ## References 1. The-nCoV Outbreak Joint Field Epidemiology Investigation Team, Li Q. An Outbreak of NCIP (2019-nCoV) Infection in China - Wuhan, Hubei Province, 2019-2020. China CDC Wkly 2020;2:79–80. doi:10.46234/ccdcw2020.022 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.46234/ccdcw2020.022&link_type=DOI) 2. Worldometers.info. Worldometer COVID-19 Data. Dover, Delaware, U.S.A. 2020.[https://www.worldometers.info/coronavirus/country/malaysia/](https://www.worldometers.info/coronavirus/country/malaysia/) 3. Charlson ME, Pompei P, Ales KL, et al. Charlson comorbidity index. J. Chronic Dis. 1987. doi:0021-9681/87 4. Tuty Kuswardhani RA, Henrina J, Pranata R, et al. Charlson comorbidity index and a composite of poor outcomes in COVID-19 patients: A systematic review and meta-analysis. Diabetes Metab Syndr Clin Res Rev 2020;14:2103–9. doi:10.1016/j.dsx.2020.10.022 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.dsx.2020.10.022&link_type=DOI) 5. Christensen DM, Strange JE, Gislason G, et al. Charlson Comorbidity Index Score and Risk of Severe Outcome and Death in Danish COVID-19 Patients. J Gen Intern Med 2020;35:2801–3. doi:10.1007/s11606-020-05991-z [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s11606-020-05991-z&link_type=DOI) 6. Neill AM, Martin IR, Weir R, et al. Community acquired pneumonia: aetiology and usefulness of severity criteria on admission. Thorax 1996;51:1010–6. doi:10.1136/thx.51.10.1010 [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToidGhvcmF4am5sIjtzOjU6InJlc2lkIjtzOjEwOiI1MS8xMC8xMDEwIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjEvMTIvMDgvMjAyMS4xMi4wNy4yMTI2NzQyNS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 7. Jones BE, Jones J, Bewick T, et al. CURB-65 Pneumonia Severity Assessment Adapted for Electronic Decision Support. Chest 2011;140:156–63. doi:10.1378/chest.10-1296 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1378/chest.10-1296&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=21163875&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F12%2F08%2F2021.12.07.21267425.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000292796700027&link_type=ISI) 8. Nguyen Y, Corre F, Honsel V, et al. Applicability of the CURB-65 pneumonia severity score for outpatient treatment of COVID-19. J Infect 2020;81:e96–8. doi:10.1016/j.jinf.2020.05.049 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.jinf.2020.05.049&link_type=DOI) 9. Satici C, Demirkol MA, Sargin Altunok E, et al. Performance of pneumonia severity index and CURB-65 in predicting 30-day mortality in patients with COVID-19. Int J Infect Dis 2020;98:84–9. doi:10.1016/j.ijid.2020.06.038 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.ijid.2020.06.038&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F12%2F08%2F2021.12.07.21267425.atom) 10. Greco S, Salatiello A, Fabbri N, et al. Early Prediction of COVID-19 Outcome: Contrasting Clinical Scores and Computational Intelligence Methods. In: Studies in Computational Intelligence. 2022. 403–23. doi:10.1007/978-3-030-74761-9_18 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/978-3-030-74761-9_18&link_type=DOI) 11. Liang W, Liang H, Ou L, et al. Development and Validation of a Clinical Risk Score to Predict the Occurrence of Critical Illness in Hospitalized Patients With COVID-19. JAMA Intern Med 2020;180:1081. doi:10.1001/jamainternmed.2020.2033 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jamainternmed.2020.2033&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32396163&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F12%2F08%2F2021.12.07.21267425.atom) 12. Knight SR, Ho A, Pius R, et al. Risk stratification of patients admitted to hospital with covid-19 using the ISARIC WHO Clinical Characterisation Protocol: development and validation of the 4C Mortality Score. BMJ 2020;370:m3339. doi:10.1136/bmj.m3339 [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE3OiIzNzAvc2VwMDlfNy9tMzMzOSI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIxLzEyLzA4LzIwMjEuMTIuMDcuMjEyNjc0MjUuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 13. Yang F, Ma D. The Proportion of SARS-CoV-2 Infections That Are Asymptomatic. Ann Intern Med 2021;174:1343–4. doi:10.7326/L21-0490 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.7326/L21-0490&link_type=DOI) 14. Wu Z, McGoogan JM. Characteristics of and Important Lessons From the Coronavirus Disease 2019 (COVID-19) Outbreak in China: Summary of a Report of 72 314 Cases From the Chinese Center for Disease Control and Prevention. JAMA 2020;323:1239–42. doi:10.1001/jama.2020.2648 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jama.2020.2648&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32091533&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F12%2F08%2F2021.12.07.21267425.atom) 15. Zhou F, Yu T, Du R, et al. Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study. Lancet 2020;395:1054–62. doi:10.1016/S0140-6736(20)30566-3 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S0140-6736(20)30566-3&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32171076&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F12%2F08%2F2021.12.07.21267425.atom) 16. Metz CE. Basic principles of ROC analysis. Semin Nucl Med 1978;8:283–98. doi:10.1016/S0001-2998(78)80014-2 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S0001-2998(78)80014-2&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=112681&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F12%2F08%2F2021.12.07.21267425.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1978FU31300002&link_type=ISI) 17. Cortes C, Vapnik V. Support-Vector Networks. Mach Learn 1995;20:273–97. doi:10.1023/A:1022627411411 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1023/A:1022627411411&link_type=DOI) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1995RX35400003&link_type=ISI) 18. Chawla N V., Bowyer KW, Hall LO, et al. SMOTE: Synthetic Minority Over-sampling Technique. J Artif Intell Res 2002;16:321–57. doi:10.1613/jair.953 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1613/jair.953&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=WOS:00017602&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F12%2F08%2F2021.12.07.21267425.atom) 19. Fonti V, Belitser E. Feature Selection using LASSO. 2017. file:///C:/Users/psn/Downloads/werkstuk-fonti_tcm235-836234(1).pdf 20. Tibshirani R. Regression Shrinkage and Selection via the Lasso. J R Stat Soc Ser B 1996;58:267–288.[https://www.jstor.org/stable/2346178](https://www.jstor.org/stable/2346178) [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.2307/2346178&link_type=DOI) 21. Byrd RH, Lu P, Nocedal J, et al. A Limited Memory Algorithm for Bound Constrained Optimization. SIAM J Sci Comput 1995;16:1190–208. doi:10.1137/0916069 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1137/0916069&link_type=DOI) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1995RR54100011&link_type=ISI) 22. Le Q, Sarlós T, Smola A. Fastfood - Approximating kernel expansions in loglinear time. In: 30th International Conference on Machine Learning, ICML 2013. 2013. 1281–9. 23. Fawcett T. An introduction to ROC analysis. Pattern Recognit Lett 2006;27:861–74. doi:10.1016/j.patrec.2005.10.010 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.patrec.2005.10.010&link_type=DOI) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000237462800002&link_type=ISI) 24. 1. Hutter F., 2. Kotthoff L., 3. Vanschoren J. Feurer M, Hutter F. Hyperparameter Optimization. In: Hutter F., Kotthoff L., Vanschoren J. (eds) Automated Machine Learning. The Springer Series on Challenges in Machine Learning. 2019. 3–33. doi:10.1007/978-3-030-05318-5_1 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/978-3-030-05318-5_1&link_type=DOI) 25. Snoek J, Larochelle H, Adams RP. Practical Bayesian optimization of machine learning algorithms. In: Advances in Neural Information Processing Systems. 2012. 2951–9. 26. Wei P, Lu Z, Song J. Variable importance analysis: A comprehensive review. Reliab Eng Syst Saf 2015;142:399–432. doi:10.1016/j.ress.2015.05.018 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.ress.2015.05.018&link_type=DOI) 27. Fisher A, Rudin C, Dominici F. All Models are Wrong, but Many are Useful: Learning a Variable’s Importance by Studying an Entire Class of Prediction Models Simultaneously. J Mach Learn Res 2019;20.[http://www.ncbi.nlm.nih.gov/pubmed/34335110](http://www.ncbi.nlm.nih.gov/pubmed/34335110) 28. Varol Y, Hakoglu B, Kadri Cirak A, et al. The impact of charlson comorbidity index on mortality from SARS-CoV-2 virus infection and A novel COVID-19 mortality index: CoLACD. Int J Clin Pract 2021;75. doi:10.1111/ijcp.13858 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/ijcp.13858&link_type=DOI) 29. Peckham H, de Gruijter NM, Raine C, et al. Male sex identified by global COVID-19 meta-analysis as a risk factor for death and ITU admission. Nat Commun 2020;11:6317. doi:10.1038/s41467-020-19741-6 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1038/s41467-020-19741-6&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F12%2F08%2F2021.12.07.21267425.atom) 30. Zinellu A, De Vito A, Scano V, et al. The The PaO2/FiO2 ratio on admission is independently associated with prolonged hospitalization in COVID-19 patients. J Infect Dev Ctries 2021;15:353–9. doi:10.3855/jidc.13288 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3855/jidc.13288&link_type=DOI) 31. Kermali M, Khalsa RK, Pillai K, et al. The role of biomarkers in diagnosis of COVID-19 – A systematic review. Life Sci 2020;254:117788. doi:10.1016/j.lfs.2020.117788 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.lfs.2020.117788&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F12%2F08%2F2021.12.07.21267425.atom) 32. Lippi G, Plebani M. Laboratory abnormalities in patients with COVID-2019 infection. Clin Chem Lab Med 2020;58:1131–4. doi:10.1515/cclm-2020-0198 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1515/cclm-2020-0198&link_type=DOI) 33. Wu C, Chen X, Cai Y, et al. Risk Factors Associated With Acute Respiratory Distress Syndrome and Death in Patients With Coronavirus Disease 2019 Pneumonia in Wuhan, China. JAMA Intern Med 2020;180:934. doi:10.1001/jamainternmed.2020.0994 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jamainternmed.2020.0994&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32167524&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F12%2F08%2F2021.12.07.21267425.atom) 34. Salje H, Kiem CT, Lefrancq N, et al. Estimating the burden of SARS-CoV-2 in France. Science (80-) 2020;369:208–11. doi:10.1126/science.abc3517 [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6Mzoic2NpIjtzOjU6InJlc2lkIjtzOjEyOiIzNjkvNjUwMC8yMDgiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMS8xMi8wOC8yMDIxLjEyLjA3LjIxMjY3NDI1LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 35. Onder G, Rezza G, Brusaferro S. Case-Fatality Rate and Characteristics of Patients Dying in Relation to COVID-19 in Italy. JAMA - J. Am. Med. Assoc. 2020;323:1775–6. doi:10.1001/jama.2020.4683 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jama.2020.4683&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32203977&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F12%2F08%2F2021.12.07.21267425.atom) 36. McMichael TM, Currie DW, Clark S, et al. Epidemiology of Covid-19 in a Long-Term Care Facility in King County, Washington. N Engl J Med 2020;382:2005–11. doi:10.1056/nejmoa2005412 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1056/nejmoa2005412&link_type=DOI) 37. Lanzani C, Simonini M, Arcidiacono T, et al. Role of blood pressure dysregulation on kidney and mortality outcomes in COVID-19. Kidney, blood pressure and mortality in SARS-CoV-2 infection. J Nephrol 2021;34:305–14. doi:10.1007/s40620-021-00997-0 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s40620-021-00997-0&link_type=DOI) 38. Patanavanich R, Glantz SA. Smoking is associated with COVID-19 progression: A meta-analysis. Nicotine Tob. Res. 2020;22:1653–6. doi:10.1093/ntr/ntaa082 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/ntr/ntaa082&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32399563&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F12%2F08%2F2021.12.07.21267425.atom) 39. Ejaz H, Alsrhani A, Zafar A, et al. COVID-19 and comorbidities: Deleterious impact on infected patients. J Infect Public Health 2020;13:1833–9. doi:10.1016/j.jiph.2020.07.014 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.jiph.2020.07.014&link_type=DOI) 40. Gold MS, Sehayek D, Gabrielli S, et al. COVID-19 and comorbidities: a systematic review and meta-analysis. Postgrad Med 2020;132:749–55. doi:10.1080/00325481.2020.1786964 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1080/00325481.2020.1786964&link_type=DOI) 41. De Giorgi A, Fabbian F, Greco S, et al. Prediction of in-hospital mortality of patients with SARS-CoV-2 infection by comorbidity indexes: an Italian internal medicine single center study. Eur Rev Med Pharmacol Sci 2020;24. doi:10.26355/eurrev\_202010\_23250 [CrossRef](http://medrxiv.org/lookup/external-ref?access\_num=10.26355/eurrev_202010_23250&link_type=DOI) 42. Tuncer G, Surme S, Bayramlar OF, et al. National Early Warning Score 2 and laboratory predictors correlate with clinical deterioration in hospitalized patients with COVID-19. Biomark Med 2021;15:807–20. doi:10.2217/bmm-2021-0061 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.2217/bmm-2021-0061&link_type=DOI) 43. Yuan Y, Sun C, Tang X, et al. Development and Validation of a Prognostic Risk Score System for COVID-19 Inpatients: A Multi-Center Retrospective Study in China. Engineering Published Online First: November 2020. doi:10.1016/j.eng.2020.10.013 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.eng.2020.10.013&link_type=DOI) 44. Zhao Z, Chen A, Hou W, et al. Prediction model and risk scores of ICU admission and mortality in COVID-19. PLoS One 2020;15:e0236618. doi:10.1371/journal.pone.0236618 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1371/journal.pone.0236618&link_type=DOI) 45. López-Escobar A, Madurga R, Castellano JM, et al. Risk score for predicting in-hospital mortality in covid-19 (Rim score). Diagnostics 2021;11. doi:10.3390/DIAGNOSTICS11040596 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3390/DIAGNOSTICS11040596&link_type=DOI) 46. Halasz G, Sperti M, Villani M, et al. A Machine Learning Approach for Mortality Prediction in COVID-19 Pneumonia: Development and Evaluation of the Piacenza Score. J Med Internet Res 2021;23:e29058. doi:10.2196/29058 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.2196/29058&link_type=DOI) 47. Sorrentino S, Cacia M, Leo I, et al. B-Type Natriuretic Peptide as Biomarker of COVID-19 Disease Severity—A Meta-Analysis. J Clin Med 2020;9:2957. doi:10.3390/jcm9092957 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3390/jcm9092957&link_type=DOI) 48. Soni M, Gopalakrishnan R, Vaishya R, et al. D-dimer level is a useful predictor for mortality in patients with COVID-19: Analysis of 483 cases. Diabetes Metab Syndr Clin Res Rev 2020;14:2245–9. doi:10.1016/j.dsx.2020.11.007 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.dsx.2020.11.007&link_type=DOI) [1]: /embed/graphic-1.gif [2]: /embed/graphic-2.gif