The influence of baseline clinical status and surgical strategy on early good to excellent result in spinal lumbar arthrodesis: a machine learning approach

Clinical and demographic data and outcomes

The study was conducted in a single spine surgery center and it is based retrospective review of prospectively collected data from the institutional registry: SpineReg [11]. The inclusion criteria were age ≥ 18 years, both genders, lumbar arthrodesis procedure identified using ICD-9 code (8106, 8107 or 8108), a follow up assessment (ODI, SF-36 and Core Outcome Measures Index - COMI back) and the capability to read and understand the Italian language. A full set of peri-operative and post-operative data along with clinical outcomes from January 2016 to December 2020 were evaluated. Exclusion criteria were a weak degree of baseline disability or pain (ODI < 20/100 and COMI back < 3/10), number of levels fused not specified and subject not stratified according to Glassman classification.

The study protocol was conducted in accordance with the Helsinki Declaration of 1957 as revised in 2000. The procedures followed the ethical standards of the responsible committee on human experimentation and was approved by the ethics committee of Ospedale San Raffaele, Milan, Italy (protocol “SPINEREG”, approved on April 13^th, 2016). The project was supported with funds from the Italian Ministry of Health (project code CO-2016-02364645). All patients gave their written informed consent for the participation in the registry. Baseline demographics, BMI, gender, comorbidities collected through Comorbidity Charlson Index (CCI) [12], diagnosis according to Glassman Classification [13], number of spinal levels of intervention, spinal level indexed surgery, clinical scores resulting from medical surveys, complication and revision surgeries were collected.

Table 1 shows that the rates of missing data ranges from 0.0% for the baseline scores of the PROMs and for the patient’s personal information, to 20.4% for Levels variables. Independent variables with at least one missing value were imputed using predictive mean matching for numerical variables, binary/multinomial logistic regression or ordered logit model for categorical variables. Outcome variables were not imputed to avoid the introduction of bias into the results. Only patients with the observed outcome variables were included in the analysis.

The primary outcome was the early (six months post op) significant clinical improvement. In particular, value of improvement higher than 12.7 for ODI [14], 6 for SF36-PCS and 2.2 for COMI Back were considered as indicator of significant clinical improvement [15, 16].

To classify surgical operations results with a “good early outcome”, we defined a delta of improvement of the ODI higher than 12.7/100. On the other hand, to identify surgical operations with an “excellent early outcome” we used combined target that identifies an excellent clinical result when all the three underlying targets ODI (Δ ≥ 12.7/100), SF-36 PCS (Δ ≥ 6/100) and COMI back (Δ ≥ 2.2/10) showed a relevant improvement.

An exploratory analysis was performed. Patients were classified as having or not the binomial risk factor (Risk +) or (Risk -). Different scenarios were simulated to verify the performance of each method of calculation in three age categories. For each scenario, a 2 × 2 table was built (Good Outcome+/- vs. Risk +/-). Chi-squared test was used for statistical association between reaching the good outcomes and the presence of risk factor. The odds-ratios with their 95 % confidence intervals, and point estimations of the sensitivity and specificity of the alignment rules to discriminate patients with final good or poor clinical outcome, the positive (PPV) and negative (NPV) predictive values and positive and negative likelihood ratios (LR+, LR-) were calculated. Differences between preoperative and postoperative clinical outcomes were tested with the two-tailed Student’s t test for paired samples. The Mann-Whitney U-test was used in the cases of abnormally distributed variables. Normality was verified with the Kolmogorov–Smirnoff test. The threshold for statistical significance was set at p < 0.05 in all the tests.

Machine learning approach

The available dataset is characterized by the non-negligible presence of missing values. Therefore, data imputation of independent variables is first performed to exploit all the instances and obtain more stable and reliable results [17]. In the dataset, different data types are available and each of them is treated with dedicated techniques. In the case of numerical variables, for each instance with at least a missing value a small subset of complete instances similar to the instance under investigation is selected. From this set, a randomly sampled instance is used to replace the missing values. Discrete variables are instead imputed with ad hoc models: logistic regression models for binary variables, multinomial logistic regression models for unordered categorical variables and ordered logit models (or proportional odds models) for categorical variables with ordered categories. The data imputation is implemented with “mice” R package [18].

A multivariate classification model is used to predict the target variables: 1) single ODI improvement or 2) combined ODI + SF36 PCS + COMI Back. For both the targets, we use a Random Forest (RF) classification method to predict the outcome of the surgical operation. RF is an ensemble model composed by multiple decision trees, each of them trained independently on randomly sampled subsets of variables. The single outputs of the multiple decision tree models are then combined with a majority vote to obtain the final decision of RF. This ensemble helps in improving the predictive performance of the individual decision tree models. Indeed, RF has been recognized as one of the best performing classifiers in extensive classification studies [19], and the R implementation provided by the “randomForest” package is empirically more accurate than other implementations [20]. We thus train a RF model using the default settings of the “randomForest” R package for both targets. To evaluate the most important features used by RF to classify instances, we used the Mean Decrease Gini index, which measures the contribution of each variable to the homogeneity of internal and leaf nodes of the tree.

We train RF in cross-validation (5 folds) and we select the classification threshold for each fold by optimizing the geometric mean of sensitivity and specificity in a nested cross-validation loop. The proposed nested cross-validation allows to robustly estimate the optimal classification threshold and assess RF performance, while balancing sensitivity and specificity. This is particularly relevant since both the target variables are slightly unbalanced (70.7% of the available data are associated with a good surgical outcome and 43.3% of the available data are associated with an excellent surgical outcome).

The performance of the model is evaluated in terms of sensitivity, i.e., True Positive Rate (TPR), specificity, i.e., True Negative Rate (TNR), accuracy and area under the receiver operating characteristic curve (AUC ROC). The exploratory and the further machine learning analysis were performed in R [21].

The entire study was performed according to TRIPOD guideline for the development of multivariate models for individual prognosis or diagnosis [22].

1. Predictive Models of surgical improvement based on clinical data

The surgical treatment for degenerative spine disorders has been shown to improve the quality of life and reduce disability in patients most severely affected [23]. Nevertheless, the association between demographic baseline factors and overall complication rates are still unclear.

Thanks to the anesthesiological and surgical implementations, a large population can safely be a spinal surgery candidate. The increase in the use of mini-invasive surgery [24] and the improvements of pre-operative planning methods [25] has allowed enlarging the cohort of patients eligible for surgery and capable of obtaining significant results [26, 27].

One of the most relevant demographic indicators is BMI (Body Mass Index). High BMI values are known to be risk factors for many diseases and directly correlated with the complication rate after spinal surgery. Even if in the literature, its role it’s still debated in the prediction of functional outcomes.

According to Mulvanay et al. [28], an increased BMI is associated with decreased effectiveness of 1- to 3-level elective lumbar fusion, despite the absence of surgical complications. A BMI value higher than 30 is considered a risk factor for surgical complications and poor spine surgery results. According to our data, a low BMI seems to present a relevant role in predicting good clinical outcomes. Despite, several studies suggest that weight does not represent a major impact on the patients’ health-related quality of life after surgery[29], obesity has a relevant impact on intraoperative blood loss, length of surgery and complication rate. It seems that BMI should always be kept in mind when planning spinal fusion.

Several clinical indicators of postoperative success are continuously analyzed to improve the surgical outcome in terms of complication rate and patients’ satisfaction. The aging of the population and the relative increase of the comorbidities can challenge the surgical decision.

According to Daniels et al. [30], the upgrading of some surgical methods increased the performances of therapeutic strategies. In particular, in a retrospective analysis of surgical cases enrolled between 2009 and 2016 the complication rates decreased over time, despite an increasingly elderly, medically compromised, and obese patient population. As a critical point, the authors identified the evolution of surgical strategies that resulted in an overall improvement of the treatment quality.

2. Predictive Models of surgical improvement based on PROMs

The proper estimation of the pre-operative degree of disability and quality of life is mandatory when surgery is required. In spine surgery, considerable controversy exists regarding spinal arthrodesis’ risk-benefit where surgery itself creates a permanent fusion of vertebral bodies. Nevertheless, several studies demonstrate a significant improvement after spinal arthrodesis in cases of degenerative spinal disorders [31–33]. A combination of scales is often used in clinical studies to assess multiple aspects of human being.

The indicators of quality of life and disability progressively gained attention, becoming the gold standard to measure the success rate after spine surgery. The post-operative clinical improvement can be evaluated based on patients’ reported outcomes such as ODI. Although post-operative improvement may be statistically significant, it is not necessarily clinically relevant. For this reason, several studies have defined the values to indicate a difference that is clinically meaningful to the patient (MCID) [34]. In particular, Monticone et. al defined as significant a cut off value of MCID at 12.7 ODI unit score of improvement [14].

3. Predictive models performances (General)

Predictive models for patient-reported outcomes can improve the surgical strategies when deciding to opt for surgery or not or potentially to adapt the surgical approach.

Despite the significant variability in the population affected by a common clinical condition, lumbar disc herniation, Staartjes et al. [35] proposed a predictive model based on deep learning-based analytics. Out of a population of 422 patients, the deep learning and logistic regression attained AUC values of 0.84 and 0.72, and an accuracy of 75% and 59%, respectively. The greatest discrepancy in performance measures regarded the models predicting back pain improvement. This could reflect the model’s weakness or the inherent difficulty of the outcome to be measured.

Models based on Naïve Bayes machine learning to predict hospitalization days and indications for discharge (for example, admission to rehabilitation facilities or back to home and hospitalization costs) showed high performances. In particular, the system proposed by Karnuta et al. revealed a predictive accuracy of 0.800 for costs of recovery, 0.874 for Length of Stay (LOS), and 0.878 for disposition with AUC for hospitalization costs (0.880), excellent AUC for LOS (0.941), and an excellent AUC for discharge disposition (0.906) [36].

The disease variability, combined with the psychological influencing factors and patients’ expectations related to the surgeries, challenges the accuracy of clinical predictions. Siccoli et al. [37] evaluated the feasibility of short- and long-term PROMs and reoperation rate by ML approach in patients affected by lumbar stenosis. According to this study, the models were able to predict the endpoints providing accurate information.

Although a progressive increase in the use of prediction models in spine surgery, little is known in spinal arthrodesis for 2 to 4 levels surgery.

Our results seem to provide comparable or higher predictive performances than other studies on spine surgery. Thanks to the recent advances in technologies, AI can involve the application of mathematical algorithms that continuously learn and make observations from existing data. The aim is to create a more accurate predictive model based on that data [38].

4. Influence of ML predictions on therapeutic strategy

With the widening of the modern dataset, the use of ML will progressively become the gold standard and the primary candidate for the data analysis. A shining future application in diagnosis, prognosis and decision-making process is desirable and will soon become an essential spine physician tool. Khan et al. introduced the application in the clinal management of cervical myelopathy and nontraumatic spinal cord injury to predict the risk of neurological impairment at one year [39]. These tools allowed the physicians to predict individual patient outcome after surgery for degenerative cervical myelopathy [40] and to apply preventive strategies as targeted physiotherapy and timing of psychological counselling.

With the application of ML techniques, several studies demonstrate the possibility to predict clinical outcomes. Ames C.P. et al. predicted the patients’ responses to SRS-22R (questionnaire) item per item up to 86.9% AUROC at 1 and 2 years following surgical treatment for ASD. The main clinical application is to aid surgical decision making during preoperative counselling [41]. In complex surgery, this approach will be capable of implementing already available surgical decision-making [42].