Predicting Donor Selection and Multi-Organ Transplantation within Organ Procurement Organizations Using Machine Learning

A. Data Collection and Preparation

B. Model Development and Evaluation

Several models well-known for binary classification were developed: K-Nearest Neighbors (KNN), Logistic Regression (LR), Random Forest (RF), Support Vector Machine (SVM), Categorical Naï ve Bayes (NB), XGBoost, and Artificial Neural Network (ANN). The outcome variable was the binary transplanted variable in RQ1, and multiple transplants in RQ2. All other features that remained after pre-processing served as input variables. For each model, a nested stratified 5-fold cross-validation (CV) with grid search was implemented, which can find the optimal hyperparameters without producing an overly-optimistic CV score, as it involved conducting CV using the best hyperparameters determined through grid search. In each CV fold, the data was subject to imputation, standardization, and oversampling. Categorical variables were coded into dummy variables using one-hot encoding. Ordinal variables (e.g., day of the week referred) were encoded with label encoding to preserve their order. A held-out test set with 30% of the data was kept imbalanced (no oversampling applied) to maintain the real-world nature of the data. Missing data in the test set was imputed using the training set’s median of numerical variables and mode of categorical and ordinal variables. This method ensures that the test set remains an unbiased evaluation of the model’s real-world performance. The model was evaluated using the mean recall, precision, F1-score, and AUC score for each cross-validation fold (for the validation set) and the held-out test set (using the best hyperparameter combinations identified in CV). Classification results from the CV stage were compared between oversampled and the raw unbalanced data. Lastly, SHapley Additive exPlanations (SHAP) analysis [13] was implemented with the best-performing model per research question to quantify each feature’s impact on the model output. To ensure replicability and facilitate further research, the source code of all the presented machine learning models is available on GitHub¹.

A. RQ1: What factors influence the selection of a referred donor for organ transplantation?

The input feature set was reduced to 16 features (8 clinical/demographic, 8 administrative) after pre-processing and feature engineering. Most notably, eye and tissue referral features were combined, while brain_death, authorized, time_approached_to_authorized, procured, and time_authorized_to_procured were dropped due to multi-collinearity with other features.

Table I shows cross-validation results and Table II shows the performance on the held-out test set, which revealed significantly different results for some models. Recall was significantly lower for LR, SVM, and NB (dropping from 0.81-0.99 in the CV stage to 0.58-0.80 in testing). In contrast, the precision values were moderately high in the CV stage (0.63-0.80) but improved considerably in testing (LR and SVM yielded 0.99). The AUC values in the testing set were similar to or even higher than the CV results for all models. Altogether, the disparity between some of the CV and held-out test results suggests overfitting in the LR, SVM, and NB models. Overfitting might have occurred because of the small amount of features after pre-processing, resulting in a simpler model. We attempted to avoid overfitting by removing multi-collinear features and applying regularization, which are valid techniques for models like LR and SVM. However, these techniques likely could not remedy the amount of information that might have been lost with feature reduction, given that the number of features in this dataset was already small to begin with. Nevertheless, XGBoost yielded the highest recall (0.80) and F1-score (0.83). The SHAP analysis of the XGBoost model revealed that the approached and time_asystole_to_referral (specifically if the referral milestone was not reached from after asystole) variables had the most impact.

Overall, all ML models were moderately effective in predicting donor selection within OPOs. Precision values were generally higher than recall, which was concerning since having many false negatives may hinder much-needed access to appropriate organ donors. The decrease in recall values between model development and testing suggested overfitting, which despite our efforts was unavoidable due to the limited size of the feature set. XGBoost showed consistently high results. The SHAP analysis (Fig. 1a) revealed that administrative milestones (approached (approaching the donor’s family) and referred (donor referral) not occurring after asystole are the most predictive of donor selection, which highlights the importance of administration and external communication in the organ procurement process. Obtaining family consent for organ donors is a known barrier to organ transplants [14], which in this study held more weight than the donor’s individual characteristics.

• Download figure
• Open in new tab

Fig. 1:

Top 15 SHAP feature importances for (a) XGBoost model predicting donor selection and (b) LR model for multi-organ transplantation. Feature labels in (a): x3 = race, x7 = circumstances of death, x12 = time between asystole-referral, x13 = time between brain death-referral, x14 = time between referral-approach. Feature labels in (b): x12 = heart transplant outcome, x13 = liver transplant outcome, x15 = lung transplant outcome, x16 = kidney transplant outcome, x17 = pancreas transplant outcome

B. RQ2: What factors influence the event of multiple organs being transplanted from one donor after selection?

The input feature set included 25 features after preprocessing and feature engineering. Multi-collinearity was found between left and right kidneys, as well as left and right lungs, resulting in combining outcomes into outcome_kidney and outcome_lung respectively. Overall, classification results were excellent (Table III). LR, RF, XGBoost, and ANN in particular yielded consistently high (≥ 0.97) results for recall, precision, and AUC.

Table IV shows that model performance on the held-out test set was similar to the results in the CV procedure, particularly LR, RF, XGBoost, and ANN, which produced metrics of at least 0.95. These models yielded consistently high precision and recall values in predicting multi-organ transplantation. LR, RF, ANN, and XGBoost yielded the highest AUC. SHAP was applied to the LR model as it yielded the highest metrics in all the categories (precision, recall, and AUC). The analysis (Fig. 1b) revealed that individual organ outcomes -those of particularly heart and lung transplants-were the most predictive of multi-organ transplantation.

Overall, the seven ML models were very effective in predicting multi-organ transplants from OPO-sourced donors. The models generally performed well and did not overfit. Successful transplantation of the heart and lungs was most likely to lead to more organs being transplanted from the same donor. These findings appear to support real-world organ allocation policies. For instance, the Organ Procurement and Transplantation Network (OPTN) policies state that when a candidate is eligible to receive a heart, lung, or liver, the second required organ would be allocated to the recipient from the same donor if the donor is located in the same local organ distribution unit where the recipient is registered [15]. The results of the present study may encourage the continued implementation of this policy in an effort to enhance organ allocation and equity.