ABSTRACT
Objective Recent advances in deep learning show significant potential in analyzing continuous monitoring electronic health records (EHR) data for clinical outcome prediction. We aim to develop a Transformer-based, Encounter-level Clinical Outcome (TECO) model to predict mortality in the intensive care unit (ICU) using inpatient EHR data.
Materials and Methods TECO was developed using multiple baseline and time-dependent clinical variables from 2579 hospitalized COVID-19 patients to predict ICU mortality, and was validated externally in an ARDS cohort (n=2799) and a sepsis cohort (n=6622) from the Medical Information Mart for Intensive Care (MIMIC)-IV. Model performance was evaluated based on area under the receiver operating characteristic (AUC) and compared with Epic Deterioration Index (EDI), random forest (RF), and extreme gradient boosting (XGBoost).
Results In the COVID-19 development dataset, TECO achieved higher AUC (0.89–0.97) across various time intervals compared to EDI (0.86–0.95), RF (0.87–0.96), and XGBoost (0.88–0.96). In the two MIMIC testing datasets (EDI not available), TECO yielded higher AUC (0.65–0.76) than RF (0.57–0.73) and XGBoost (0.57–0.73). In addition, TECO was able to identify clinically interpretable features that were correlated with the outcome.
Discussion TECO outperformed proprietary metrics and conventional machine learning models in predicting ICU mortality among COVID-19 and non-COVID-19 patients.
Conclusions TECO demonstrates a strong capability for predicting ICU mortality using continuous monitoring data. While further validation is needed, TECO has the potential to serve as a powerful early warning tool across various diseases in inpatient settings.
LAY SUMMARY In intensive care units (ICUs), accurately estimating the risk of death is crucial for timely and effective medical intervention. This study developed a new AI algorithm, TECO (Transformer-based, Encounter-level Clinical Outcome model), which uses electronic health records to continuously predict ICU mortality after admission, with the capability to update predictions on an hourly basis. TECO was trained on data from over 2,500 COVID-19 patients and was designed to analyze multiple types of continuous monitoring data collected during a patient’s ICU stay. We tested TECO’s performance against a widely used proprietary tool, the Epic Deterioration Index (EDI), and other machine learning methods, such as random forest and XGBoost, across three patient groups: COVID-19, ARDS (acute respiratory distress syndrome), and sepsis. TECO consistently showed better performance and was able to predict death risk earlier than other methods. Additionally, TECO identified key health indicators associated with ICU mortality, making its predictions more interpretable for clinicians. These findings suggest that TECO could become a valuable early warning tool, helping doctors monitor patients’ health and take timely action in a range of critical care situations.
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
This study was supported in part by the National Institutes of Health under award number R01GM140012 (GX), R01DE030656 (GX), R01GM115473 (GX), U01CA249245 (GX), U01AI169298 (YX), R35GM136375 (YX), the Cancer Prevention and Research Institute of Texas (CPRIT RP180805: YX; CPRIT RP230330: GX), and the Texas Health Resources Clinical Scholars Program.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
The institutional review boards at Texas Health Resources and the University of Texas Southwestern Medical Center approved this study (Protocol #STU-2020-0786; activated on 8/24/2020). All patient identifiers were removed before EHR data extraction.
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.
Yes
The Chan Zuckerberg Initiative, Cold Spring Harbor Laboratory, the Sergey Brin Family Foundation, California Institute of Technology, Centre National de la Recherche Scientifique, Fred Hutchinson Cancer Center, Imperial College London, Massachusetts Institute of Technology, Stanford University, University of Washington, and Vrije Universiteit Amsterdam.