PT  - JOURNAL ARTICLE
AU  - Brajer, Nathan
AU  - Cozzi, Brian
AU  - Gao, Michael
AU  - Nichols, Marshall
AU  - Revoir, Mike
AU  - Balu, Suresh
AU  - Futoma, Joseph
AU  - Bae, Jonathan
AU  - Setji, Noppon
AU  - Hernandez, Adrian
AU  - Sendak, Mark
TI  - Prospective and External Evaluation of a Machine Learning Model to Predict In-Hospital Mortality
AID  - 10.1101/19000133
DP  - 2019 Jan 01
TA  - medRxiv
PG  - 19000133
4099  - http://medrxiv.org/content/early/2019/06/26/19000133.short
4100  - http://medrxiv.org/content/early/2019/06/26/19000133.full
AB  - The ability to accurately predict in-hospital mortality for patients at the time of admission could improve clinical and operational decision-making and outcomes. Few machine learning models have been developed to predict in-hospital death that are both broadly applicable to all adult patients across a health system and readily implementable, and, to the best of our knowledge, none have been implemented, evaluated prospectively, or externally validated.The primary objective of this study was to prospectively and externally validate a machine learning model that predicts in-hospital mortality for all adult patients at the time of hospital admission. Model performance was quantified using the area under the receiver operating characteristic curve (AUROC) and area under the precision recall curve (AUPRC). Secondary objectives were to design the model using commonly available EHR data and accessible computational methods.A total of 75,247 hospital admissions (median [IQR] age 59.5 [29.0] years; male [45.9%]) were included in the study. The in-hospital mortality rates for the training validation, retrospective validations at Hospitals A, B, and C, and prospective validation cohorts, respectively, were 3.0%, 2.7%, 1.8%, 2.1%, and 1.6%. The area under the receiver operating characteristic curves (AUROCs), respectively, were 0.87 (0.83-0.89), 0.85 (0.83-0.87), 0.89 (0.86 – 0.92), 0.84 (0.80-0.89), and 0.86 (0.83-0.90). The area under the precision recall curves (AUPRCs), respectively, were 0.29 (0.25-0.37), 0.17 (0.13-0.22), 0.22 (0.14-0.31), 0.13 (0.08-0.21), and 0.14 (0.09-0.21).The results demonstrated accurate prediction of in-hospital mortality for adult patients at the time of admission. The data elements, methods, and patient selection make the model implementable at a system-level.Competing Interest StatementThe authors have declared no competing interest.Funding StatementNo external funding was received.Author DeclarationsAll relevant ethical guidelines have been followed and any necessary IRB and/or ethics committee approvals have been obtained.YesAll necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.YesAny clinical trials involved have been registered with an ICMJE-approved registry such as ClinicalTrials.gov and the trial ID is included in the manuscript.NAI have followed all appropriate research reporting guidelines and uploaded the relevant Equator, ICMJE or other checklist(s) as supplementary files, if applicable.NAThis study used electronic health record data from Duke University Health System. The study was approved by the Duke University Health System Institutional Review Board, which waived the need for informed consent for the use of identifiable patient data.