PT - JOURNAL ARTICLE AU - Brajer, Nathan AU - Cozzi, Brian AU - Gao, Michael AU - Nichols, Marshall AU - Revoir, Mike AU - Balu, Suresh AU - Futoma, Joseph AU - Bae, Jonathan AU - Setji, Noppon AU - Hernandez, Adrian AU - Sendak, Mark TI - Prospective and External Evaluation of a Machine Learning Model to Predict In-Hospital Mortality AID - 10.1101/19000133 DP - 2019 Jan 01 TA - medRxiv PG - 19000133 4099 - http://medrxiv.org/content/early/2019/06/26/19000133.short 4100 - http://medrxiv.org/content/early/2019/06/26/19000133.full AB - The ability to accurately predict in-hospital mortality for patients at the time of admission could improve clinical and operational decision-making and outcomes. Few machine learning models have been developed to predict in-hospital death that are both broadly applicable to all adult patients across a health system and readily implementable, and, to the best of our knowledge, none have been implemented, evaluated prospectively, or externally validated.The primary objective of this study was to prospectively and externally validate a machine learning model that predicts in-hospital mortality for all adult patients at the time of hospital admission. Model performance was quantified using the area under the receiver operating characteristic curve (AUROC) and area under the precision recall curve (AUPRC). Secondary objectives were to design the model using commonly available EHR data and accessible computational methods.A total of 75,247 hospital admissions (median [IQR] age 59.5 [29.0] years; male [45.9%]) were included in the study. The in-hospital mortality rates for the training validation, retrospective validations at Hospitals A, B, and C, and prospective validation cohorts, respectively, were 3.0%, 2.7%, 1.8%, 2.1%, and 1.6%. The area under the receiver operating characteristic curves (AUROCs), respectively, were 0.87 (0.83-0.89), 0.85 (0.83-0.87), 0.89 (0.86 – 0.92), 0.84 (0.80-0.89), and 0.86 (0.83-0.90). The area under the precision recall curves (AUPRCs), respectively, were 0.29 (0.25-0.37), 0.17 (0.13-0.22), 0.22 (0.14-0.31), 0.13 (0.08-0.21), and 0.14 (0.09-0.21).The results demonstrated accurate prediction of in-hospital mortality for adult patients at the time of admission. The data elements, methods, and patient selection make the model implementable at a system-level.Competing Interest StatementThe authors have declared no competing interest.Funding StatementNo external funding was received.Author DeclarationsAll relevant ethical guidelines have been followed and any necessary IRB and/or ethics committee approvals have been obtained.YesAll necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.YesAny clinical trials involved have been registered with an ICMJE-approved registry such as ClinicalTrials.gov and the trial ID is included in the manuscript.NAI have followed all appropriate research reporting guidelines and uploaded the relevant Equator, ICMJE or other checklist(s) as supplementary files, if applicable.NAThis study used electronic health record data from Duke University Health System. The study was approved by the Duke University Health System Institutional Review Board, which waived the need for informed consent for the use of identifiable patient data.