Abstract
Importance No medical artificial intelligence (AI) has been robustly validated and deployed in a clinical laboratory in real-world settings, and the clinical impact of the medical AI remains unknown.
Objective To deploy a medical AI platform for rapid antibiotics susceptibility test (AST) prediction, and evaluate its clinical impacts.
Design A medical AI platform, XBugHunter, was extensively validated (internal validation, time-wise validation, and independent testing) with data between May 22, 2013 and June 30, 2019. The clinical impact was evaluated based on a prospective observation from February 1 to September 30, 2020 during deployment.
Setting Data was collected in two tertiary medical centers in Taiwan, and the AI was deployed in a tertiary medical center.
Participants For validation, 90,064 consecutive cases were included. During the deployment, a prospective observational cohort of 1,490 consecutive cases was collected.
Exposures AST prediction from XBugHunter
Main outcomes and Measures Diagnostic performance of XBugHunter was evaluated during validation. The clinical impact was evaluated in terms of the saving of inappropriate antibiotics prescription, AST turn-around-time, and mortality of bacteremia during deployment.
Results Predictive models consistently performed well in the extensive validations. In the deployment, XBugHunter’s predictive sensitivity and specificity for Staphylococcus aureus (oxacillin) were 0.95 (95% CI, 0.82–0.98) and 0.97 (95% CI, 0.94–0.99), respectively. For Acinetobacter baumannii (multiple drugs), the sensitivity was 0.95 (95% CI, 0.91–0.99) and specificity was 0.93 (95% CI, 0.88–0.98). The turn-around-time reduction on reporting AST of blood cultures was 35.72 h (standard deviation: 15.55 h). Death within 28 days occurred in 28 of 162 S. aureus bacteremia patients (17.28%) in the XBugHunter intervention group, which was lower than the 28 days’ mortality rate (28.06% [55/196]) in the same period of time in 2019, without XBugHunter. The relative risk reduction was 38.4% (relative risk, 0.62; 95% CI, 0.41–0.92). Regarding antibiotic prescriptions, 2723.7 defined daily dose per year of inappropriate antibiotics could be avoided for treating S. aureus by deploying XBugHunter.
Conclusions and Relevance Among S. aureus bacteremia patients, this study demonstrated that XBugHunter can prevent inappropriate antibiotic use, and the adjustment of antibiotic treatment can yield lower mortality.
Question What is the clinical impact of XBugHunter, a machine learning-based, antibiotic susceptibility test prediction platform?
Findings In the prospective observational cohort of deploying XBugHunter, the reduction of turn-around-time of reporting antibiotic susceptibility test was 35.72h. The reduction of S. aureus bacteremia mortality rate was 10.78%, and the estimated saving of inappropriate antibiotics uses was 2723.7 defined daily dose per year.
Meaning Deployment of XBugHunter provides a more rapid report of antibiotic susceptibility test, and thus reduces inappropriate antibiotics prescription and mortality of S. aureus bacteremia.
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
This work was supported by Chang Gung Memorial Hospital (CMRPG3L0401, CMRPG3L0431)
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
The study was approved by the Institutional Review Board of the Chang Gung Medical Foundation (No. 201900767B0)
All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.
Yes
Data Availability
Data are available for individuals that are approved.
