Summary
Background Central line-associated bloodstream infections (CLABSI) are preventable hospital-acquired infections. Predicting CLABSI helps improve early intervention strategies and enhance patient safety.
Aim To develop and temporally evaluate dynamic prediction models for continuous CLABSI risk monitoring.
Methods Data from hospitalized patients with central catheter(s) admitted to University Hospitals Leuven between 2014 and 2017 were used to develop five dynamic models (a cause-specific landmark supermodel, two random forest models, and two XGBoost models) to predict 7-day CLABSI risk, accounting for competing events (death, discharge, and catheter removal). The models’ predictions were then combined using a superlearner model. All models were temporally evaluated on data from the same hospital from 2018 to 2020 using performance metrics for discrimination, calibration, and clinical utility.
Findings Among 61629 catheter episodes in the training set, 1930 (3.1%) resulted in CLABSI, while in the test set of 44544 catheter episodes, 1059 (2.4%) experienced CLABSI.
Among individual models, one XGBoost model reached an AUROC of 0.748. Calibration was good for predicted risks up to 5%, while the cause-specific and XGBoost models overestimated higher predicted risks. The superlearner displayed a modest improvement in discrimination (AUROC up to 0.751) and better calibration than the cause-specific and XGBoost models, but worse than the random forest models. The models showed clinical utility to support standard care interventions (at risk thresholds between 0.5-4%), but not to support advanced interventions (at thresholds 15-25%). A deterioration in model performance over time was observed on temporal evaluation.
Conclusion Hospital-wide CLABSI prediction models offer clinical utility, though temporal evaluation revealed dataset shift.
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
This work was supported by the Internal Funds KU Leuven [grant C24M/20/064]. The funding sources had no role in the conception, design, data collection, analysis, or reporting of this study.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
The Ethics Committee Research UZ / KU Leuven (EC Research, https://admin.kuleuven.be/raden/en/ethics-committee-research-uz-kuleuven#) gave ethical approval for this work on 19 January 2022 (S60891). The Ethics Committee Research UZ / KU Leuven waived the need to obtain informed consent from participants. All patient identifiers were coded using the pseudo-identifier in the data warehouse by the Management Information Reporting Department of UZ Leuven, according to the General Data Protection Regulation (GDPR).
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.
Yes
Data Availability
The data underlying this article cannot be shared publicly due to privacy of individuals that participated in the study. Data are located in controlled access database at UZ Leuven.
Abbreviations
- AUPRC
- Area Under the Precision Recall Curve
- AUROC
- Area Under the Receiver Operating Characteristic curve
- BS
- Brier Score
- BSI
- Bloodstream Infection
- BSS
- Brier Skill Score
- CICC
- Centrally Inserted Central Catheters (CICC)
- CIF
- Cumulative Incidence Functions
- CLABSI
- Central Line-Associated Bloodstream Infections
- CR
- Competing Risks
- CRP
- C-reactive protein
- CVC
- Central Venous Catheter
- ECI
- Estimated Calibration Index
- EHR
- Electronic Health Records
- FWO
- Research Foundation - Flanders
- ICD
- International Classification of Diseases
- ICU
- Intensive Care Unit
- IQR
- Interquartile Range
- KWS
- Klinisch Werkstation (EHR System)
- LC-BSI
- Laboratory-Confirmed Bloodstream Infection
- LM
- Landmark
- LR
- logrank
- LRCR
- logrankCR (logrank competing risks)
- LWS
- Laboratorium Werkstation (Laboratory System)
- ML
- Machine Learning
- NMSE
- Normalized Mean Square Error
- OOB
- Out of bag
- PICC
- Peripherally Inserted Central Catheter
- PDMS
- Patient Data Management System (ICU Management System)
- RAM
- Random Access Memory
- RF
- Random Forest
- tc-CICC
- tunnelled cuffed Centrally Inserted Central Catheters (CICC)
- TIVAD
- Totally Implanted Vascular Access Devices
- tnc-CICC
- tunnelled non-cuffed Centrally Inserted Central Catheters (CICC)
- TPN
- Total Parenteral nutrition
- VSC
- Flemish Supercomputer Center
- WBC
- White Blood Cells count
