PT  - JOURNAL ARTICLE
AU  - Albu, Elena
AU  - Gao, Shan
AU  - Stijnen, Pieter
AU  - Rademakers, Frank E.
AU  - Janssens, Christel
AU  - Cossey, Veerle
AU  - Debaveye, Yves
AU  - Wynants, Laure
AU  - Van Calster, Ben
TI  - Hospital-wide, dynamic, individualized prediction of central line-associated bloodstream infections - development and temporal evaluation of six prediction models
AID  - 10.1101/2024.11.04.24316689
DP  - 2024 Jan 01
TA  - medRxiv
PG  - 2024.11.04.24316689
4099  - http://medrxiv.org/content/early/2024/11/04/2024.11.04.24316689.short
4100  - http://medrxiv.org/content/early/2024/11/04/2024.11.04.24316689.full
AB  - Background Central line-associated bloodstream infections (CLABSI) are preventable hospital-acquired infections. Predicting CLABSI helps improve early intervention strategies and enhance patient safety.Aim To develop and temporally evaluate dynamic prediction models for continuous CLABSI risk monitoring.Methods Data from hospitalized patients with central catheter(s) admitted to University Hospitals Leuven between 2014 and 2017 were used to develop five dynamic models (a cause-specific landmark supermodel, two random forest models, and two XGBoost models) to predict 7-day CLABSI risk, accounting for competing events (death, discharge, and catheter removal). The models’ predictions were then combined using a superlearner model. All models were temporally evaluated on data from the same hospital from 2018 to 2020 using performance metrics for discrimination, calibration, and clinical utility.Findings Among 61629 catheter episodes in the training set, 1930 (3.1%) resulted in CLABSI, while in the test set of 44544 catheter episodes, 1059 (2.4%) experienced CLABSI.Among individual models, one XGBoost model reached an AUROC of 0.748. Calibration was good for predicted risks up to 5%, while the cause-specific and XGBoost models overestimated higher predicted risks. The superlearner displayed a modest improvement in discrimination (AUROC up to 0.751) and better calibration than the cause-specific and XGBoost models, but worse than the random forest models. The models showed clinical utility to support standard care interventions (at risk thresholds between 0.5-4%), but not to support advanced interventions (at thresholds 15-25%). A deterioration in model performance over time was observed on temporal evaluation.Conclusion Hospital-wide CLABSI prediction models offer clinical utility, though temporal evaluation revealed dataset shift.Competing Interest StatementThe authors have declared no competing interest.Funding StatementThis work was supported by the Internal Funds KU Leuven [grant C24M/20/064]. The funding sources had no role in the conception, design, data collection, analysis, or reporting of this study.Author DeclarationsI confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.YesThe details of the IRB/oversight body that provided approval or exemption for the research described are given below:The Ethics Committee Research UZ / KU Leuven (EC Research, https://admin.kuleuven.be/raden/en/ethics-committee-research-uz-kuleuven#) gave ethical approval for this work on 19 January 2022 (S60891). The Ethics Committee Research UZ / KU Leuven waived the need to obtain informed consent from participants. All patient identifiers were coded using the pseudo-identifier in the data warehouse by the Management Information Reporting Department of UZ Leuven, according to the General Data Protection Regulation (GDPR).I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.YesI understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).YesI have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.YesThe data underlying this article cannot be shared publicly due to privacy of individuals that participated in the study. Data are located in controlled access database at UZ Leuven.AUPRCArea Under the Precision Recall CurveAUROCArea Under the Receiver Operating Characteristic curveBSBrier ScoreBSIBloodstream InfectionBSSBrier Skill ScoreCICCCentrally Inserted Central Catheters (CICC)CIFCumulative Incidence FunctionsCLABSICentral Line-Associated Bloodstream InfectionsCRCompeting RisksCRPC-reactive proteinCVCCentral Venous CatheterECIEstimated Calibration IndexEHRElectronic Health RecordsFWOResearch Foundation - FlandersICDInternational Classification of DiseasesICUIntensive Care UnitIQRInterquartile RangeKWSKlinisch Werkstation (EHR System)LC-BSILaboratory-Confirmed Bloodstream InfectionLMLandmarkLRlogrankLRCRlogrankCR (logrank competing risks)LWSLaboratorium Werkstation (Laboratory System)MLMachine LearningNMSENormalized Mean Square ErrorOOBOut of bagPICCPeripherally Inserted Central CatheterPDMSPatient Data Management System (ICU Management System)RAMRandom Access MemoryRFRandom Foresttc-CICCtunnelled cuffed Centrally Inserted Central Catheters (CICC)TIVADTotally Implanted Vascular Access Devicestnc-CICCtunnelled non-cuffed Centrally Inserted Central Catheters (CICC)TPNTotal Parenteral nutritionVSCFlemish Supercomputer CenterWBCWhite Blood Cells count