ABSTRACT
Background Despite wide utilisation of severity scoring systems for case-mix determination and benchmarking in the intensive care unit, the possibility of scoring bias across ethnicities has not been examined. Recent guidelines on the use of illness severity scores to inform triage decisions for allocation of scarce resources such as mechanical ventilation during the current COVID-19 pandemic warrant examination for possible bias in these models. We investigated the performance of three severity scoring systems (APACHE IVa, OASIS, SOFA) across ethnic groups in two large ICU databases in order to identify possible ethnicity-based bias.
Method Data from the eICU Collaborative Research Database and the Medical Information Mart for Intensive Care were analysed for score performance in Asians, African Americans, Hispanics and Whites after appropriate exclusions. Discrimination and calibration were determined for all three scoring systems in all four groups.
Findings While measurements of discrimination -area under the receiver operating characteristic curve (AUROC) -were significantly different among the groups, they did not display any discernible systematic patterns of bias. In contrast, measurements of calibration -standardised mortality ratio (SMR) -indicated persistent, and in some cases significant, patterns of difference between Hispanics and African Americans versus Asians and Whites. The differences between African Americans and Whites were consistently statistically significant. While calibrations were imperfect for all groups, the scores consistently demonstrated a pattern of over-predicting mortality for African Americans and Hispanics.
Interpretation The systematic differences in calibration across ethnic groups suggest that illness severity scores reflect bias in their predictions of mortality.
Funding LAC is funded by the National Institute of Health through NIBIB R01 EB017205. There was no specific funding for this study.
Competing Interest Statement
RS received writing fees for healthcare reports from Crystallise UK Ltd. None of the other authors declare any conflict of interest.
Funding Statement
LAC is funded by the National Institute of Health through NIBIB R01 EB017205. There was no specific funding for this study.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
Research using the eICU-CRD is exempt from institutional review board (IRB) approval due to the retrospective design, lack of direct patient intervention, and the security schema, for which the re-identification risk was certified as meeting safe harbour standards by an independent privacy expert (Privacert, Cambridge, MA) (Health Insurance Portability and Accountability Act Certification no. 1031219-2). The data in MIMIC-III has been previously de-identified, and the IRBs of the Massachusetts Institute of Technology (No. 0403000206) and BIDMC (2001-P-001699/14) both approved the use of the database for research. Registration for access to this data was given by the Physionet data repository to the analysts, Chris Martin and Rahuldeb Sarkar, after completion of a required training course (COTIprogram) and signature of a usage agreement.
All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.
Yes
The Chan Zuckerberg Initiative, Cold Spring Harbor Laboratory, the Sergey Brin Family Foundation, California Institute of Technology, Centre National de la Recherche Scientifique, Fred Hutchinson Cancer Center, Imperial College London, Massachusetts Institute of Technology, Stanford University, University of Washington, and Vrije Universiteit Amsterdam.
Subject Area
Reviews and Context
0
Comment
0
TRIP Peer Reviews
0
Community Reviews
0
Automated Services
0
Blogs/Media
Author Videos