Abstract
The optimal treatment strategy for volume resuscitation and vasopressor dosing to combat hypotensive episodes in septic patients remains a subject of ongoing controversy and can vary from clinician to clinician. We develop a machine learning approach to guide a fluid and vasopressor dosing strategy that adapts to patient-specific clinical states to improve the survival of septic patients. We adopt a model-free reinforcement learning (RL) framework in a continuous action space with a clinically significant reward function, and use a Switching Generalized Linear Model (SGLM) to characterize patient-specific clinical states. We use retrospective data from the MIMIC III database to train this model to learn volume resuscitation and vasopressor dosing strategies among the 5,366 patients (totalling 352,328 unique hourly measurements) with ICU-onset sepsis or septic shock, as diagnosed by the Sepsis-3 definition. The RL agent receives short- and long-term rewards associated with optimizing in-hospital survival and avoiding end-organ damage to learn volume resuscitation and vasopressor dosing strategies. On average, the RL agent learns to resuscitate patients earlier than clinicians with a fluid bolus (one hour vs. four hours after the diagnosis of sepsis), and improves the expected survival by ≈ 3%. Our preliminary results indicate that adherence to RL-based individualized fluid and vasopressor dosing recommendations is associated with a significant mortality reduction in septic patients, even after adjusting for severity of illness.
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
This work was supported by the National Institutes of Health (NIH) early career development award in biomedical big data science (1K01ES025445-01A1) and NIH grant 2R01GM104987-09. The content of this article is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
De-identified data generally does not constitute human subjects research, and so IRB approval wasn't needed.
All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.
Yes
Data Availability
We have used the publicly available MIMIC-III (Medical information for intensive care) dataset of critically ill patients, available at https://mimic.physionet.org/