Graph-Based Patient Representation for Multimodal Clinical Data: Addressing Data Heterogeneity

ABSTRACT

Patient data comprises of different modalities like clinical notes, lab results, and radiological investigations. Predictive modeling on this patient data is challenging due to the heterogeneity among patients and modalities captured, such as no ECG recordings for a few patients. Ensuring model applicability to all admitted patients requires addressing three key factors: i) handling modalities at disparate time-scales (e.g., once in a few hours medication and 125 Hz ECG need to be handled by same model), ii) handling missing modality (e.g., ECG not recorded), and iii) modeling temporal interactions between modalities over time (e.g., the ECG fluctuation triggered some lab test order). Existing literature often doesn’t simultaneously address these requirements. Therefore, we propose a novel patient representation approach inspired by clinical workflows, representing each patient as a graph. We categorize patient data into two main components: observations and care team actions, allowing cross-modality temporal interaction when actions consider the previous observations. We define observation nodes to capture modality-specific data within the time between two actions, and action nodes capture data of the actions undertaken, with edges to capture the temporal dependencies. To address missing modalities and time-scale disparities, we define node types for different modalities and use modality-specific representation for the nodes; implying a missing modality is equivalent to a missing node type in the graph. Aligned with clinical workflows, this patient-graph representation aims to enhance the practicality of predictive systems for various healthcare tasks, from mortality risk assessment to medication recommendations, thereby improving clinical decision support.