Serum Potassium Monitoring using AI-enabled Smart Watch Electrocardiograms

Patient identification and data sources

A primary cohort of patients with paired ECGs and serum potassium levels was curated from the CSMC between 2008 to 2022. A total of 153,971 adult patients had at least one 12-lead ECG and serum potassium test within one hour, leading to 297,914 twelve-lead ECGs and corresponding potassium values. ECGs were mapped to the temporally closest potassium test. Given an interest in assessing performance in the ESRD population, all recordings from non ESRD patients (n = 152,508) were used for initial model training, and the model was then finetuned on ESRD patients alone (n = 1,463). The cohorts were randomly split at the patient level into 80% for training set, 10% for validation, and 10% reserved as a held-out test set.

For external validation, we identified patient cohorts from two other healthcare systems. We identified 7,586 ECGs among 3,107 ESRD patients at Stanford HealthCare (SHC) from 8/2005 to 6/2018 (Figure 1). Additionally, A prospective cohort of patients with ESRD at Chang Gung Memorial Hospital (CGMH) were enrolling from October 2022 to July 2023 with serial smartwatch based ECG testing prior to hemodialysis. Prior to participation, written consent was secured from each individual. This study was approved by the institutional review boards of the CSMC, SHC, and CGMH. Patients’ informed consent was waived at CSMC and SHC due to the retrospective nature of the study using de-identified ECG and electronic health record data.

• Download figure
• Open in new tab

Figure 1:

Selection of study subjects in the primary and external validation cohorts.

Model Development

We developed a convolutional neural network (CNN) designed for ECG interpretation, Kardio-Net, with the capability to integrate digitized waveform for predicting serum potassium levels. All ECGs obtained from CSMC were recorded using a GE machine, adhering to the standard 10-second, 12-lead ECG protocol. These ECG waveform data were captured at a sampling rate of 500 Hz and represented as 10-second, 12×5000 matrices of amplitude values. In external validation at SHC, ECGs were stored using the Phillips TraceMaster system with the same sampling rate with recorded also as 12×5000 matrices and were independently used as input examples for external validation.

Kardio-Net was trained on PyTorch starting from random initialization. Training utilized a mean square error loss for up to 100 epochs, employing an ADAM optimizer with a starting learning rate of 1e-2. Early stopping based on minimizing validation cohorts mean absolute error (MAE). For the fine-tuning in ESRD patients, weights from the initial model were used as the starting point and trained using the same loss function and optimizer with a learning rate of 1e-4. To ensure compatibility with the smartwatch ECG data, we trained the Kardio-Net by utilized lead I from the 12-lead ECGs, as it most closely resembles the ECG vector of smartwatch recordings. For Kardio-Net inference on ECG waveforms smartwatch, we split the waveform data into consecutive 5-second intervals. During the inference, a 30-second ECG waveform was split at 1-second intervals, yielding 26 5-second waveform data inputs. These were then averaged to formulate the final prediction.

Validation on smartwatch ECG

After consent and enrollment of the patients, digital ECG recordings were captured by the Apple Watch ECG within 5 minutes before their routine serum potassium tests prior to hemodialysis. The smartwatch ECG waveforms, initially recorded at a sampling rate of 512 Hz, were resized to 500 Hz to match the sampling rate of 12 lead ECGs. The digitized ECG waveforms stored on the hospital server via the Apple Developer HealthKit API for subsequent analysis²⁰. This process yielded a validation dataset of 589 ECGs waveforms paired with its corresponding potassium levels. Given that smartwatch waveforms can contain artifacts and baseline wandering distinct from 12-lead ECGs, we developed a noise ECG classifier to exclude excessively noisy waveforms before averaging the predictions for final analysis. 23 (3.9%) ECGs were excluded from the noise classifier. A domain adaptation technique was employed to facilitate the Kardio-Net’s application to Apple Watch ECG waveforms¹⁹. This adjustment aims to mitigate domain discrepancies, thereby enhancing Kardio-Net‘s performance on Apple Watch ECGs.

Statistical analysis

All analyses were conducted on the held-out test dataset, external validation set, and smartwatch dataset, which were not used during the model’s training. The primary metric for assessing Kardio-Net’s performance in predicting serum potassium levels was the MAE. Additionally, the model’s ability to identify severe hyperkalemia, defined as potassium levels greater than 6.5 mEq/L, was evaluated using the area under the receiver operating characteristic curve (AUC). These metrics allow for a comprehensive assessment of the model’s classification accuracy and precision in detecting cases of hyperkalemia. For each metric, we calculated two-sided 95% confidence intervals using 1000 bootstrapped samples to ensure robust statistical inference. The modeling pipeline was implemented using Python (3.8) with PyTorch (2.0) as the deep learning framework. Signal processing and data analysis were facilitated by Python libraries such as SciPy (1.11), Scikit-learn (1.3.2), pandas (2.0.2), and matplotlib (3.7.1).

Primary cohort characteristics

We identified a total of 153,971 patients at Cedars-Sinai Medical Center who had at least one pair of ECG and serum potassium measurement within 1 hour, resulting in 293,557 ECG recordings. 1,463 patients diagnosed with ESRD contributed 4,337 ECGs were selected as ESRD cohort, where other 293,557 ECGs from 152,508 patients formed the general cohort. The ESRD cohort was divided into a training set of 1,170 patients, a validation set of 146 patients, and a test set of 147 patients. The median age within the ESRD group was 62 years (interquartile range [IQR]: 50-72), with 57.8% (846 patients) being male. The median serum potassium level was 4.6 mEq/L (IQR: 4.1-5.2), with 190 ECGs (4.4%) indicating a potassium level higher than 6.5 mEq/L. Additional demographic and clinical characteristics are detailed in Table 1.

Kardio-Net performance in the primary cohort

The initial deployment of our 12-lead Kardio-Net model, which was pre-trained on a general cohort, yielded an MAE of 0.593 mEq/L in the test set of ESRD cohort, with predictions concentrating around the 4.0 - 5.0 mEq/L level (Extended Figure 1). Upon fine-tuning this model with training set in ESRD cohort, there was a notable improvement in prediction accuracy, with the MAE reducing to 0.527 mEq/L. Kardio-Net achieved discrimination of hyperkalemia (potassium > 6.5 mEq/L) with an AUC of 0.852 (95% CI 0.745–0.956) (Figure 2).

• Download figure
• Open in new tab

Central Illustration:

Kardio-Net is a convolutional neural network (CNN) designed to predict serum potassium levels in patients with end-stage renal disease (ESRD) using ECG data. Capable of processing both 12-lead and single-lead ECGs, this model adapts to traditional and smartwatch-based systems to provide continuous monitoring. Evaluated on retrospective and prospective cohorts, including smartwatch ECGs, Kardio-Net demonstrates robust performance with an AUC ranging from 0.807 to 0.876 in detecting hyperkalemia.

• Download figure
• Open in new tab

Extended Figure 1:

Scatter plots and MAE illustrating improved performance of the model in predicting serum potassium levels in the primary ESRD patient cohort after fine-tuning compared to the pre-training phase, using data from 12-lead (A) and single-lead (B) ECGs.

• Download figure
• Open in new tab

• Download figure
• Open in new tab

Figure 2:

Deep learning model performance in internal and external validations. Figure 2A displays a scatter plot of serum potassium predictions from the 12-lead ECG model alongside the ROC curve for hyperkalemia recognition. Figure 2B presents the results from the single-lead ECG model.

To adapt to apple watch, we trained a single-lead ECG model using only lead I data through the same process. The model achieved an AUC of 0.876 (95% CI 0.765–0.987) on hyperkalemia detection and MAE of 0.575 mEq/L. Further evaluation of Kardio-Net’s performance across various patient subgroups, including older individuals and those with diabetes, hypertension, atrial fibrillation, heart failure, and coronary artery disease, was conducted. These groups are generally considered high-risk or exhibit ECG morphology changes. The model’s discrimination AUC for detecting hyperkalemia varied from 0.725 to 0.983 across these subgroups. Detailed subgroup results are shown in figure 3.

• Download figure
• Open in new tab

Figure 3:

Subgroup analysis of hyperkalemia detection performance. Figure 3A illustrates the AUC for different subgroups within CSMC, and Figure 3B for SHC.

• Download figure
• Open in new tab

Figure 4.

Correlated of ECG Waveforms with Serum Potassium Levels. Figure 4A shows mean ECG waveforms associated with measured serum potassium concentrations. The graph displays waveforms for potassium levels categorized into four ranges: 3.5 to 4.5 mEq/L (blue), 4.5 to 5.5 mEq/L (orange), 5.5 to 6.5 mEq/L (green), and >6.5 mEq/L (red), demonstrating the expected ECG changes corresponding to each potassium range. Figure 4B shows mean ECG waveforms corresponding to serum potassium level as predicted by Kardio-Net.

Kardio-Net performance in the external validation dataset

The external validation cohort in SHC consisted of a total of 7,586 ECGs among 3,107 patients. The median age was 60 years (IQR: 49-70), with 58.6% (1,820 patients) being male. Atrial fibrillation is present in 25.3% of cases, heart failure in 37.0%, CAD in 35.0%, hypertension in 70.1%, and diabetes mellitus in 48.7%. The median serum potassium level was 4.5 mEq/L (IQR: 4.0-5.1), with 340 ECGs (4.5%) mapping to a severe hyperkalemia result. Other demographic and clinical characteristics are presented in Table 1.

In the external validation dataset, the performances of our 12-lead and single-lead Kardio-Net were consistent with the primary cohort. The 12-lead model demonstrated an AUC of 0.849 (95% CI 0.823-0.875) for hyperkalemia discrimination and achieved an MAE of 0.599 mEq/L. The single-lead model yielded an AUC of 0.807 (95% CI 0.778–0.835) and an MAE of 0.740 in predicting potassium levels. Further analysis in external validation was also conducted using the same subgroup to primary cohort. Kardio-Net’s discrimination AUC for detecting hyperkalemia varied from 0.780 to 0.860 across these subgroups with consistent performance.

Validation in Smartwatch ECG Waveforms

In the prospective international cohort, 589 ECG waveforms were collected from 40 CGMH patients. The median age was 65 years ([IQR]: 58-71), with 16 male patients (40.0%). The median serum potassium level was 4.9 mEq/L (IQR: 4.4-5.3), with 8 ECGs (1.5%) associated with severe hyperkalemia. 23 (3.9%) ECGs were excluded for excessive noise and baseline wander. Single-lead Kardio-Net demonstrated an MAE of 0.580 mEq/L in predicting serum potassium base on smartwatch ECG. The model achieved an AUC of 0.831 (95% CI 0.693–0.975) for hyperkalemia detection using smartwatch ECG waveforms.

Study Limitations

Several limitations of the study merit consideration. First, the retrospective design of training data cohort may introduce selection bias of individuals who would receive 12-lead ECGs and serum electrolyte checks within 1 hour. This clinical setting might not reflect the variability found in broader outpatient settings or during daily activities where wearable technologies are used. However, we were able to evaluate the model’s performance on a prospectively collected cohort of patients immediately prior to hemodialysis. The prospective international cohort, which utilized smartwatch-generated waveforms from diverse racial groups, supports the generalizability of Kardio-Net. The use of real-world clinical data and the innovative application of deep learning techniques to a critical clinical biomarker substantially increase the relevance and potential impact of our findings in patient care management.

Perspectives