Development and validation of a multivariate model for predicting heart failure hospitalization and mortality in patients receiving maintenance hemodialysis ============================================================================================================================================================ * Wenwu Tang * Xinzhu Yuan * Zhixin Wang * Ying Zhang * Xiaoxia Chen * Xiaohua Yang * Zhirui Qi * Ju Zhang * Jie Li * Xisheng Xie ## Abstract **Background** Heart failure (HF) is a common cardiovascular disease in patients receiving maintenance hemodialysis (MHD). Once these patients on MHD exhibit HF, their hospitalization rate, mortality, and economic burden will be significantly increased. Early identification and prediction of hospitalization and death are of great importance for reducing hospitalization and mortality. This study used multicenter clinical data to develop and externally validate clinical risk models to predict expected mortality and HF hospitalization rates in MHD patients with HF. **Materials and Methods** From January 2017 to October 2022, 348 patients receiving MHD from four participating centers were enrolled. Demographic data, MHD treatment modalities, laboratory tests, and echocardiography data were collected when the initial event occurred. Three centers were randomly assigned to the modeling dataset (n=258), and one center was assigned to the external validation set (n=90). Considering a composite outcome of HF hospitalization and death as the primary endpoint and hospitalization due to HF or death as the secondary endpoint, a COX clinical prediction model was constructed and verified using internal and external datasets. **Results** The median age of patients in the modeling cohort was 63 years old, 41.5% of patients were women; 165 (61%) had a history of HF; 81 (31.4%) were hospitalized for HF; and 39 (15.1%) patients had died. The c-statistic values for composite outcome, hospitalization for HF, and mortality were 0.812, 0.808, and 0.811, respectively. The predictors of death and hospitalization outcomes caused by HF are significantly different. The strongest predictors of HF hospitalization outcomes were advanced age, multiple HF hospitalizations, hyponatremia, high levels of NT-proBNP and hs-cTnT, and larger MVe values. The strongest predictors of mortality were longer dialysis age, combined atrial fibrillation, calcification of the aortic or mitral valve (especially calcification, and in particular aortic valve calcification), pleural effusion, low serum sodium, and higher levels of hs-cTnT. The median age of the patients in the external validation cohort was 63 years old; 28.8% were female; 35 (38.1%) had a history of HF; 11 (12.2%) were hospitalized for HF; and 5 (5.6%) died. The c-statistic of the predictive models for composite outcome, hospitalisation for HF, and mortality was comparable to that of the modelling cohort. **Conclusion** The model established in this study is stable and reliable and the included variables are easily obtained from the routine clinical environment. The model can provide useful risk factors and prognostic information for patients with MHD combined with HF. Keywords: heart failure, MHD patients, mortality, predictive model, external validation. ## 1. Introduction The number of patients receiving renal replacement therapy (RRT) worldwide exceeded 2.5 million in 2010, and is expected to increase to approximately 5.4 million by 2030[1]. The risk of cardiovascular disease (CVD) in patients on maintenance hemodialysis (MHD) is 20-times higher than that in the general population, and heart failure (HF) is the second most common CVD in patients undergoing MHD (about 10.2%)[2,3]. According to the United States Renal Data System data report, the medical insurance expenditure for patients with MHD in 2016 alone was as high as $35 billion; in addition, the 2-year mortality rate of patients with chronic kidney disease (CKD) but without HF was approximately 16.5%, while the overall 2-year mortality rate those with HF was as high as 33.3%[4–6]. Compared to the general population of patients with HF, patients receiving MHD face more complex risk factors for uremic cardiomyopathy based on traditional risk factors (hypertension, diabetes) due to impaired glomerular filtration and secretory function; Furthermore, although patients receiving MHD can prolong survival with long-term RRT treatment, this also increases risks related to dialysis (such as hemodynamics, rapid changes in electrolyte composition, and micro-inflammatory state induced by dialysis materials). Therefore, it is becoming an urgent need to effectively identify individuals with HF at high-risk of hospitalization or death among these patients[7–9]. Although the clinical condition is severe, it is still difficult to predict the mortality and hospitalization rate of HF patients using the published prediction models. A meta-analysis reported that only 33% of the 117 published predictive models were validated in separate cohort studies, and most studies only reported moderate predictive performance[9,10]. Joanne’s predictive model had a c-statistic of 0.71 and 0.70 for HF hospitalization and death, respectively[11]; while the c-statistics of Joshua’s prediction model were 0.80 and 0.79, respectively[10–12]. Unfortunately, these prediction models almost all excluded or did not focus on patients receiving MHD, so they are not suitable for this specific population. Recently, the model by Gotta et al. focused mainly on the prediction of death in patients with MHD under 30 years of age but did consider the effect of HF in these patients[13]. The purpose of the present study is to establish a prediction model suitable for patients with MHD to accurately predict the risk of hospitalisation and death from HF, to strengthen the early identification and intervention of clinicians, and to assist doctors and patients in making better personalized management decisions to effectively improve the prognosis of patients receiving MHD. In addition, the model also helps to select high-risk patients more accurately for future clinical trials and to enrich the reported incidence of clinical events, and reduce the sample size of future studies[14]. ## 2. Materials and methods ### 2.1 Materials We used the Transparent Reporting of a Multivariate Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) as a guide to develop and validate our model. #### 2.1.1 Subjects We enrolled subjects From January 2017 to October 2022 (with at least 60 days of follow-up), 1079 patients receiving MHD from four centers (Nanchong Central Hospital, Guangyuan Central Hospital, Suining Central Hospital and Peng’an County People’s Hospital). Patients with CKD5 dialysis who met the Kidney Disease Improving Global Outcomes (KDIGO) criteria developed by the American Society of Nephrology, and regular hemodialysis time ≥3 months, at least 2 times a week, and met the clinical diagnostic criteria determined by the International RLS Study Group in 2014 were enrolled. #### 2.1.2 Inclusion criteria All patients (1) aged >18 years old; (2) who had received MHD treatment for more than 3 months; (3) conformed to the diagnostic criteria of HF; (4) who had provided informed consent and voluntary participation in this study; and (5) those with complete clinical data were selected for the final analysis. #### 2.1.3 Exclusion criteria Patients meeting the following criteria were excluded: (i) having a history of liver disease, malignant tumor, mental illness, or other serious diseases; (ii) patients who were unable to cooperate, unwilling to participate, or had incomplete clinical data. Following the application of the inclusion and exclusion criteria, 693 patients without hospitalization history and 38 patients with missing biochemical or echocardiographic data were excluded, and 348 patients were included in the study (Figure 1).  [Figure 1](http://medrxiv.org/content/early/2023/03/15/2023.03.14.23287278/F1) Figure 1 Research flow chart (A) Patient inclusion process according to nadir criteria; (B) schematic diagram of start-and endpoint events; (C) schematic diagram of allocation of the modelling and validation datasets. ### 2.2 Methods #### 2.1.1 Diagnostic criteria for HF The following criteria were used to diagnose HF in this study: (a) new or worsening HF symptoms or previous hospitalisation history; (b) left ventricular ejection fraction (LVEF) ≤40%; if LVEF >40%, abnormal N-terminal B-type natriuretic peptide (NT-proBNP) (>11215.2 pg/mL), (c) with either ‘left ventricular hypertrophy and/or left atrial enlargement’ or ‘cardiac diastolic dysfunction’. A diagnosis of HF was made by meeting either a/b or a/c of the above criteria. The diagnosis of HF was made by two trained physicians, and a specialist was consulted in case of disagreement. #### 2.1.2 NT-proBNP In this study, NT-proBNP > 11215.2 pg/mL was selected as the diagnostic threshold for HF in MHD patients. #### 2.2.3 Starting point and endpoint of observation The time of first hospitalization of the patient was taken as the starting point of observation, and the times of hospitalization due to HF, death, transfer, renal transplantation, and arrival after follow-up was taken as the endpoint of observation. The time between the starting point and the endpoint was considered as the follow-up time. In this study, the composite outcome of HF hospitalization and death was the primary endpoint, and HF hospitalization or death was the secondary endpoint (Figure 1). #### 2.2.4 Study Indicators A total of 112 variables were included in this study. (1) General and dialysis-related data: including sex, age, dialysis age, type of dialysis vascular access, reasons for dialysis (diabetic nephropathy, glomerular inflammatory disease, or hypertension), previous cumulative HF hospitalization times and total hospitalization days, body mass index (BMI), dialysis times (weekly), dialysis duration (h/time). (2) Clinical data: vital signs at admission (pulse, pulse pressure difference), typical symptoms at HF (exertional or nocturnal paroxysmal dyspnea, sitting breathing, edema, rales, third heart sound, jugular vein filling, or positive hepatic neck reflux sign), history of coronary heart disease or coronary stent surgery, other surgical history (thyroid radiofrequency ablation or arterial dissection), history of cerebrovascular accident (cerebral infarction or cerebral hemorrhage), history of atrial fibrillation, bradycardia (conduction block), metabolic acidosis, infection (pneumonia, urinary tract infection, dialysis catheter-related infection, and sepsis), thyroid function, history of diabetes and diabetes-related complications (such as diabetic retinopathy, and diabetic peripheral neuropathy), chronic obstructive pulmonary disease (COPD), history of other lung diseases (respiratory failure or pulmonary edema, bronchitis, bronchiectasis, emphysema or bullae, silicosis, pneumoconiosis, lung damage, pulmonary fibrosis), liver or pancreatic diseases (fatty liver, cirrhosis and decompensation, pancreatic injury, hepatitis B), history of cholecystitis or cholecystectomy, gastrointestinal ulcer, gastrointestinal inflammation and diarrhea, gastrointestinal bleeding, history of serous cavity effusion (pericardium, chest, pelvis), coagulation abnormalities or less platelets, immunodeficiency, tumor, NYHA classification, HF stage, and length of hospital stay at the beginning of the event. (3) Laboratory examination: white blood cell count, C-reactive protein, hemoglobin, parathyroid hormone, NT-proBNP, high sensitivity troponin T (hs-cTnT), total protein, albumin, uric acid, creatinine, cystatin, triglyceride, total cholesterol, high density lipoprotein cholesterol, low density lipoprotein cholesterol, aspartate aminotransferase (AST), alanine aminotransferase (ALT), total bilirubin, serum potassium, serum sodium, serum calcium, and serum phosphorus. (4) Echocardiographic data: left atrial changes, left ventricular enlargement (atrial and ventricular), right atrial changes, right ventricular enlargement, whole heart enlargement, aortic or pulmonary artery changes (widened), interventricular septum changes (thickened), left ventricular wall changes (hypertrophy), ventricular wall motion ability (reverse movement of interventricular septum and left ventricular posterior wall, and diffuse weakening of left ventricular wall), calcification of aortic valve or mitral valve, valvular regurgitation (aortic valve, pulmonary valve, mitral valve, and tricuspid valve), left ventricular diastolic dysfunction, left ventricular or right ventricular systolic dysfunction, pulmonary hypertension, pericardial effusion (mm), left atrium (LA), left ventricle (LV), right atrium (RA), right ventricle (RV), interventricular septum (IVS), left ventricular posterior wall (LVPW), main pulmonary artery (MPA), aorta or aorta root (AO), ascending aorta (AAO), anterior mitral flow velocity (MVe), aortic valve (AV), fractional shortening (FS), and LVEF. The reasons for including the above variables are as follows: First, factors related to death and hospitalization rates have been identified in combination with guidelines and related studies. Secondly, compared to non-MHD patients, risk factors for HF in MHD patients were more complex. This study collected data relative to the above variables based on the data availability of each center and cases with missing data were excluded. ### 2.3. Design of the prediction model #### 2.3.1 Construction and evaluation of prediction model Patients were divided into training and test groups based on center enrolment, and the clinical features were selected from all independent variables. The importance of each index in different model training and test groups was compared using a COX prediction model, and the best model was then used for evaluation and verification. The detailed steps are described below. ##### 2.3.1.1 Allocation of modeling and validation sets The four centers were divided into two groups with the composite endpoint of “death and HF hospitalization” or “HF hospitalization”. Three centers were assigned to the modelling set (n=258) and one center was assigned to the validation set (n=90). The patients in the modelling cohort (n=258) were then divided into a modelling set (n=194) and an internal validation set (n=64) in a 3:1 ratio. The median follow-up for the modeling cohort was 10 months (IQR, 5– 16 months). Because the sample size of death events included only 44 cases, for the model established with “death” as the outcome, all patients (n=348) were randomly divided into the modeling cohort (n=279) and the internal validation cohort (n=69) according to the ratio of 4:1. The median follow-up time of the modeling cohort was 12 months (IQR: 7–20 months). The allocation process is detailed in Figure 1. ##### 2.3.1.2 Statistical analysis of clinical features for screening patients SPSS v26.0 (IBM, USA)was used for COX univariate analysis and R software (University of Auckland, New Zealand, glmnet 4.2.2) for Least Absolute Shrinkage and Selection Operator (LASSO) regression analysis. LASSO regression can compress variable coefficients, prevent overfitting, and resolve serious collinearity concerns[15]. SPSS v26.0 was used for multivariate COX regression analysis, and finally the characteristic variables with two-tailed P-values <0.05 were obtained [16]. ##### 2.3.1.3 Model verification R software (version 4.2.2) and the “survivalROC” package were used to construct receiver operating characteristic (ROC) curves of the modeling set and the validation set, and the area under the curve and c-statistics were calculated to validate the accuracy of the prediction model [17]. The R software package “survivalROC” was used to draw calibration curves to measure the predictive ability of the model and to reflect the concordance between the predicted risk and the actual risk[17]. To assess the clinical applicability of the prediction model, the R software package ‘stdca’ was used to perform decision curve analysis (DCA)[18]. ### 2.4 Statistical analysis Variables included in the modelling and validation sets were compared. Continuous variables were expressed as median and interquartile ranges (IQR) and compared using the Mann–Whitney U test. Categorical variables were expressed as numbers and percentages and compared using the chi-square test. A statistically significant differences was defined by two-sided tests with a P-value<0.05. SPSS v26.0 and the R software package v4.2.2 were used for statistical analysis. ## 3. Results ### 3.1 Comparison of baseline data All variables were investigated for all patients at the time of the initiation event: laboratory tests and ultrasound data were the first results after admission, and data related to dialysis treatment were the closest to the pre-initiation schedule. Hyponatremia was defined as a serum sodium level of 135 mmol/L. Infections included pneumonia, sepsis, urinary tract, or dialysis catheter-related infections. The specific baseline data for the modelling set and the external validation set are shown in Table 1. The differences between the two sets were not statistically significant (P >0.05). View this table: [Table 1](http://medrxiv.org/content/early/2023/03/15/2023.03.14.23287278/T1) Table 1 Baseline characteristics of the modeling and external validation sets ### 3.2 Model construction The results of COX univariate analysis and LASSO regression analysis on the dependent variables composite endpoint and HF hospitalization showed that the number of predictors could be reduced to 16 and 10, respectively. Taking death as the dependent variable, 19 predictors were identified as independent variables after univariate analysis (Supplementary Tables 1 and 2). To further control for the influence of confounding factors, multivariate COX stepwise regression analysis was performed on the above variables, and a risk prediction model was constructed. The composite endpoint prediction model contained seven variables. A higher number of hospitalizations due to HF (pulmonary rales, hyponatremia, or hypoalbuminemia), previous history of cerebral hemorrhage, aortic or mitral valve calcification, on by echocardiography, and higher NT-proBNP levels could predict the composite outcome (Table 2). The model for predicting HF hospitalization included six variables: older age, previous hospitalizations for HF, hyponatremia at the time of HF, higher NT-proBNP and high-sensitivity troponin T (hs-TnT) levels, higher MVe values measured by cardiac ultrasound could predict HF hospitalization (Table 2). The predictive death model contained seven variables. Longer dialysis age, previous history of atrial fibrillation, presence of aortic or mitral valve calcification (VC) on echocardiography, the occurrence of HF with infection and pleural effusion, low serum sodium levels, and higher hs-TnT could predict death (Table 2 and Supplementary Table 3). View this table: [Table S1](http://medrxiv.org/content/early/2023/03/15/2023.03.14.23287278/T4) Table S1 Single factor COX analysis View this table: [Table S2:](http://medrxiv.org/content/early/2023/03/15/2023.03.14.23287278/T5) Table S2: Variable screening process View this table: [Table S3:](http://medrxiv.org/content/early/2023/03/15/2023.03.14.23287278/T6) Table S3: Multi-factor Cox model predictor distribution comparison View this table: [Table 2](http://medrxiv.org/content/early/2023/03/15/2023.03.14.23287278/T2) Table 2 Univariate model built based on the composite outcome ### 3.3 Model verification According to the composite outcome prediction model, a nomogram was drawn to predict the composite endpoint at 1-2 years. Drawing a line directly to the 1-2–year probability axis after summing the points for each variable to obtain total score, indicates the probability of HF hospitalization or death in 1-2 years. The receiver operating characteristic (ROC) curves of the modeling set and the validation set were plotted, and the area under the curve and C statistics (Table 3) were calculated. The calibration curve, decision curve analysis (DCA), and Kaplan-Meier (K-M) curves were plotted (Figure 2).  [Figure 2:](http://medrxiv.org/content/early/2023/03/15/2023.03.14.23287278/F2) Figure 2: Validation of composite outcome prediction model (A) Predictive composite outcome model and nomogram. (B-C) The AUC curve of a 1-year and 2-year composite outcome. (D-E) Calibration chart for predicting 1- and 2-year composite outcomes. Clinical decision curve analysis of the nomogram: the Y axis represents fraction surviving, the X axis represents the threshold probability, the red line represents the net benefit of the nomogram model in the training cohort, and the green line represents the net benefit of the nomogram model in the validation cohort. (F) Clinical decision curve analysis of the nomogram for the training cohort and (G) for the validation cohort. The Kaplan–Meier survival curve was drawn according to the risk of death or hospitalization due to HF. View this table: [Table 3](http://medrxiv.org/content/early/2023/03/15/2023.03.14.23287278/T3) Table 3 C-statistic values of composite endpoint, HF hospitalization and death model A nomogram for predicting HF hospitalization was drawn based on the results of the HF hospitalization prediction model. Drawing a line to the 1-2–year probability axis after summing the points of each variable to obtain a total score, indicates the 1-2 year survival probability following HF hospitalization. The receiver operating characteristic (ROC) curves of the modeling set and the validation set were plotted, and the area under the curve and C statistics (Table 3) were calculated. The calibration curve, decision curve analysis (DCA), and Kaplan-Meier (K-M) curves were plotted (Figure 3).  [Figure 3:](http://medrxiv.org/content/early/2023/03/15/2023.03.14.23287278/F3) Figure 3: HF hospitalization prediction model validation (A) Nomogram for predicting heart failure (HF) hospitalization. (B-C) AUC curve predicting a 1-year HF hospitalization. (D-E) The 1-year heart failure outcome calibration chart. Clinical decision curve analysis of the nomogram The Y axis represents the fraction surviving; the X axis represents the threshold probability; the red line represents the net benefit of the nomogram model in the training cohort; the green line represents the net benefit of the nomogram model in the validation cohort. (F) Clinical decision curve analysis of the nomogram in the training cohort and (G) in the validation cohort. (H) Kaplan–Meier survival curve according to the risk of HF hospitalization. Based on the death outcome prediction model, a nomogram was drawn to predict 1–2 year death outcomes. After the summing the points of the individual variables this correspond to a total score on the total points axis, which is issued to indicated the probability of 1-2-year death by drawing a line to the 1-2 year survival probability axis. The receiver operating characteristic (ROC) curves of the modeling set and the validation set were plotted, and the area under the curve and C statistics (Table 3) were calculated. The calibration curve, decision curve analysis (DCA), and Kaplan-Meier (K-M) curves were plotted (Figure 4).  [Figure 4:](http://medrxiv.org/content/early/2023/03/15/2023.03.14.23287278/F4) Figure 4: Verification of death prediction model (A) Nomogram for predicting death. (B-C) AUC curve for the 1-year composite outcome. (D-E) Calibration plot for predicting 1- and 2-year heart failure outcomes. Clinical decision curve analysis of the nomogram. The Y axis represents the fraction surviving; the X axis represents the threshold probability; the red line represents the net benefit of the nomogram model in the training cohort; the green line represents the net benefit of the nomogram model in the validation cohort. (F) Clinical decision curve analysis of the nomogram for the training cohort and (G) for the validation cohort. (H) Kaplan-Meier survival curve according to the risk of HF hospitalization. COX regression analysis and 10-fold cross validation were performed on the training cohort and the internal and external validation sets. The results showed that the AUC values (Figures 2–4) and c-statistics of the three prediction models were remained above 0.8 (Table 3), indicating the model’s discrimination ability was good. For both internal and external validation cohort, the calibration plot showed a good consistency between the predicted HF hospitalization, mortality risk, and actual risk of the nomogram. The DCA curve indicated that the prediction model © had good clinical usability. The respective Kaplan-Meier survival curves showed that the three models could effectively distinguish high-risk and low-risk groups (Figures 2–4). We use variables in the compact model for risk scoring. The decision tree algorithm performs the best classification according to the critical points shown in the figure. For each ‘bad’ variable, the patient increases the score by 1 point. The score range for mortality, HF hospitalization rate, and the combined endpoint is 0-3. Finally, the corresponding K-M curve is drawn according to the score. (Figure 5).  [Figure 5](http://medrxiv.org/content/early/2023/03/15/2023.03.14.23287278/F5) Figure 5 (A) composite outcome prediction model; (B) heart failure hospitalization prediction model; (C) death prediction model ## 4. Discussion HF is a common cardiovascular disease in MHD patients, which significantly increases the hospitalization rate, mortality, and poses significant economic burden on MHD treatment. However, it is still difficult to predict the mortality and hospitalization rate of MHD patients with HF[4–6]. Most of the published prediction models have not been externally validated and are not suitable for MHD patients. We screened different variables using 112 predictors by LASSO regression and multivariate COX regression analysis and constructed a prediction model with HF hospitalization, death, and composite outcome as outcome events. To our knowledge, this study established the first model suitable for the prediction of HF risk in patients receiving MHD. The c-statistics of the internal and external validation sets were >0.8, indicating that the model’s prediction performance was good. The modeling dataset and the verification dataset were completely independent cohorts obtained from different centers, which verifies the applicability and stability of the prediction model. Secondly, because patients receiving MHD will regularly attend a hemodialysis center for treatment, we obtained follow-up information relatively accurately, thus ensuring the authenticity and reliability of the data obtained. In addition, the predictive model provided in this study can provide effective prognostic information for patients receiving MHD with only a small number of easily available clinical variables, indicating that the model is suitable for the real-life clinical setting and is convenient to use. With regard to the choice of diagnostic cutoff value for NT**-**proBNP levels, there are currently different viewpoints. First, in patients undergoing MHD, the glomerular filtration rate (GFR) is reduced, NT-proBNP clearance rate decreases, and volume overload occurs. This increased volume state causes BNP to be secreted in conjunction with increased myocardial wall relaxation. Second, a cutoff of NT-proBNP <125 pg/mL (negative exclusion) has been proposed by the latest guidelines in Europe and the United States for common HF or NT-proBNP levels >2000 pg/mL have been established by similar studies, but are not applicable to HF in patients receiving MHD. In contrast, the study by Jafri et al. suggested that the sensitivity and specificity of NT-proBNP values ≥112,15.2 pg/mL for the diagnosis of HF in patients with CKD5 were 94.7% and 100%, respectively[19]. In addition, in our study, we also diagnosed HF using the above three cut-off values and constructed a diagnostic model and found that using the NT-proBNP <125 pg/mL or >2000 pg/mL cutoff values led to overdiagnosis of HF. Furthermore, the model constructed using the criteria of NT-proBNP >11,215.2 pg/mL had better predictive power (Figure S3). In conclusion, NT-proBNP >11,215.2 pg/mL was selected as the cutoff value for the diagnosis of HF in patients receiving MHD patients for this study.  [Figure S1:](http://medrxiv.org/content/early/2023/03/15/2023.03.14.23287278/F6) Figure S1: HR values of single-factor COX regression analysis  [Figure S2:](http://medrxiv.org/content/early/2023/03/15/2023.03.14.23287278/F7) Figure S2: LASSO Regression analysis **Death outcome:** After single-factor COX analysis, the predictors of death outcome entered the multiple regression equation. A multivariate COX stepwise regression (forward method) was directly performed, and the model was constructed.  [Figure S3:](http://medrxiv.org/content/early/2023/03/15/2023.03.14.23287278/F8) Figure S3: Diagnosis of heart failure and construction of logistic models according to different criteria: Diagnosis of heart failure and construction of logistic models according to different criteria: (A) The percentage of HF diagnosed with three different NT-proBNP cutoff values was significantly different. (B) Different NT-proBNP (from top to bottom: <125 pg/mL, > 2000 pg/mL, > 11 215.2 pg/mL) cut-off values were used to diagnose heart failure, construct a prediction model, and draw a calibration chart. It was suggested that the model constructed with NT-proBNP > 11215.2 pg/mL was clinically realistic. Interestingly, the predictors of the three prediction models partially overlapped. In our prediction model, hyponatremia simultaneously predicted the risk of HF hospitalization and death in MHD patients. Studies in the general population have shown that hyponatremia is a good predictor of adverse outcomes in acute or chronic HF morbidity and mortality[14,20,21]. The study by Marroquin et al. showed that hyponatremia was associated with CVD before the transition to end-stage kidney disease (ESKD) and a high hospitalization rate after ESKD transition[22]. The results of this study further indicated that hyponatremia is still an important predictor of HF hospitalization and composite outcomes in MHD patients. Although the predictive factor included in the mortality prediction model is serum sodium, it also indicates that lower serum sodium is associated with a higher risk of death. However, it is still not clear whether hyponatremia directly leads to higher hospitalization rates and mortality or is a sign of potential complications (such as severe infection or lung disease). Recent studies by Marroquin et al. found that there is still a close relationship between hyponatremia and higher mortality and hospitalization rates after adjusting for comorbidities[22]. We also obtained similar results after adjusting for comorbidities in our models (Supplementary Table 4). View this table: [Table S4:](http://medrxiv.org/content/early/2023/03/15/2023.03.14.23287278/T7) Table S4: Association of hyponatremia with heart failure hospitalization and death after adjustment for comorbidities Our study found that NT-proBNP appeared in both the HF hospitalization and composite outcome prediction models, while hs-cTnT appeared in both HF hospitalization and death risk prediction models. A recent meta-analysis found that NT-proBNP serum levels in patients with ESKD were positively correlated with cardiovascular events and mortality risks[23]. In addition, we also used the best decision tree method to determine that NT-proBNP >16,386 pg/mL and >12,414 pg/mL were high-risk groups for compound outcomes and HF hospitalization in MHD patients, respectively (Figure 5). Clinically, hs-cTnT is most commonly used to determine the presence of myocardial infarction or HF-related injury in patients with acute HF (AHF). However, in the absence of myocardial ischemia, the elevation of hs-cTnT indicates the presence of strong myocardial cell injury, progressive ventricular remodeling, and an increased risk of death[24]. Chesnaye et al. found that high levels of hs-cTnT predicted a higher risk of death and a poorer prognosis in patients with stage 4-5 CKD[25]. This study shows that hs-cTnT is an independent predictor of death in MHD patients, and the combination of NT-proBNP and hs-cTnT could effectively predict the risk of HF hospitalization in these patients. Interestingly, our study also found that cardiac VC was a predictive factor in both the death and compound outcome models. The study found that the prevalence of aortic and mitral valve VC in MHD patients was eight times higher than that in the general population[26–28]. In patients receiving MHD, hyperphosphatemia, hyperparathyroidism, increased fibroblast growth factor-23, and decreased co-receptor klotho levels are significantly associated with VC[27]. Cardiac VC can lead to impaired valve opening and closing function and then rapidly progresses to valve stenosis. In particular, aortic valve stenosis increases blood flow resistance, causing left ventricular hypertrophy and HF leading to premature death[26,27]. In summary, compared with the prediction model for HF in the general population, VC has a unique and good predictive value for predicting death in patients receiving MHD. It is noteworthy that there were also significant differences in the predictors for different models. Low albumin, pulmonary rales, and a history of cerebral hemorrhage only appeared in the composite outcome prediction model, while age and MVe only appeared in the HF hospitalization model. Although E/A and E/e ratios have been repeatedly used to predict cardiovascular death and HF hospitalization in previous studies, they were not included in our model analysis due to the large differences in ultrasound reports collected from various centers[29]. Surprisingly, this study also effectively predicted the risk of HF hospitalization in MHD patients through the simple MVe value in echocardiography, which seemed to increase the general applicability of the model. In addition, we also found that atrial fibrillation (AF), infection, dialysis age, and pleural effusion only appeared in the death prediction model. A number of studies have shown that AF is closely related to HF, which significantly increases the incidence and mortality due to CVD in MHD patients. Early screening and intervention in patients with AF can effectively improve the poor prognosis of high-risk groups[30–34]. Recently, Evangelos et al. together with the numerical citation.proposed that the factors hyponatremia, AF, and HF constitute a difficult triangle, and any side of it will affect the other side, increasing morbidity and mortality[35]. The death prediction model established in this study includes these three important factors, which shows that our model is suitable for the real-world clinical setting. Pleural effusion has rarely been mentioned in previous predictive models or mechanism studies. In this study, the results of logistic multivariate analysis showed that the formation of pleural effusion may be closely related to respiratory failure, pulmonary edema, HF, pulmonary infection, and AF (Supplementary Table 5), and that pleural effusion may reflect the existence of severe complications in MHD patients and may predict the occurrence of death. It is expected that future research will demonstrate its correlation with death outcomes and its mechanism. View this table: [Table S5:](http://medrxiv.org/content/early/2023/03/15/2023.03.14.23287278/T8) Table S5: A logistics model with pleural effusion as an outcome event and multiple comorbidities as predictors was constructed. Our research has some limitations that should be considered. First, this study was a multi-center retrospective cohort study. The clinical data derived from four hospitals in different regions. The health care system and patient treatment vary greatly, which may affect management, outcomes, and prediction. Second, in order to ensure the integrity of the data, we only collected the patient’s previous hospitalization data and did not collect data with mild symptoms or remission after outpatient treatment, which may have led to a selection bias in the data, but nonetheless allowed our model to distinguish the HF hospitalization risk. Compared with prospective randomized controlled trials, we failed to include variables that are currently a focus of research attention into the analysis (such as biomarkers, genomics, and proteomics information) and did not effectively control for confounding factors. Finally, the outcome was determined by the attending physician alone and not by a ruling committee. Although, a systematic meta-analysis did not find any effect due to differences in event adjudication on the conclusions of cardiovascular outcome trials[36]. ## 5. Conclusion This study developed and validated models for predicting mortality, HF hospitalization, and composite outcomes in patients receiving MHD. The models are suitable for application in MHD patients and for use at different levels of medical institutions. The prediction performances of the models were good, stable, and easy to use. In addition, the prediction models propose a simplified risk score for clinical practice to effectively stratify patients and provide effective prognostic information. ## Funding Sichuan Provincial Administration of Traditional Chinese Medicine Research Fund (2020JC0079); Sichuan Provincial Department of Science and Technology Research Special Fund (2021YFS0259); Nanchong Science and Technology Plan Project (22JCYJPT0005) ## Institutional Review Board Statement The study was conducted in accordance with the Declaration of Helsinki, and approved by the Medical Ethics Committee of Nanchong North Sichuan Medical College Second Clinical College (Nanchong Central Hospital) (No: 2022-055 and date of approval 07.18. 2022) for studies involving humans. ## Informed Consent Statement Informed consent was obtained from all subjects involved in the study. ## Author contributions All the authors participated in the research and design. Wenwu Tang, Ying Zhang, and Xinzhu Yuan contributed to the statistical analysis. Wenwu Tang, Zhixin Wang, and Xiaoxia Chen provided these data. Wenwu Tang and Zhirui Qi participated in the data interpretation. Ju Zhang and Jie Li drafted the manuscript. Xishengxie and Xiaohua Yang provide supervision or guidance. Each author contributed important knowledge content in the process of drafting or revising the manuscript. ## Conflicts of Interest No potential conflict of interest was declared. ## Data Availability The data that support the findings of this study are available from the corresponding author upon reasonable request. ## List of supplementary materials Table S1: Single factor COX analysis Figure S1: HR values of single-factor COX regression analysis Figure S2: LASSO Regression analysis Table S2: Variable screening process Table S3: Multivariate COX analysisMulti-factor Cox model predictor distribution comparison Figure S3: Diagnosis of heart failure and construction of logistic models according to different criteria Table S4: Association of hyponatremia with heart failure hospitalization and death after adjustment for comorbidities Table S5: A logistics model with pleural effusion as an outcome event and multiple comorbidities as predictors was constructed ## Abbreviations AST : aspartate aminotransferase AAO : ascending aorta ALT : alanine aminotransferase AO : aorta or aorta root AV : aortic valve BMI : body mass index CKD : chronic kidney disease COPD : chronic obstructive pulmonary disease CVD : cardiovascular disease DCA : decision curve analysis FS : fractional shortening HF : heart failure hs-cTnT : high sensitivity troponin T IQR : interquartile ranges IVS : interventricular septum KDIGO : Kidney Disease Improving Global Outcomes LA : left atrium LASSO : Least Absolute Shrinkage and Selection Operator LV : left ventricle LVEF : left ventricular ejection fraction LVPW : left ventricular posterior wall MHD : maintenance hemodialysis MPA : main pulmonary artery MVe : anterior mitral flow velocity NT-proBNP : N-terminal B-type natriuretic peptide NYHA : New York Heart Association RA : right atrium RLS ROC : construct receiver operating characteristic RV : right ventricle TRIPOD : Transparent Reporting of a Multivariate Prediction Model for Individual Prognosis or Diagnosis * Received March 14, 2023. * Revision received March 14, 2023. * Accepted March 15, 2023. * © 2023, Posted by Cold Spring Harbor Laboratory The copyright holder for this pre-print is the author. All rights reserved. The material may not be redistributed, re-used or adapted without the author's permission. ## References 1. [1].Liyanage T, Ninomiya T, Jha V, et al. Worldwide access to treatment for end-stage kidney disease: a systematic review. Lancet. 2015; 385:1975–1982. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S0140-6736(14)61601-9&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=25777665&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F03%2F15%2F2023.03.14.23287278.atom) 2. [2].Cozzolino M, Mangano M, Stucchi A, et al. Cardiovascular disease in dialysis patients. Nephrol Dial Transplant. 2018;33(Suppl_3):iii28–iii34. [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F03%2F15%2F2023.03.14.23287278.atom) 3. [3].Wang F, Yang C, Long J, et al. Executive summary for the 2015 Annual Data Report of the China Kidney Disease Network (CK-NET). Kidney Int. 2019;95:501–505. [PubMed](http://medrxiv.org/lookup/external-ref?access_num=30784660&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F03%2F15%2F2023.03.14.23287278.atom) 4. [4].National Institutes of Health NIODaDa KD, Md. United States Renal Data System: 2018 USRDS Annual Data Report: Epidemiology of kidney disease in the United States. 2018: 71; 2018. 5. [5].Nichols GA, Ustyugova A, Déruaz-Luyet A, et al. Health care costs by type of expenditure across eGFR stages among patients with and without diabetes, Cardiovascular disease, and heart failure. J Am Soc Nephrol. 2020;31:1594–1601. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiam5lcGhyb2wiO3M6NToicmVzaWQiO3M6OToiMzEvNy8xNTk0IjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjMvMDMvMTUvMjAyMy4wMy4xNC4yMzI4NzI3OC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 6. [6].Dis AJK. USRDS. Chapter 8: Cardiovascular disease in patients with ESRD: S417–S432. 7. [7].Samanta R, Chan C, Chauhan VS. Arrhythmias and sudden cardiac death in end stage renal disease: Epidemiology, risk factors, and management. Can J Cardiol. 2019;35:1228–1240. 8. [8].Lai AC, Bienstock SW, Sharma R, et al. A personalized approach to chronic kidney disease and cardiovascular disease: JACC review topic of the week. J Am Coll Cardiol. 2021;77:1470–1479. 9. [9].Weiss G, Theurl I, Eder S, et al. Serum hepcidin concentration in chronic haemodialysis patients: associations and effects of dialysis, iron and erythropoietin therapy. Eur J Clin Invest. 2009;39:883–890. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/j.1365-2362.2009.02182.x&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=19563467&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F03%2F15%2F2023.03.14.23287278.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000269865600007&link_type=ISI) 10. [10].Ouwerkerk W, Voors AA, Zwinderman AH. Factors influencing the predictive power of models for predicting mortality and/or heart failure hospitalization in patients with heart failure. JACC Heart Fail. 2014;2:429–436. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiamhmIjtzOjU6InJlc2lkIjtzOjc6IjIvNS80MjkiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMy8wMy8xNS8yMDIzLjAzLjE0LjIzMjg3Mjc4LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 11. [11].Simpson J, Jhund PS, Lund LH, et al. Prognostic models derived in PARADIGM-HF and vlidated in ATMOSPHERE and the Swedish Heart Failure Registry to predict mortality and morbidity in chronic heart failure. JAMA Cardiol. 2020;5: 432–441. 12. [12].Bradley J, Schelbert EB, Bonnett LJ, et al. Predicting hospitalisation for heart failure and death in patients with, or at risk of, heart failure before first hospitalisation: a retrospective model development and external validation study. Lancet Digit Health. 2022;4:e445–e454. 13. [13].Gotta V, Tancev G, Marsenic O, et al. Identifying key predictors of mortality in young patients on chronic haemodialysis-a machine learning approach. Nephrol Dial Transplant. 2021;36:519–528. 14. [14].Voors AA, Ouwerkerk W, Zannad F, et al. Development and validation of multivariable models to predict mortality and hospitalization in patients with heart failure[J]. Eur J Heart Fail, 2017, 19(5): 627–634. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/ejhf.785&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F03%2F15%2F2023.03.14.23287278.atom) 15. [15].Sauerbrei W, Royston P, Binder H. Selection of important variables and determination of functional form for continuous predictors in multivariable model building. Stat Med. 2007;26:5512–5528. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/sim.3148&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=18058845&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F03%2F15%2F2023.03.14.23287278.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000252595600014&link_type=ISI) 16. [16].Pourhoseingholi MA, Baghestani AR, Vahedi M. How to control confounding effects by statistical analysis. Gastroenterol Hepatol Bed Bench. 2012;5:79–83. [PubMed](http://medrxiv.org/lookup/external-ref?access_num=24834204&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F03%2F15%2F2023.03.14.23287278.atom) 17. [17].Eom BW, Ryu KW, Nam BH, et al. Survival nomogram for curatively resected Korean gastric cancer patients: multicenter retrospective analysis with external validation. PLoS One. 2015;10:e0119671. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1371/journal.pone.0119671&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=25723182&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F03%2F15%2F2023.03.14.23287278.atom) 18. [18].Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making. 2006;26:565–574. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/0272989X06295361&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=17099194&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F03%2F15%2F2023.03.14.23287278.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000242172200001&link_type=ISI) 19. [19].Jafri L, Kashif W, Tai J, et al. B-type natriuretic peptide versus amino terminal pro-B type natriuretic peptide: selecting the optimal heart failure marker in patients with impaired kidney function. BMC Nephrol, 2013, 14: 117. 20. [20].Gheorghiade M, Abraham WT, Albert NM, et al. Relationship between admission serum sodium concentration and clinical outcomes in patients hospitalized for heart failure: an analysis from the OPTIMIZE-HF registry. Eur Heart J. 2007;28:980–988. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/eurheartj/ehl542&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=17309900&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F03%2F15%2F2023.03.14.23287278.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000246121900015&link_type=ISI) 21. [21].Khazanie P, Heizer GM, Hasselblad V, et al. Predictors of clinical outcomes in acute decompensated heart failure: acute study of clinical effectiveness of nesiritide in decompensated heart failure outcome models. Am Heart J. 2015;170:290–297. 22. [22].Marroquin MV, Sy J, Kleine CE, et al. Association of pre-ESKD hyponatremia with post-ESKD outcomes among incident ESKD patients. Nephrol Dial Transplant. 2022;37:358– 365. 23. [23].Harrison TG, Shukalek CB, Hemmelgarn BR, et al. Association of NT-proBNP and BNP with future clinical outcomes in patients with ESKD: a systematic review and meta-analysis. Am J Kidney Dis. 2020;76:233–247. 24. [24].Aimo A, Januzzi JL, Jr., Vergaro G, et al. Prognostic value of high-sensitivity troponin T in chronic heart failure: an individual patient data meta-analysis. Circulation. 2018;137:286–297. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTQ6ImNpcmN1bGF0aW9uYWhhIjtzOjU6InJlc2lkIjtzOjk6IjEzNy8zLzI4NiI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIzLzAzLzE1LzIwMjMuMDMuMTQuMjMyODcyNzguYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 25. [25].Chesnaye NC, Al-Sodany E, Szummer K, et al. Association of longitudinal high-sensitivity troponin T with mortality in patients with chronic kidney disease. J Am Coll Cardiol. 2022;79:327–336. 26. [26].Blaser MC, Kraler S, Lüscher TF, et al. Multi-omics approaches to define calcific aortic valve disease pathogenesis. Circ Res. 2021;128:1371–1397. 27. [27].Ureña-Torres P, D’marco L, Raggi P, et al. Valvular heart disease and calcification in CKD: more common than appreciated. Nephrol Dial Transplant. 2020;35:2046–2053. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/NDT/GFZ133&link_type=DOI) 28. [28].Moshar S, Bayesh S, Mohsenikia M, et al. The association of calcium-phosphorus product with the severity of cardiac valves failure in patients under chronic hemodialysis. Cardiol Res. 2016;7:80–83. 29. [29].Mitter SS, Shah SJ, Thomas JD. A Test in Context: E/A and E/e’ to Assess Diastolic Dysfunction and LV Filling Pressure. J Am Coll Cardiol, 2017, 69(11): 1451–1464. [FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6MzoiUERGIjtzOjExOiJqb3VybmFsQ29kZSI7czo0OiJhY2NqIjtzOjU6InJlc2lkIjtzOjEwOiI2OS8xMS8xNDUxIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjMvMDMvMTUvMjAyMy4wMy4xNC4yMzI4NzI3OC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 30. [30].Ding WY, Gupta D, Wong CF, et al. Pathophysiology of atrial fibrillation and chronic kidney disease. Cardiovasc Res. 2021;117:1046–1059. 31. [31].Bansal N, Xie D, Tao K, et al. Atrial fibrillation and risk of ESRD in adults with CKD. Clin J Am Soc Nephrol. 2016;11:1189–1196. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiY2xpbmphc24iO3M6NToicmVzaWQiO3M6OToiMTEvNy8xMTg5IjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjMvMDMvMTUvMjAyMy4wMy4xNC4yMzI4NzI3OC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 32. [32].Farmakis D, Chrysohoou C, Giamouzis G, et al. The management of atrial fibrillation in heart failure: an expert panel consensus. Heart Fail Rev. 2021;26:1345–1358. 33. [33].Zimmerman D, Sood M M, Rigatto C, et al. Systematic review and meta-analysis of incidence, prevalence and outcomes of atrial fibrillation in patients on dialysis. Nephrol Dial Transplant. 2012;27:3816–3822. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/ndt/gfs416&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=23114904&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F03%2F15%2F2023.03.14.23287278.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000310631500024&link_type=ISI) 34. [34].Brundel B, Ai X, Hills MT, et al. Atrial fibrillation. Nat Rev Dis Primers, 2022, 8(1): 21. 35. [35].Oikonomou E, Mystakidi VC, Tousoulis D. Hyponatremia in patients with atrial fibrillation and heart failure: The difficult triangle. Hellenic J Cardiol. 2019;60:122–123. 36. [36].Pogue J, Walter S D, Yusuf S. Evaluating the benefit of event adjudication of cardiovascular outcomes in large simple RCTs. Clin Trials. 2009;6:239–251. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/1740774509105223&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=19528133&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F03%2F15%2F2023.03.14.23287278.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000267619600005&link_type=ISI)