Quantifying absolute treatment effect heterogeneity for time-to-event outcomes across different risk strata: divergence of conclusions with risk difference and restricted mean survival difference

Background Risk-based analyses are increasingly popular for understanding heterogeneous treatment effects (HTE) in clinical trials. For time-to-event analyses, the assumption that high-risk patients benefit most on the clinically important absolute scale when hazard ratios (HRs) are constant across risk strata might not hold. Absolute treatment effects can be measured as either the risk difference (RD) at a given time point or the difference in restricted mean survival time (ΔRMST) which aligns more closely with utilitarian medical decision-making frameworks. We examined risk-based HTE analyses strata in time-to-event analyses to identify the patterns of absolute HTE across risk strata, and whether ΔRMST may lead to more meaningful treatment decisions than RD.

Methods Using artificial and empirical time-to-event data, we compared RD—the difference between Kaplan-Meier estimates at a certain time point—and ΔRMST—the area between the Kaplan-Meier curves—across risk strata and show how these metrics can prioritize different subgroups for treatment. We explored scenarios involving constant HRs while varying both the overall event rates and the discrimination of the risk models.

Results When event rates and discrimination were low, RD and ΔRMST increased monotonically, with high-risk patients benefitting more than low-risk patients. As the event rate increased and/or discrimination increased: 1) a “sweet spot” pattern emerged: intermediate-risk patients benefit more than low-risk and high-risk patients; and 2) RD understates the benefit in high-risk patients.

Conclusions The pattern of HTE characterized by RD may diverge substantially from ΔRMST, potentially leading to treatment mistargeting. Therefore, we recommend ΔRMST for assessing absolute HTE in time-to-event data.

Key messages

1. To quantify absolute heterogeneous treatment effect (HTE) in time-to-event data, the difference in restricted mean survival time (ΔRMST) is more intuitive and comprehensive, less dependent on the time horizon, and better captures HTE when the hazard ratio (HR) of treatment varies over time, compared to the risk difference (RD).

2. We examined risk-based HTE analyses in time-to-event analyses to identify the patterns of absolute HTE across different risk strata, and whether ΔRMST may lead to more meaningful treatment decisions than RD.

3. Even with a constant HR, intermediate-risk patients may benefit more than low-risk and high-risk patients as event rates increase, a phenomenon known as a “sweet spot” pattern.

4. The RD does not accurately reflect the benefit for high-risk patients when event rates and/or discrimination of the risk model are high, unlike to the ΔRMST.

5. We recommend the ΔRMST for assessing absolute HTE, as the RD may potentially lead to treatment mistargeting.