Noninvasive Staging of Lymph Node Status (NILS) in Breast Cancer using Machine Learning: External Validation and Further Model Development

Background Most patients diagnosed with breast cancer present with node-negative disease. Sentinel lymph node biopsy (SLNB) is routinely used to stage the axilla, leaving patients with healthy axillary lymph nodes without therapeutic effects but at risk of morbidities from the intervention. Numerous studies have developed nodal (N) status prediction models for noninvasive axillary staging using postoperative data or imaging features that are not part of the diagnostic workup. Lymphovascular invasion (LVI) is a top-ranked predictor of N metastasis; however, it is challenging to assess preoperatively.

Objective To externally validate a multilayer perceptron (MLP) model for noninvasive lymph node staging (NILS) in a large population-based register cohort (n=18 633) while developing a new N MLP in the same cohort. Data were extracted from the Swedish National Quality Register for Breast Cancer (NKBC), 2014–2017, comprising only routinely and preoperatively available documented clinicopathological variables. Furthermore, we aimed to develop and validate an LVI MLP to predict missing values of LVI status to increase the preoperative feasibility of the original NILS model.

Methods Three non-overlapping cohorts were used for model development and validation. Four N MLPs and one LVI MLP were developed using 11–12 routinely available predictors.

Three N models were used to account for the different availabilities of LVI status in the cohorts and external validation in NKBC. The fourth N model was developed for 80% of NKBC cases (n=14 906) and validated in the remaining 20% (n=3727). Three alternatives for imputing missing values of the LVI status were compared using the LVI model. The discriminatory capacity was evaluated by validation area under the receiver operating characteristics curve (AUC) in three of the N models. The clinical feasibility of the models was evaluated using calibration and decision curve analyses.

Results External validation of the original NILS model was performed in NKBC (AUC 0.699, 95% CI: 0.690-0.708) with good calibration and the potential of sparing 16% of patients with node-negative disease from SLNB. The LVI model was externally validated (AUC 0.747, 95% CI: 0.694-0.799) with good calibration but did not improve the discriminatory performance of the N models. The new N model was developed in NKBC without information on LVI (AUC 0.709, 95% CI: 0.688-0.729) with excellent calibration in the hold-out internal validation cohort, resulting in the potential omission of 24% of patients from unnecessary SLNB.

Conclusions The NILS model was externally validated in NKBC, where the imputation of LVI status did not improve the model’s discriminatory performance. A new N model demonstrated the feasibility of using register data comprising only the variables available in the preoperative setting for NILS using machine learning. Future steps include ongoing preoperative validation of the NILS model and extending the model with, for example, mammography images.

Trial Registration Registered in the ISRCTN registry with study ID ISRCTN14341750. Date of registration 23/11/2018.