Uncertainty-Aware Machine Learning for Predicting Axillary Lymph Node Metastasis Using Breast MRI

Sevgi Ünal,1 Remzi Gürfidan2 1Department of Radiology, Izmir Katip Celebi University Ataturk Training and Research Hospital, Izmir, Türkiye; 2Database, Network Design and Management, Isparta University of Applied Science, Isparta Vocational School of Information Technologies, Isparta, TürkiyeCorrespondence: Sevgi Ünal, Email sevgiunal84@gmail.comPurpose: Axillary lymph node metastasis (ALNM) is a significant prognostic factor in breast cancer and has an impact on staging, treatment and survival. The objective of this study is to create a machine learning model that will be able to predict axillary lymph node metastasis (ALNM) in a preoperative setting using breast MRI-derived and clinicopathological variables, while also achieving probability calibration and uncertainty-aware prediction for more reliable risk estimates.Patients and Methods: For this retrospective single-centre study, 204 patients who underwent contrast-enhanced breast MRI from 2021 to 2024 were selected. The dataset comprised of 23 independent variables, respectively, representing demographic, clinical, radiological, histopathological, molecular characteristics along with a binary target variable indicating ALNM status. The data was split into 60% called training set, 20% calibration set, and 20% test set. Candidate models were evaluated based on ROC-AUC on the training subset that was used for screening. Subsequently, calibration was performed in a held-out calibration set, following which class-conditional conformal prediction was applied to quantify predictive uncertainty.Results: Out of all the models we evaluated, the Conformal-Calibrated Interpretable Risk Model (CCIRM) was found to be the best model, achieving a test accuracy of 0.9268, a weighted F1 score of 0.9270 and AUC of 0.937. With a precision of 0.9545 and a recall of 0.9130 in the ALNM-positive class, the model was potent. In addition to discrimination strength, the framework produces calibrated risk estimates and uncertainty-aware prediction sets that enable transparent interpretation of model outputs in clinically borderline cases.Conclusion: The proposed approach includes the model selection based on the machine learning and probability calibration, as well as conformal prediction to achieve reliability, beyond label prediction, and uncertainty-aware risk estimation for preoperative ALNM assessment. The results from this analysis suggest that CCIRM is a potential methodological framework for trustworthy clinical decision support, although the prospective and multicentre external validation should be done before it becomes applicable to the clinical setting.Keywords: axillary lymph node, breast MRI, interpretable machine learning, conformal prediction, breast cancer