Joint synthesis of longitudinal MRI and enhanced prediction of neoadjuvant chemotherapy response in breast cancer: a multicohort study

Abstract Neoadjuvant chemotherapy (NAC) induces critical morphological and vascular changes in breast cancer tumors, yet existing deep learning techniques in medical imaging tend to neglect these longitudinal changes throughout the treatment course. Here, we propose simultaneously synthesizing early-NAC MR images and improving the prediction of therapeutic responses from preoperative MRI. A cycle generative adversarial network (GAN)-based Longitudinal Image Synthesis and Prediction Convolutional Network (LISPCN) was developed by decoding preoperative MR images into a latent feature vector and then encoding it to jointly synthesize early-NAC DCE-MR images and predict therapeutic responses. This approach balances the shared information between the two tasks, thereby enhancing the overall model performance. We developed a specialized semantic and perceptual loss optimized for the multitask learning architecture, adopting an Efficient Channel Attention (ECA) module into the generators to enhance the model’s discriminative ability. Our LISPCN model surpassed others in image quality, achieving a structural similarity index of 0.909 and a peak signal-to-noise ratio of 34.995. For the internal validation dataset, the model achieved an AUC of 0.903, outperforming the model using solely preoperative image. For the external validation dataset, it achieved an AUC of 0.832. Model visualization demonstrates an increased focus on the lesion, indicating its capability to learn longitudinal changes during treatment. By leveraging intermediate NAC information, the LISPCN model enables the generation of early-stage NAC images from preoperative images, serving as a supportive tool to monitor treatment progress. This method has significant clinical implications for personalized diagnostic and treatment decisions in breast cancer patients.