Deep learning analysis for enhanced prediction of heat transfer in Maxwell hybrid nanofluids with non-Fourier law and radiation effects

Abstract Nanofluids have emerged as promising heat transfer media due to their superior thermal performance associated to conventional fluids, yet their behavior under mutual impacts of radiation, heat source/sink, and magnetic fields remains an active area of research. In this study, movement and heat transfer characteristics of a Maxwell-based hybrid nanofluid containing carbon nanotubes are examined over a stretching/shrinking sheet with slip boundary conditions and Cattaneo–Christov heat flux. Governing nonlinear partial differential equations (PDEs) are reduced through similarity transformations and resolved numerically by means of MATLAB bvp4c routine. Numerical outcomes are analyzed for the effects of important parameters such as magnetic field strength, Biot number, velocity slip, and thermal radiation over velocity and temperature distributions. For the investigated parameter ranges (magnetic parameter 0.1–1.2, radiation parameter 0.3–0.9, and Biot number 0–0.8), thermal radiation and Biot number are found to enhance heat transfer rates, while increasing magnetic effects suppress the velocity field. The hybrid nanofluid consistently exhibits improved thermal performance compared to the corresponding nanofluid case. To complement numerical framework, artificial neural networks (ANNs) are implemented as surrogate predictive models. A multilayer perceptron structure qualified with Levenberg–Marquardt algorithm (LMA) is employed, with datasets from bvp4c distributed into 70% for training, 15% for validation, and 15% for testing. ANN demonstrates outstanding predictive performance, achieving mean squared error (MSE) values as low as $$\,6.9182 \times 10^{ - 12} \,$$ for velocity gradients and $$\,1.4653 \times 10^{ - 11} \,$$ for temperature profiles, with correlation coefficients of $$\,R \approx 1\,$$ across all training, validation, and test subsets. These results confirm that ANN models can accurately reproduce the nonlinear boundary-layer solutions with negligible error. The combined numerical–ANN approach establishes a novel and computationally efficient framework for analyzing hybrid nanofluid flows. The outcomes highlight the strong influence of magnetic and thermal parameters on stream and heat transfer features and demonstrate ability of ANNs to serve as reliable predictive models for complex nonlinear systems.