DP-FedHybrid: A Differentially Private Federated Stacking Framework Integrating Tree-Based and Transformer Models for Secure Heart Disease Prediction

Heart disease is rapidly increasing. For early and accurate prediction, advanced machine learning (ML) models are needed. However, the sensitive nature of clinical data, along with strict regulatory constraints, creates significant challenges to traditional centralized learning systems. Federated learning (FL) has emerged as a decentralized paradigm for collaborative model training, enabling multiple clients to jointly learn a global model without exchanging raw data. However, various existing FL frameworks rely on single-model architectures. They struggle to capture complex feature interactions in tabular healthcare data. To address these limitations, we propose a novel DP-FedHybrid, differentially private federated stacking framework that integrates heterogeneous models within a secure and decentralized learning architecture. We use a multi-client FL setup, where each client independently trains CatBoost and Transformer models in parallel, and their predictive outputs are combined through a stacking-based meta-learning mechanism. The Transformer component is optimized using differentially private stochastic gradient descent (DP-SGD), incorporating gradient clipping and calibrated Gaussian noise injection to ensure formal privacy guarantees. To further strengthen, we use a stacking-based meta-learning layer that aggregates probabilistic outputs from client-side models. It enables effective knowledge fusion and enhances robustness and generalization under non-independent and identically distributed (non-IID) data. The proposed framework is evaluated on a benchmark heart disease dataset, where we obtained an accuracy of 95.12% under standard FL and 94.15% under DP constraints, outperforming closely related works. The proposed work advances the existing literature by providing a scalable, hybrid, and privacy-preserving FL paradigm for heart disease prediction.