DriveEmo-FL: in-cabin radar-based emotion sensing for autonomous vehicles smart response

Abstract DriveEmo-FL presents a privacy-preserving, radar-based emotion-recognition framework tailored for autonomous-vehicle (AV) cabins. Leveraging a compact embedded mmWave radar, the system captures upper-body emotional gestures using a CFAR-enhanced preprocessing pipeline, extracting both micro-Doppler signatures and velocity-time profile (VTP) features. These features are processed via EmoNet, a lightweight dual-stream deep learning model that performs early fusion of spatial-temporal and motion statistics. EmoNet achieves a top classification accuracy of 94.5% (Precision: 0.945, Recall: 0.943, F1-Score: 0.944) with an average latency of 9.7 ms on edge hardware. DriveEmo-FL’s effectiveness is validated through extensive testing across four real-world vehicular scenarios: motion, low light, direct sunlight, and gesture overlap, demonstrating robust performance under diverse conditions. Additionally, we incorporate federated learning to preserve passenger privacy and enable model generalization across multiple AV fleets without sharing raw data. Comparative evaluation against six state-of-the-art models confirms EmoNet’s superiority in both accuracy and computational efficiency. By linking emotional state detection to adaptive AV behaviors, DriveEmo-FL offers a proactive, intelligent interface for future emotion-aware intelligent transportation systems.