Mixture of Experts Framework Based on Soft Actor-Critic Algorithm for Highway Decision-Making of Connected and Automated Vehicles

Abstract Decision-making of connected and automated vehicles (CAV) includes a sequence of driving maneuvers that improve safety and efficiency, characterized by complex scenarios, strong uncertainty, and high real-time requirements. Deep reinforcement learning (DRL) exhibits excellent capability of real-time decision-making and adaptability to complex scenarios, and generalization abilities. However, it is arduous to guarantee complete driving safety and efficiency under the constraints of training samples and costs. This paper proposes a Mixture of Expert method (MoE) based on Soft Actor-Critic (SAC), where the upper-level discriminator dynamically decides whether to activate the lower-level DRL expert or the heuristic expert based on the features of the input state. To further enhance the performance of the DRL expert, a buffer zone is introduced in the reward function, preemptively applying penalties before insecure situations occur. In order to minimize collision and off-road rates, the Intelligent Driver Model (IDM) and Minimizing Overall Braking Induced by Lane changes (MOBIL) strategy are designed by heuristic experts. Finally, tested in typical simulation scenarios, MOE shows a 13.75% improvement in driving efficiency compared with the traditional DRL method with continuous action space. It ensures high safety with zero collision and zero off-road rates while maintaining high adaptability.