Risk-Assessment and Visual Perception of Autonomous Microrobots Using Model-Free Reinforcement Learning to Cope With Multi-Challenging Environments

Autonomous microrobotic navigation in life sciences is highly challenging due to environmental disturbances, unpredictable phenomena, diverse obstacles, and dynamic changes. These significantly limit motion of microrobots manipulated by remote magnetic fields. While conventional control, including trajectory planning strategies, is feasible when dynamic models of environments are known, they lack robustness in highly complex scenarios. Herein, we present a modern control based on model-free reinforcement learning (MFRL: SAC algorithm) combined with visually augmented perception. This is implemented into Electromagnetic Actuation System to autonomously manipulate microrobots in dynamic challenges. As physical sensors cannot be embedded within the robots, sensing is enhanced via virtual tentacles that mimic proximity sensors to detect in-range obstacles. Meanwhile, the system processes environmental feedback—such as angular deviation, motion cues, and distances—as feedback to perform diverse actions, including trajectory planning, obstacle avoidance, and self-navigation. Experiments were conducted across three levels of unpredictable environments featuring moving and chasing obstacles. These tests evaluate the microrobot’s real-time decision-making and its ability to reach a target despite restricted motion. A key advantage of this control technique is that, although the RL policy is trained in simulation, it is seamlessly transferred to the physical control system without tuning parameters. The results demonstrate that the microrobot can effectively cope with unseen environments, providing a practical and robust approach for operating in uncertain biological conditions.