SAC-Based Gait Generation and Robust Balance Control for Cassie Robots in Unknown Terrain

Abstract

The development of robust locomotion strategies for bipedal robots remains a significant challenge in robotics, particularly when navigating unstructured and unknown environments. The Cassie robot, a dynamic bipedal platform with high degrees of freedom and underactuated passive dynamics, presents specific control difficulties that classical model-based approaches often fail to address adequately due to modeling mismatches and computational latency. This paper proposes a novel framework utilizing Soft Actor-Critic (SAC), an off-policy deep reinforcement learning algorithm, to generate stable gait patterns and ensure robust balance control. Unlike standard on-policy methods, SAC optimizes a maximum entropy objective, which encourages substantial exploration and provides greater robustness to external disturbances. We introduce a comprehensive reward function design and a domain randomization strategy that enables the policy to generalize across varying terrain irregularities without requiring exteroceptive mapping during the training phase. Extensive simulation results demonstrate that the proposed SAC-based controller outperforms baseline algorithms in terms of convergence speed, energy efficiency, and stability on uneven terrain. The learned policy exhibits emergent behaviors capable of recovering from significant perturbations, suggesting a promising pathway for deploying autonomous bipedal systems in real-world scenarios.