Event
SMPC Demonstrations with Sparse Reward Training for Robot Locomotion-Manipulation
1 reports · 1 sources
Martin Schuck and co-authors propose a method for learning locomotion-manipulation policies: in simulation, sampling-based model predictive control (SMPC) is used as an automatic expert to generate a large-scale offline dataset, which is then used to train an off-policy reinforcement learning agent with purely sparse task rewards, thereby eliminating the need for manually designed dense rewards. The authors report that, by combining a high-level agent with a low-level dynamic stabilization controller, the learned policy can surpass the original optimal-control teacher on task objectives. The authors report having completed sim-to-real deployment on a Spot quadruped robot equipped with a robotic arm and on a G1 humanoid robot, validating complex locomotion-manipulation skills across morphologies. The paper is the CoRL 2026 accepted version.
Report timeline
- Learning Loco-Manipulation From SMPC Demonstrations With Sparse Offline-to-Online RL
This work proposes using sample-based model predictive control (SMPC) in simulation as an automated expert to generate large-scale offline datasets, then training an off-policy reinforcement learning agent with purely sparse task rewards, eliminating the need for manual dense reward design. Combined with a low-level dynamic stability controller, the high-level agent's learned policies can exceed the original optimal control teacher on task objectives. The authors completed sim-to-real deployment on an arm-equipped Spot quadruped and a G1 humanoid; the paper is the accepted version for CoRL 2026.
arXiv cs.RO ·