Event
AtomicRL improves VLA policy reinforcement learning post-training with atomic task structure
1 reports · 1 sources
Likui Zhang et al. propose AtomicRL, a reinforcement learning post-training framework for long-horizon robotic manipulation that leverages atomic task structure to improve credit assignment and exploration in vision-language-action (VLA) policies. The paper reports that on LIBERO-LONG, AtomicRL achieves absolute performance improvements of 3.6% and 4.0% over PPO on two VLA architectures; on CALVIN ABC-D, full-task performance improves by up to 9.5% over PPO; and when combined with a Real2Sim2Real pipeline, real-world task performance improves by an average of 21.3%. The framework consists of two parts: Semantic-Aware Policy Optimization (SAPO) uses semantic gating to prevent TD residuals from failed atomic subtasks from propagating across tasks, thereby mitigating credit contamination; and Bottleneck-Adaptive Targeted Exploration (BATE) dynamically coordinates task sampling and local exploration incentives based on the mastery level of each subtask.
Report timeline
- AtomicRL: An Atomic Reinforcement Learning Post-Training Framework for Long-Horizon Robotic Manipulation
The paper proposes AtomicRL, a framework that uses atomic task structure to improve reinforcement learning post-training of vision-language-action (VLA) policies. Its core components include Semantic-Aware Policy Optimization (SAPO), which uses semantic gating to prevent TD residuals from failed atomic subtasks from propagating across tasks, thereby mitigating credit contamination; and Bottleneck-Adaptive Target Exploration (BATE), which dynamically coordinates task sampling and local exploration incentives based on the mastery level of each subtask.
arXiv cs.RO ·