Event
Rollback: World Model Rollback Improves Robot Policies
1 reports · 1 sources
Jiayi Luo and other authors propose the Rollback policy improvement paradigm, which lets the world model roll back to key decision points, branch out to try different actions, and compare the outcomes, distilling counterfactual results into action-level supervision without the need to separately learn a value model. The world model serves as a training environment, while the real trajectories produced by the policy trained with it are used to correct the model dynamics through a lightweight LoRA procedure. On three real tasks—block extraction, ball throwing, and tabletop curling—the success rates improve by an absolute 10.67%, 11.11%, and 11.85%, respectively, over the strongest baseline. The authors state that although the initialized teleoperation demonstrations are far below human level, Rollback surpasses the average success rate of human players on the ball-throwing and tabletop-curling tasks within 8 hours of training.
Report timeline
- Rollback: Using World-Model Experience Backtracking to Improve Robot Policies
Rollback proposes a policy-improvement paradigm in which a world model rewinds to critical decision points, branches to try different actions, and compares the outcomes, distilling counterfactual results into action-level supervision without a separately learned value model. On three real-world tasks—block extraction, ball throwing, and tabletop curling—success rates improved by 10.67%, 11.11%, and 11.85% in absolute terms over the strongest baseline, and ball throwing and tabletop curling surpassed the average human player success rate within 8 hours of training.
arXiv cs.RO ·