Event
Guardian framework and failure dataset detect robot errors
1 reports · 1 sources
Paul Pacaud and other authors propose a framework that automatically generates robot planning and execution failure data, and on this basis construct the GuardianFail-36k failure reasoning dataset based on the RLBench simulator and the BridgeDataV2 real-robot dataset, training the multi-view reasoning vision-language model Guardian. The authors state that Guardian achieves the best performance on three unseen real-world benchmarks: RoboFail, RoboVQA, and the newly proposed UR5-Fail; when combined with large-language-model-based manipulation policies, it improves task success rates in both simulation and real deployment. The code, data, and model have been made public. These performance and success-rate improvements are claims made by the authors in the paper and await independent verification.
Report timeline
- Guardian: Detecting Robotic Planning and Execution Errors with Vision-Language Models
The study proposes an automatic framework for generating robotic planning and execution failure data, builds the GuardianFail-36k failure reasoning dataset based on RLBench and BridgeDataV2, and trains Guardian, a multi-view reasoning vision-language model. Guardian achieves state-of-the-art performance on three unseen real-world benchmarks: RoboFail, RoboVQA, and the newly introduced UR5-Fail. When combined with LLM-based manipulation policies, it improves task success rates in both simulation and real-world deployment. Code, data, and models are publicly available.
arXiv cs.RO ·