Event
Authors propose WorldFailBench, a benchmark for evaluating policy failure in world models
1 reports · 1 sources
Zheng Zhang and other authors propose and release WorldFailBench, an evaluation benchmark for testing whether world models can faithfully simulate policy failures. The benchmark sets three tasks—failure cause, failure stage, and failure boundary—covering manipulation tasks in the LIBERO and RoboTwin simulation environments, as well as real-world manipulation tasks on the G1 humanoid robot. The authors state that experiments on seven world models show these models have notable gaps in understanding policy failures, and that this result can provide a reference for developing more reliable world models for policy evaluation.
Report timeline
- WorldFailBench: Can World Models Understand Policy Failures?
The authors propose WorldFailBench, which uses three tasks—failure reason, failure stage, and failure boundary—to test whether world models faithfully simulate policy failures. The benchmark covers manipulation tasks in the LIBERO and RoboTwin simulation environments, as well as real-world manipulation tasks with the G1 humanoid robot. Experiments on seven world models show clear gaps in their understanding of policy failures.
arXiv cs.RO ·