Event
RoboWorM: An Embodied Video World Model Evaluation Benchmark Released
1 reports · 1 sources
Doanh Le Thien and co-authors propose and release RoboWorM, a deployment-aware evaluation benchmark for robotic manipulation video world models, which uses hardware priors such as the robot kinematic tree and fixed camera configurations to examine the structural and spatiotemporal fidelity of generated videos. The benchmark brings together 1,605 clips across single-arm, dual-arm, and humanoid embodiments, covering Markov and long-horizon non-Markov tasks as well as single-view and multi-view settings. The authors also propose two deterministic metrics that do not rely on vision-language models: KIVA measures kinematic stability within a single view, and RIGIS measures geometric consistency across multiple views. The authors report that, after evaluating 9 acceleration techniques on 3 mainstream world models and generating more than 29,000 rollouts, these two metrics enable up to 2.3x inference speedup with very little quality loss.
Report timeline
- RoboWorM: A Benchmark for Evaluating Motion Realism and Acceleration Effects in Embodied Video World Models
The authors propose RoboWorM, a deployment-aware benchmark for robotic manipulation video world models that examines structural and spatiotemporal fidelity using hardware priors such as robot kinematic trees and fixed camera configurations. The benchmark collects 1,605 episodes across single-arm, bimanual, and humanoid morphologies, covering Markovian and long-horizon non-Markovian tasks under single-view and multi-view setups, and proposes two deterministic metrics that do not rely on vision-language models: KIVA and RIGIS.
arXiv cs.RO ·