Event
WorldOracle example: successful rollout may misjudge grasping
1 reports · 1 sources
WorldOracle presents a case: the real gripper did not grasp the fruit, but the generated rollout shows the fruit being picked up, even though the model was given the same initial observation and recorded actions. This case shows that a seemingly successful predicted video is not necessarily correct. Based on this, WorldOracle proposes that evaluation of predictions should separately check whether the robotic arm follows the trajectory and whether the object's final state is correct; a failed grasp action predicting a failed grasp is the correct answer.
Report timeline
- WorldOracle example: A seemingly successful robot rollout may predict the wrong grasp
WorldOracle presents a case where the real gripper failed to grasp the fruit, but the generated rollout showed the fruit being picked up, despite the model receiving the same initial observation and recorded actions. When evaluating predictions, check separately whether the robotic arm follows the trajectory and whether the object's final state is correct; for an empty grasp, predicting an empty grasp is the correct answer, while a seemingly successful grasp video may actually be wrong.
Reddit r/robotics ·