Event

Frozen frontier VLM used directly as robot policy achieves 92.1% success rate on RoboMME

1 reports · 1 sources

The paper reports that the frozen frontier vision-language model GPT-6-Astra, used directly as a robot policy, achieved a 92.1% success rate over 800 test episodes on the RoboMME benchmark, which contains 16 memory-dependent manipulation tasks. This is higher than the previously reported human score of 90.5% (that human result was obtained using an oracle skill interface), and also higher than the best leaderboard entry of 60.8% and the 79.1% achieved by a hierarchical system pairing the same model with a RoboMME-trained policy. The model retains the full episode context throughout and directly outputs each end-effector waypoint and gripper command, without any training specific to this benchmark; it reached or exceeded the human score on 13 of the 16 tasks, and its 63 failures were concentrated in two tasks, mostly premature completion and mistimed or unintended button presses.