Event

Frontier AI Model LIBERO-10 Manipulation Evaluation: GPT-6 Astra Leads with 44% Success Rate

1 reports · 1 sources

May Espinola, Qifan Zhang, and Yu Xiang propose a unified closed-loop manipulation evaluation framework, evaluating six frontier AI models on the LIBERO-10 benchmark, with a total of 18 configurations and 1,800 evaluation episodes. Results show that GPT-6 Astra achieved the highest success rate at 44%, followed by Gemini-4 Argon at 43%. The framework allows models to directly control the robot through visual observations, robot proprioception, and structured tool calls, primarily specifying end-effector poses and gripper commands, without task-specific fine-tuning or pre-trained manipulation skills. The study states that increasing reasoning intensity generally improves manipulation performance, but the gains vary by model and increase reasoning costs; the evaluation framework and experimental results have been open-sourced.