Event

RoboColiseum Releases Four-Model Embodied Evaluation: Opus Tops

2 reports · 2 sources

Da Huang et al. conducted a cross-model embodied capability evaluation of four frontier multimodal large models—GPT-6 Astra, GPT-6.1 Sol, Claude Opus 5.5, and Gemini 4 Argon—on RoboColiseum, and proposed the Agent2Act control framework, which connects the four models to the evaluation environment through shared observation, action execution, and feedback interfaces. The evaluation covers four tracks: instruction following, spatial reasoning, general manipulation, and robustness. The paper states that under unified task settings and scoring criteria, the four models exhibit consistent capability profiles; in the configurations evaluated, Astra leads in instruction following and robustness, while Opus leads in spatial reasoning, general manipulation, and overall performance. According to the total scores reported by Machine Heart on October 22, Opus ranks first with 76.28 points, followed closely by Astra with 75.99 points, while Sol and Argon score 70.91 and 66.86 points, respectively; the top two on the official overall leaderboard remain the specialized embodied policy models Paimon (77.54 points) and HiDream-O1-Embodied (76.92 points). The report states that Astra, Sol, and Opus used the high setting, Argon used medium, while the specialized policy models retained their own training and execution settings and did not adopt the same control scheme. The paper additionally uses Astra for a case study examining whole-body coordination, cross-embodiment manipulation, fine manipulation, dexterous hand manipulation, and real-robot tasks; Machine Heart states that this case comes from supplementary exploration outside the formal leaderboard submission and is not counted in the leaderboard comparison of the four models using shared original instructions.