Event

GPT-6 Astra Robot Manipulation Evaluation and New Benchmark Release

2 reports · 2 sources

Youwei Zhou et al. used a lightweight agent execution framework to evaluate the closed-loop robot manipulation capabilities of GPT-6 Astra, finding a success rate of 26.92% on RoboDojo and 76.0% on RoboTwin 2.0, but there is a clear gap between semantic reasoning and physical execution, especially in consequence-aware action planning, precision, and complex motion coordination. Real experiments show that simple tasks can be executed zero-shot, but motion feasibility, precise control, and physical safety remain persistently insufficient, and relatively large inference latency also hinders practical deployment. To this end, the authors propose a new benchmark, LIBERO-GeoMotion, comprising geometrically constrained tasks that require precise pose alignment, coordinated translation and rotation, and anticipation of the physical consequences of actions. A measured report on SenseTime Astra cited by an OFweek commentary article gives different numbers: 100% on LIBERO-Long++ and 81.9% on RoboDojo, and states that stacking bowls on a real robot took an average of about 131 minutes and cost about 19 USD, exposing an architectural breakpoint between second-level cloud inference latency and millisecond-level force-feedback closed loops.