Event
Release of the Embodied Agent Arena evaluation benchmark
1 reports · 1 sources
Haojian Huang and other authors released the Embodied Agent Arena evaluation benchmark and evaluated ten VLM agents across five dimensions: geometry, spatial reasoning, affordance, task planning, and manipulation. The benchmark contains a total of 1,000 cases, composed of 32 existing sources and GeoProbe, which the authors built themselves; GeoProbe contains 168 cases and is used for camera, motion, depth, and scale estimation. Evaluation uses a minimal test framework, handing task decisions to the models through each source's native interface.
Report timeline
- Can Frontier VLM Agents Handle General Robot Tasks? An Empirical Study with Embodied Agent Arena
The study proposes Embodied Agent Arena, evaluating ten VLM agents across five dimensions—geometry, spatial reasoning, affordance, task planning, and manipulation—with 1,000 cases, including the 168-case GeoProbe benchmark for camera, motion, depth, and scale estimation.
arXiv cs.RO ·