Event
ArtifactArena releases a robot physics evaluation platform
1 reports · 1 sources
Kushagra Tiwary and other authors have released ArtifactArena, an open evaluation platform that has models complete hardware-software co-design in a physics simulation arena: build working robots and have them compete against each other. The platform guides models toward improving robot artifacts through three means: text descriptions, physics simulator feedback, and battle data, and it compares frontier models using Elo rankings produced by head-to-head tournaments. The authors have also open-sourced the framework and tournament infrastructure for ongoing community submissions, with the goal of forming an unsaturated long-term testbed.
Report timeline
- ArtifactArena: Evaluating models by the robots they build in the physical world
ArtifactArena is an open-ended evaluation platform that challenges models with hardware-software co-design: building working robots in a simulated arena and pitting them against each other. The platform guides models to improve their robot artifacts through three channels: text descriptions, physics simulator feedback, and gameplay data, and compares frontier models using Elo rankings from head-to-head tournaments. The authors are releasing the framework and tournament infrastructure for ongoing community submissions, creating a non-saturating long-term testbed.
arXiv cs.RO ·