Event
EmbodiedLab Releases an Experimental Benchmark for Embodied Agents
2 reports · 2 sources
Hongcheng Gao and other authors have released the EmbodiedLab benchmark and evaluation code, which requires robots to "learn by doing" in a physics-simulation laboratory: agents receive a goal and experimental apparatus, choose the experimental procedure themselves, read instruments via camera, and are ultimately evaluated by the benchmark on the knowledge they acquire and the physical state required by the task. The benchmark covers 30 quantitative science experiments across 5 disciplines on 14 experimental apparatus platforms. The authors state that the system parameters of each instance are resampled, so results cannot be known in advance, yet can be scored precisely; all experiments have been verified to be solvable. The benchmark and evaluation code have been made public.
Report timeline
- EmbodiedLab releases embodied agent benchmark: letting robots "act to know" in physics experiments
EmbodiedLab is a benchmark that requires robots to "act to know": the agent receives an objective and experimental apparatus, chooses its own experimental procedure, reads instruments via camera in a physics simulation laboratory, and is evaluated on the knowledge it ultimately gains as well as the physical state required by the task.
arXiv cs.RO ·
- EmbodiedLab releases embodied agent benchmark: robots must acquire knowledge by operating experimental apparatus
EmbodiedLab is a benchmark that requires robots to "acquire knowledge through action," covering 30 quantitative science experiments across 5 disciplines on 14 apparatus platforms. The parameters of each instance are resampled, so results cannot be known in advance but can be scored precisely.
HuggingFace Daily Papers ·