Event
HATCH: A Sensor-Scored Benchmark for Long-Horizon Robotic Manipulation Released
1 reports · 1 sources
Shiza Charania proposed and released HATCH, a robot escape-room-style benchmark for evaluating long-horizon manipulation, which has been accepted to the 5th Open Robot Software Workshop at IROS 2026. The benchmark has an open specification and is physically reproducible; its tasks span five skill categories and three difficulty levels, and the sensor-confirmed result of the previous task unlocks the next task, chaining the entire room into a single long-horizon evaluation. HATCH's success determination is confirmed by sensor instrumentation rather than human judgment, scoring is independent of the robot body, and object placement is determined by a random seed on the day of the run. Participants submit certified self-test runs (unedited video, sensor logs, and seed), and calibration events will be organized from time to time. Task specifications, build guides, bills of materials, and scoring rules are all released under an open license.
Report timeline
- HATCH: A Hardware-Attested Task-Chain Benchmark for Verifiable Manipulation
HATCH is an open-specification, physically replicable, sensor-scored, and unrehearsable robot manipulation benchmark, accepted to the 5th Open Robot Software Workshop at IROS 2026 (2 pages, 1 table). Tasks span five skill categories and three difficulty levels, and a sensor that confirms completion of one task unlocks the next, chaining an entire room into a single long-horizon evaluation.
arXiv cs.RO ·