Event
RoboReason proposes spatiotemporal ordering benchmark and data generation method for robot VLMs
1 reports · 1 sources
Sagnik Anupam and co-authors propose RoboReason, which uses images from real robot stereo cameras and wrist cameras to construct spatial and temporal ordering tasks, evaluating the robot-related reasoning capabilities of vision-language models. Experiments show that several advanced VLMs perform poorly on the ordering tasks. The benchmark contains three types of questions: ordering shuffled frames according to a natural-language description of the task, depth-ordering objects by distance from the camera, and ordering objects from left to right by pixel coordinates; the authors state that task complexity can be adjusted by changing the size of the set to be ordered. Based on this, the authors propose a scalable autonomous training-data generation pipeline, and state that both supervised fine-tuning and reinforcement learning on this data significantly improve model performance on spatiotemporal ordering and downstream robot and visual reasoning tasks. The work was accepted to the EMNLP 2026 main conference.
Report timeline
- RoboReason: Improving robotics-relevant VLM reasoning with scalable data generation
RoboReason proposes constructing spatial and temporal ordering tasks from real robot stereo camera and wrist camera images to evaluate the robotics-relevant reasoning capabilities of vision-language models; experiments show that several advanced VLMs perform poorly on the ordering tasks. The authors accordingly propose a scalable autonomous training data generation pipeline, and supervised fine-tuning and reinforcement learning on this data both significantly improve model performance on spatiotemporal ordering and downstream robotics and visual reasoning tasks. The work was accepted to the EMNLP 2026 main conference.
arXiv cs.RO ·