Event
OSMA-Bench and SSI Evaluate Home Robot Obstacle Avoidance Safety
1 reports · 1 sources
Hyunwoong Bae and other authors propose the OSMA-Bench benchmark and the Semantic Sensitivity Index (SSI) to test whether home robots adjust spacing and speed according to obstacle type during navigation. The benchmark's scenario sets are spatially matched, varying only the obstacle type. The authors evaluated vision-language model (VLM)- and vision-language-action (VLA)-based policies on the benchmark and found that even when given explicit safety prompts, neither consistently adjusted motion according to obstacle safety levels.
Report timeline
- OSMA-Bench: Evaluating Safety Under Obstacle Conditions in Household Robot Navigation
The authors propose OSMA-Bench, a benchmark that uses spatially matched scenario sets differing only in obstacle type to test whether robot navigation adjusts clearance and speed according to obstacle semantics, and they propose the Semantic Sensitivity Index (SSI) to measure this adaptation. Evaluating VLM and vision-language-action (VLA) policies on the benchmark, they find that even with explicit safety prompts, neither consistently adjusts its motion according to obstacle safety levels.
arXiv cs.RO ·