Event

MolmoSpaces-CommonSense robot commonsense reasoning benchmark proposed

1 reports · 1 sources

Arjun Guru, Max Argus, Jae Sung Park, Jiafei Duan, Ali Farhadi, and Ranjay Krishna propose MolmoSpaces-CommonSense, a commonsense reasoning benchmark for robot manipulation, and build a reasoning taxonomy based on developmental cognitive science, examining commonsense reasoning such as occlusion, containers, support relations, semantically correct grasp points, spatial affordances, and hidden objects. The authors use this benchmark to evaluate several zero-shot general manipulation policies, and report that: as distractors increase, the success rate in cluttered scenes drops by more than 50%; policies act on the wrong object about 30% of the time in cluttered environments; and the proportion of correctly selecting object parts is only about 50%. These numbers come from the authors' evaluation of the above policies and reflect the performance of the tested policies on this benchmark.