Event

Researchers propose SO-101 Bench simulation benchmark

1 reports · 1 sources

Researchers propose SO-101 Bench, a language-conditioned spatial manipulation simulation benchmark based on an Isaac Lab digital twin of a real SO-101 setup, comprising four task categories, 138 objects, 916 evaluation episodes, and 3,629 simulated demonstrations. The researchers state that existing SO-101 benchmarks run only on physical hardware, with simple tasks and few held-out objects. All learning policies are fine-tuned on this benchmark with the same compute budget. The researchers state that simulation overestimates success rates, with the best-performing policy scoring close to 61%, and that spatial placement, unseen object categories, and rotational assembly planning remain unsolved problems.