Event
Release of RoboCasa-Shift: A Phase-Aware Robustness Benchmark
1 reports · 1 sources
Haocheng Wang et al. propose RoboCasa-Shift, a diagnostic robustness benchmark built on RoboCasa365. Using controlled single-factor perturbations, paired clean-perturbed rollouts, and simulation phase-aware diagnostics, they completed 1,040 formal rollout evaluations of GR00T N1.5 on four long-horizon household manipulation tasks that passed a clean-performance threshold, covering four perturbation factors and three severity levels; some conditions caused the evaluated policy to fail completely. The paper reports that robustness performance varies significantly across perturbation factors and tasks, and that increasing severity does not always lead to a monotonic decline in performance.
Report timeline
- RoboCasa-Shift: A Stage-Aware Robustness Evaluation Benchmark for VLA Policies in Long-Horizon Household Manipulation
The paper proposes RoboCasa-Shift, a diagnostic robustness benchmark built on RoboCasa365 that combines controlled single-factor perturbations, paired clean-perturbed rollouts, and simulator-grounded stage-aware diagnostics, completing a total of 1,040 rollouts evaluating GR00T N1.5 across four long-horizon tasks, four perturbation factors, and three severity levels.
arXiv cs.RO ·