Event
SimuGuard audits simulation contact artifacts
1 reports · 1 sources
Alex Zhou and other authors propose SimuGuard, an audit plugin for LIBERO, RoboTwin, RoboCasa, and RoboDojo that identifies failures caused by the simulator rather than the policy. It replays recorded episodes bit-for-bit and reruns the episode after changing only one simulation parameter, without altering the original evaluation. The authors trace these failures on RoboTwin to the physics engine's collision detection: when geometries are 1 to 2 centimeters apart, the engine reports about 2 centimeters of penetration, and the solver ejects the object at several meters per second, because the distance algorithm discards triangles below a fixed area threshold. The authors recommend that benchmarks running on PhysX 5.3.1 or earlier be rerun in the fixed engine; if the engine cannot be changed, they recommend reporting both the success rate and the interval obtained when all flagged episodes fail or all pass—for example, for the 83.1% reported by LingBot-VA, the interval is 75.8% to 98.5%.
Report timeline
- SimuGuard: Auditing Contact Artifacts in Simulation Benchmarks Such as LIBERO and RoboTwin
The authors propose SimuGuard, an auditing plugin for LIBERO, RoboTwin, RoboCasa, and RoboDojo that identifies failures caused by the simulator rather than the policy. On RoboTwin, the authors trace these failures to the physics engine's collision detection: when geometry is 1 to 2 centimeters apart, the engine reports a penetration of about 2 centimeters, and the solver ejects the object at several meters per second because the distance algorithm discards triangles below a fixed area threshold.
arXiv cs.RO ·
Related topics
Interest trend
- Current
- 3
- Peak
- 4Nov 4 01:00
- Past 24 hours
- –
The trend compares participants observed throughout the same period, so its range may be smaller than the current interest count. Move or tap to inspect each hour; use the arrow keys on a keyboard.