Event
MRO-GWM: A Gaussian World Model for Rigid Objects
1 reports · 1 sources
Jens U. Kreber, Lukas Mack, and Joerg Stueckler propose MRO-GWM, an object-centric Gaussian world model for learning action-conditioned dynamics of 3D rigid objects. The model uses Gaussian representations in object coordinate frames to represent arbitrary shapes and multi-object scenes, and predicts rigid-body motion from historical Gaussians and future actions via a temporal-spatial Transformer. The model is trained on multi-view reconstructions and therefore must handle partial object observations caused by occlusion. The authors report that they analyzed prediction performance on a synthetic dataset composed of typical household objects, containing multi-object dynamics and robot end-effector interactions; they also evaluated model-predictive control for non-prehensile manipulation in simulation, and image-based object motion tracking and prediction in both simulation and real-world scenarios.
Report timeline
- MRO-GWM: Action-Conditional and Object-Centric Gaussian Splatting World Model for Rigid Objects
The authors propose MRO-GWM, an object-centric Gaussian world model for learning action-conditional dynamics of 3D rigid objects. The model represents arbitrary shapes and multi-object scenes with Gaussians in an object-centric frame, and uses a temporal-spatial Transformer to predict rigid body motion from past Gaussians and future actions; training is based on multi-view reconstruction, which requires handling partial observations caused by occlusion.
arXiv cs.RO ·