Event
SWIM: Learning State Weights with Inverse Dynamics for Training JEPA World Models
1 reports · 1 sources
Jiaming Hu and co-authors propose the SWIM method, which uses an inverse dynamics model to learn state feature weights from offline trajectories for state-supervised training of JEPA world models. The method predicts recorded actions from the current state and observed future displacement, quantifies the contribution of each state feature to action prediction, and uses this to construct a weighted state metric that aligns the latent space with state distances. The authors state that the inverse dynamics model is used only during training, and planning still relies solely on images, adding no computation at the planning stage. On five simulated robotic manipulation tasks, SWIM's planning success rate consistently outperforms uniform state weighting.
Report timeline
- SWIM: Learning State Weights from Behavior to Train JEPA World Models
SWIM proposes using an inverse dynamics model to learn state feature weights from offline trajectories for state-supervised training of JEPA world models. The method predicts recorded actions from current states and observed future displacements, quantifies each state feature's contribution to action prediction, and uses this to build a weighted state metric that aligns the latent space with state distances. The inverse dynamics model is used only during training; planning still relies solely on images. On five simulated robotic manipulation tasks, SWIM's planning success rate consistently outperforms uniform state weighting.
arXiv cs.RO ·