Article

SWIM: Learning State Weights from Behavior to Train JEPA World Models

SWIM proposes using an inverse dynamics model to learn state feature weights from offline trajectories for state-supervised training of JEPA world models. The method predicts recorded actions from current states and observed future displacements, quantifies each state feature's contribution to action prediction, and uses this to build a weighted state metric that aligns the latent space with state distances. The inverse dynamics model is used only during training; planning still relies solely on images. On five simulated robotic manipulation tasks, SWIM's planning success rate consistently outperforms uniform state weighting.

Part of the event: SWIM: Learning State Weights with Inverse Dynamics for Training JEPA World Models

This page provides a summary and source link only.