CLAW: Learning Continuous Latent Action World Models from Action-Free Videos

Tewodros Ayalew and co-authors released CLAW, a fully end-to-end self-supervised framework that can jointly learn a world model and continuous latent action representations directly from videos without action annotations. Its approach uses adversarial latent regularization to prevent the latent variables from encoding future observations, so that the latent variables capture action structure; the world model uses a diffusion model to model environment dynamics as a distribution over future observations rather than a single deterministic rollout. The authors state that this latent action world model supports imitation learning from observations and goal-directed planning: for imitation learning, latent actions extracted from raw videos can be used for behavior cloning; for planning, CLAW generates latent action sequences and maps them to executable actions to achieve goals. The authors state that it outperforms existing methods in multi-task, multi-embodiment experiments.

AI 根据报道生成 · 2 days ago更新