Event

Trace-WAM: Assisting World Action Model Learning with 3D Motion Prediction

1 reports · 1 sources

Xu Hu and other authors propose Trace-WAM, which adds future 3D motion prediction as an auxiliary objective to the video-action learning of a world action model, supervising temporal point correspondences, image-plane displacements, and depth changes to characterize interaction dynamics and spatial changes in manipulation. The method adopts a three-branch Mixture-of-Transformer (Tri-MoT) architecture that jointly learns video prediction, motion prediction, and action generation. The authors state that because the action expert does not consume the predicted trajectories, inference still follows the policy path based on the current observation: a single visual forward pass plus action denoising. The authors report that experiments on RoboCasa-GR1, LIBERO, and LIBERO-Plus show that the method has strong in-distribution performance and out-of-distribution generalization ability, with particularly notable improvements over Fast-WAM under limited demonstration data and distribution shift.