Event

Complementary Predictive Representation Framework Proposed to Improve Embodied Manipulation Policies

1 reports · 1 sources

Xiaoxu Xu et al. propose a two-stage framework that learns complementary predictive representations from multi-view video clips to enhance embodied manipulation policies. In the pretraining stage, a frozen 3D foundation model provides dense geometric features, which are compressed into predictive geometric representations and optimized to preserve scene structure and its temporal evolution; action supervision is then imposed on geometric transitions to extract motion representations that emphasize action-relevant transitions. In the downstream post-training stage, predictive geometry is transferred via geometric queries, and motion representations guide intermediate features along the action generation path; future observations serve only as privileged supervision during training and are not needed at inference time. The paper reports that on LIBERO, LIBERO-Plus, SimplerEnv, and real-world dual-arm and single-arm tasks, the method consistently improves downstream policies, including an absolute improvement of 10.4 percentage points on the WidowX evaluation. Code will be open-sourced.