Event

Multi-view Latent Action Pretraining Improves Cross-Embodiment Robot Policy Success Rates

1 reports · 1 sources

Oliver Hausdörfer et al. propose a latent action model (LAM) pretraining method that can ingest synchronized multi-camera views for cross-embodiment robot video learning; evaluation shows its downstream policy success rate improves by 13.5% over single-view LAM (single-view: 6.2%). The method uses cross-attention to fuse multiple views to handle depth and occlusion, then compresses them into latent actions; the input is a policy coupling a latent expert and an action expert on a frozen VLM backbone. The paper states that dual-view LAM pretraining improves the decodability of real 6D actions by 17% over single-view, while scaling single-view data by two orders of magnitude is insufficient to fully recover out-of-plane motion; skill transfer requires only one cross-embodiment robot, and adding a second view is more effective than adding up to four more embodiments. Results transfer on hardware, with success rates 20% higher than action-only policies in scenes demonstrated by human hands (850 rollouts).