Event

ALPS: A Visual Encoder Unifying Physical State and Language

1 reports · 1 sources

The ALPS paper author team proposes ALPS, a pretrained visual state encoder that unifies three capabilities within a set of patch tokens: physical state information, language alignment, and pixel reconstruction. The model is pretrained from scratch on ViT-B/16 using only static images and text; it is first trained with language, reconstruction, and spatial correspondence objectives, then further optimized via cross-view self-distillation. In a unified evaluation against fifteen public encoders, ALPS achieves the lowest aggregate LIBERO state-probing error among ViT-B models, and the authors state its performance is comparable to V-JEPA 2-L, which is 3.5 times larger; frozen ALPS tokens paired with its native text tower achieve the highest average policy success rate on LIBERO, at 85.5%.