Event
DreamTacVLA: Predicting Future Contact with a Tactile World Model
1 reports · 1 sources
Guo Ye and other authors propose DreamTacVLA, a framework that grounds vision-language-action (VLA) models in contact physics: it first trains a unified policy, using a hierarchical spatial alignment (HSA) loss to align tactile tokens with their spatial counterparts in wrist-camera and third-person views, and then fine-tunes with a tactile world model that predicts future tactile signals. The authors claim that, after training on a mixed dataset (high-fidelity digital twins and real experiments), the framework outperforms existing VLA baselines, with success rates up to 95%. The approach treats high-resolution tactile images as microscopic visual input, combines them hierarchically with local wrist vision and third-person macroscopic vision, and lets the model learn contact dynamics by "anticipating future touch." The above claims of success rates and outperforming baselines are the authors' own statements and have not yet been independently verified.
Report timeline
- DreamTacVLA: A VLA framework that uses a tactile world model to predict future contact
DreamTacVLA proposes a framework that grounds VLA models in contact physics, learning contact dynamics by predicting future tactile signals. The model uses hierarchical perception, taking high-resolution tactile images as micro-vision inputs combined with wrist-camera local vision and third-person macro vision, and uses a Hierarchical Spatial Alignment (HSA) loss to align tactile tokens with their spatial counterparts in the wrist and third-person views.
arXiv cs.RO ·