Event
Proposes a training data method for EmbodiedMidtrain
1 reports · 1 sources
Yiyang Du and other researchers propose a training data engine for EmbodiedMidtrain and the EmbodiedMidtrain-1M dataset, aimed at bridging the gap between vision-language models (VLMs) and vision-language-action models (VLAs). The authors first characterize the difference in data distribution between the two, finding that VLA data occupy a compact region largely separated from the VLM distribution; they then use a lightweight learnable proximity estimator to select, from a large-scale VLM pool, the candidates most aligned with VLA, producing the EmbodiedMidtrain-1M dataset for midtraining VLMs. The paper states that experiments on simulated robotic manipulation benchmarks and real-world tasks show consistent VLA performance improvements compared with off-the-shelf checkpoints, across different VLM backbones and VLA action heads, and that these improvements extend to downstream tasks not used to guide data selection. The authors state that they will release all code, data, and models.
Report timeline
- EmbodiedMidtrain: Bridging the gap between vision-language models and VLAs via mid-training
The paper proposes EmbodiedMidtrain, which bridges the gap between vision-language models (VLMs) and vision-language-action models (VLAs) through mid-training. The authors first characterize the data distribution gap between the two, finding that VLA data occupy compact regions largely separated from the VLM distribution. They then build a mid-training data engine that uses a lightweight learnable proximity estimator to select the candidates most aligned with VLA from a large VLM pool, producing the EmbodiedMidtrain-1M dataset for mid-training VLMs.
arXiv cs.RO ·