PDRVLA: Physical Token-Conditioned Action Generation
What happened
Hanqing Wang and other authors released a paper proposing the PDRVLA vision-language-action model: its lightweight physical dynamics module extracts physical tokens from vision-language features to condition action generation, without the need for online video generation. This module learns by directly predicting future RGB, depth, and optical flow features, with a frozen pretrained video world model providing auxiliary supervision. All training data comes from public data and was augmented through human-to-robot synthesis, subtask annotation, and Embodied Chain-of-Thought (ECoT) annotation. The authors report success rates on four simulation benchmarks: LIBERO 99.3%, LIBERO-Plus 89.9%, RoboTwin 2.0 93.6%, and SimplerEnv 81.5%.
AI 根据报道生成 · 6 days ago更新
Reporting timeline
Follow the reporting to explore different perspectives.
- arXiv cs.ROPDRVLA: A vision-language-action model with physical dynamics representations
PDRVLA proposes a lightweight Physical Dynamics Module that extracts physical tokens from vision-language features to condition action generation, without online video generation. The module learns through direct prediction of future RGB, depth, and optical-flow features, with auxiliary supervision from a frozen pretrained video world model. All training data comes from public data and was expanded through human-to-robot synthesis, subtask annotation, and embodied chain-of-thought (ECoT) annotation.
- HuggingFace Daily PapersPDRVLA: A Vision-Language-Action Model with Physical Dynamics Representations
PDRVLA introduces a lightweight Physical Dynamics Module that extracts physical tokens from vision-language features to condition action generation, without requiring online future-video generation. The module learns by directly predicting future RGB, depth, and optical-flow features, with auxiliary supervision from a frozen pretrained video world model. Training uses only public data, expanded through human-to-robot synthesis, subtask annotation, and embodied chain-of-thought (ECoT) annotation.