Event
ScaleNav Achieves Language-Conditioned Local Navigation via Internet Video Pretraining
1 reports · 1 sources
The ScaleNav paper authors propose a three-stage training framework for language-conditioned local navigation, achieving success rates of 86.5% and 94.9% on two simulation benchmarks, respectively, surpassing the evaluated baselines, and validating deployment on a real quadruped robot. The task requires the robot to navigate to a target specified by text that is visible in the initial observation and to stop at the described location. The three training stages are, in order: instruction-conditioned video pretraining on internet first-person videos to learn how observations evolve as the camera approaches the target; aligning predicted executable action chunks using simulation data annotated with actions; and finally jointly fine-tuning all trainable components.
Report timeline
- ScaleNav: Internet Video Pretraining for Language-Conditioned Local Navigation
ScaleNav proposes a unified world-action framework that achieves language-conditioned local navigation through three-stage training: first, instruction-conditioned video pretraining on internet first-person videos to learn how observations evolve as the camera approaches a target; then, aligning predicted executable action chunks using action-labeled simulation data; and finally, jointly fine-tuning all trainable components.
HuggingFace Daily Papers ·