DexRoam: Training Dual-Arm Robots with Human Whole-Body Demonstrations

The Hong Kong University of Science and Technology, Beijing Academy of Artificial Intelligence, Peking University and other institutions jointly proposed DexRoam, which uses first-person human whole-body demonstrations to train mobile dual-arm dexterous manipulation policies; the work has been accepted by CoRL 2026. Demonstrators only need to wear a Meta Quest 3 headset and a head-mounted ZED Mini stereo camera to synchronously record first-person video, whole-body joint motion and hand keypoints, without external cameras, environment markers or additional body trackers. The system narrows the human-robot motion distribution gap through three stages: embodiment alignment, action semantic alignment and temporal alignment. The team reports that the MMD and SWD metrics decreased by an average of 41.7% and 31.1%, respectively. On the two VLA models GR00T N1.7 and π₀.₅, combining human demonstrations and halving the amount of robot data still achieves task performance close to the baseline with full robot data. The team has open-sourced the collection and alignment code as well as the policy training code.

AI 根据报道生成 · 4 days ago更新