Event
Grasp Pretraining Enhances Hierarchical Imitation Learning for Dexterous Manipulation
1 reports · 1 sources
The research team proposes a hierarchical imitation learning framework: a high-level policy predicts hand subgoals, and a low-level policy is a goal-conditioned controller. Using large-scale dexterous grasp annotations, they built a grasp pretraining dataset of 355k trajectories, pretrained the low-level controller first, and then fine-tuned it on downstream task demonstrations for functional dexterity in articulated tool use. The team also introduces a simulation benchmark, DexCraft, comprising six articulated tool-use tasks that require coordinated finger movement. The authors report that, in simulation and real-robot experiments, the method outperforms an end-to-end diffusion policy baseline and a hierarchical policy trained from scratch; the real-robot all-task success rate is 33.3 percentage points higher than DP3.
Report timeline
- From Grasping to Dexterous Manipulation: Large-Scale Grasp Pretraining for Dexterous Manipulation
The research team proposes pretraining the low-level controller of a hierarchical imitation learning framework with large-scale dexterous grasp data, then fine-tuning it on downstream task demonstrations to support functional dexterity in articulated tool use. They built a 355k-trajectory grasp-pretraining dataset from large-scale dexterous grasp annotations and introduced DexCraft, a simulation benchmark with six articulated tool-use tasks. In simulation and real-world experiments, the method outperforms end-to-end diffusion policy baselines and hierarchical policies trained from scratch, improving real-world full-task success by 33.3 percentage points over DP3.
arXiv cs.RO ·