Event
WARPED Synthesizes Wrist-View Training for Robot Policies from Monocular Human Demonstrations
1 reports · 1 sources
Harry Freeman et al. propose the WARPED framework, which synthesizes realistic wrist-view observations and actions from first-person monocular RGB human demonstration videos to directly train robot visuomotor policies. The system initializes the interaction scene with vision foundation models, tracks the hand and manipulated object through a hand-object interaction pipeline and retargets the trajectories to the robot end effector, then uses Gaussian Splatting to synthesize photorealistic wrist-view observations. The paper reports that WARPED achieves success rates comparable to teleoperation on four of five rigid-body tabletop tasks and on one articulated-object task, while reducing data collection time by 4-9x.
Report timeline
- WARPED: Synthesizing wrist-view observations from first-person human demonstrations to train robot visuomotor policies
WARPED is a framework that synthesizes realistic wrist-view observations and actions from human demonstration videos, enabling visuomotor policy training with only monocular RGB data. The system uses vision foundation models to initialize the interactive scene, employs a hand-object interaction pipeline to track the hand and manipulated object and retarget trajectories to a robotic end-effector, and finally uses Gaussian Splatting to synthesize photorealistic wrist-view observations for directly training policies.
arXiv cs.RO ·