Article

Real2Sem2Real: Generating Cross-Instance Robot Training Data from a Single Human RGB Demonstration

Real2Sem2Real (R2Sem2R) proposes a real-semantic-real pipeline that turns a single monocular RGB human demonstration video into thousands of robot demonstrations covering multiple semantic object instances within the same category. The method tracks object SE(3) poses using DINOv3-distilled Gaussian splats, outperforming existing methods and FoundationPose with estimated metric depth, and maps the human hand to a feasible robot grasp that can be transferred to digital twins.

Part of the event: Generating thousands of robot demonstration data from a single human RGB video

This page provides a summary and source link only.