Event
SMR: Learning Humanoid Streaming Motion Retargeting from Semantically Conditioned Physics Experts
1 reports · 1 sources
Chun Gu and other researchers propose SMR, a streaming neural motion retargeter learned from semantically conditioned physics experts, for converting human motion into task-space targets for a humanoid robot. The approach has physics experts track these targets in simulation under contact, dynamics, and actuator constraints, and then uses a single strictly causal Transformer to learn the expert execution trajectories that are completed and pass semantic and physical validity checks. At use time, each new frame generates a robot motion reference based only on current and past human observations, which is passed to a downstream whole-body controller. The authors claim that the semantically conditioned physics experts outperform direct optimization-based retargeting in semantic fidelity and physical validity, and that the resulting streaming retargeter outperforms kinematic and physics-refinement baselines in closed-loop performance under a unified whole-body controller.
Report timeline
- SMR: Learning streaming motion retargeting for humanoids from human semantics
SMR is a streaming neural motion retargeter learned from semantic-conditioned physics experts. It converts human motion into robot task-space targets, which the physics experts track in simulation, and then uses a strictly causal Transformer to learn their executed trajectories. Each new frame uses only current and past human observations to generate a robot motion reference for a downstream whole-body controller. Experiments show that the semantic-conditioned physics experts outperform direct-optimization retargeting in semantic fidelity and physical validity, and that the resulting streaming retargeter performs better in closed loop under a unified whole-body controller than kinematic and physics-refinement baselines.
arXiv cs.RO ·