Event
REACT framework enables humanoid robots to learn interaction policies from human-human demonstrations
1 reports · 1 sources
Yuto Shibata et al. proposed the REACT framework, which learns deployable humanoid robot-human interaction policies from human-human demonstrations. It uses a VLM-guided contact modeling and retargeting pipeline to identify semantic contacts and transfer them onto humanoid-human geometry, then distills a privileged interaction policy into an egocentric depth policy that relies entirely on onboard sensing. The framework was evaluated on five contact-intensive interactions in simulation and on a real Unitree G1 humanoid robot. On real hardware, REACT completed all five interactions using only onboard sensing; generative partner state modeling improved contact recall in four of the five interactions, with the most pronounced improvement under severe occlusion.
Report timeline
- REACT: VLM-Guided Contact Retargeting and Generative Partner Modeling for Humanoid-Human Interaction
REACT is a framework for learning deployable humanoid-human interaction policies from human-human demonstrations, using semantic contact structure as the supervisory signal that links demonstration processing, interaction learning, and onboard control. It uses a VLM-guided contact modeling and retargeting pipeline to identify semantic contacts and transfer them onto humanoid-human geometry, then distills a privileged interaction policy into an egocentric depth policy that relies entirely on onboard sensing, and introduces a generative partner state modeling objective to predict partner motion from occluded viewpoints.
arXiv cs.RO ·