Event

Ego2Contact: Predicting Touch from Human Video for Robot Use

1 reports · 1 sources

Qianqian Yang and other researchers propose Ego2Contact, which learns visual-tactile priors from human first-person RGB videos and transfers them to robot hands: the predictor fuses temporal scene context with local visual features of the hand to infer a bimanual tactile field, then aligns different robot tactile layouts to the predictor's canonical grid through anatomical region correspondences, making robot measurements the fine-tuning target and predictions the policy input. The predictor provides tactile input for simulated reinforcement learning and real-robot policy post-training. The authors report that policies using predicted touch in simulation outperform no-touch baselines and recover part of the gains from ground-truth tactile sensing.