Event
Multimodal social robot framework underperforms teleoperation in VR tests
1 reports · 1 sources
The research team proposed a multimodal framework integrating speech, vision, and touch to generate context-appropriate robot speech and actions, and compared it with a manual teleoperation baseline in virtual reality using 18 participants and five social interaction scenarios. Results showed that the autonomous system scored above the neutral midpoint on both naturalness and social appropriateness, but the teleoperation baseline received significantly higher ratings on these two measures. Behavior coding and feedback indicated that dialogue timing, clarification of ambiguous requests, and spatial coordination were key factors behind the gap between the autonomous system and teleoperation.
Report timeline
- A Multimodal Social Perception Framework for Human-Robot Interaction in Virtual Reality
The study proposes a multimodal framework integrating speech, vision, and touch to generate contextually appropriate robot speech and movement. The team tested it in virtual reality with 18 participants across five social interaction scenarios and compared it against a human teleoperation baseline: the autonomous system scored above the neutral midpoint on both naturalness and social appropriateness, but the teleoperation baseline received significantly higher ratings on both measures. Behavioral coding and feedback showed that conversational timing, clarification of ambiguous requests, and spatial coordination were key factors behind the gap.
arXiv cs.RO ·