Article

A Multimodal Social Perception Framework for Human-Robot Interaction in Virtual Reality

The study proposes a multimodal framework integrating speech, vision, and touch to generate contextually appropriate robot speech and movement. The team tested it in virtual reality with 18 participants across five social interaction scenarios and compared it against a human teleoperation baseline: the autonomous system scored above the neutral midpoint on both naturalness and social appropriateness, but the teleoperation baseline received significantly higher ratings on both measures. Behavioral coding and feedback showed that conversational timing, clarification of ambiguous requests, and spatial coordination were key factors behind the gap.

Part of the event: Multimodal social robot framework underperforms teleoperation in VR tests

This page provides a summary and source link only.