Event
Evaluating the Social Reasoning Capabilities of Gemini Robotics-ER
1 reports · 1 sources
Daniel Weiner evaluated Gemini Robotics-ER on the SHREC human-robot social interaction benchmark and compared it with published general-purpose vision-language model baselines: it outperformed GPT-4o-mini on error/capability detection, but performed worse on fine-grained social attribute judgments. The author notes that because the comparison spans different model families, these differences cannot be attributed solely to embodied specialization; this evaluation is a preliminary result, and broader cross-context and video-based analyses are left for future work.
Report timeline
- Evaluating Social Reasoning of Gemini Robotics-ER on the SHREC Benchmark
An evaluation tested the social reasoning capabilities of Gemini Robotics-ER on the SHREC human-robot social interaction benchmark and compared it with published general-purpose vision-language model baselines. Results show that Gemini Robotics-ER outperforms GPT-4o-mini on error/competence detection but performs worse on fine-grained social-attribute judgments; adding a visual frame helped one task while hurting the other.
arXiv cs.RO ·