Event

Research team proposes benchmark evaluation process for service robots in public environments

1 reports · 1 sources

A research team led by Memmesheimer et al. published interdisciplinary research proposing a process for benchmark evaluation of service robots in public everyday environments, addressing the gap between laboratory metrics and actual operation, with evaluation conducted at technology readiness levels 6-7. Between 2023 and 2025, the team evaluated three robots across three use cases through seven benchmark evaluation activities (one consensus workshop and six on-site evaluations, two per use case), covering outdoor park cleaning, pedestrian underpass cleaning, and interactive assistance in a public library, with scenarios transitioning progressively from controlled conditions to public operation. The evaluation does not rank the robots, but instead examines the interplay of task completion, interaction quality, safety, and economic feasibility in the transition from laboratory prototypes to operational systems. The research notes that direct quantitative comparability across different use cases remains limited, which is characteristic of benchmark evaluation at this maturity stage.