Event
Multimodal foundation models generate expressive robot motions
1 reports · 1 sources
Till Hielscher, Fabio Scaparro, and Kai O. Arras published a paper proposing a method to automatically generate and modulate complex full-body expressive gesture sequences from multimodal contextual descriptions composed of text, audio, and video, and to realize the animation principles uniformly with dynamic movement primitives (DMP). The authors state that in a user study conducted on a humanoid platform, participants considered the motions generated by the system to fit the interaction context better than motions generated under unmodulated or API-level conditions; the study also demonstrated that the system can generate expressive motions for robots with different kinematic structures.
Report timeline
- Context-Aware Generation and Modulation of Expressive Motion Behavior Using Multimodal Foundation Models
This research proposes a method to automatically generate and modulate complex full-body expressive gesture sequences from multimodal context descriptions spanning text, audio, and video. It implements the principles of animation in a unified way based on Dynamic Movement Primitives (DMP) and uses multimodal foundation models to generate highly context-aware parametric motion variants.
arXiv cs.RO ·