Event
KiRoSe Segments Robot Demonstrations Using Kinematic Data
1 reports · 1 sources
Shuangyu Xie and other authors propose and release the KiRoSe framework, which uses robot kinematic data rather than multi-camera video streams to segment robot demonstration data. The method encodes motion trajectories as sparse motion tokens (SMT) to compactly represent recurring motion events, and uses subtask annotations to supervise an open language model in interpreting token sequences. The authors report that, in comparisons with human-annotated segmentation results, KiRoSe's F1 metric is clearly superior to baseline segmentation methods, and that it can transfer zero-shot to unseen robot embodiments and first-person human motion; in addition to segmentation, it can also track task progress and detect action reversals on unseen robots.
Report timeline
- KiRoSe: Efficiently Segmenting Robot Demonstrations with Kinematic Data
KiRoSe proposes segmenting demonstration data using robot kinematic data rather than multi-camera video streams, compactly representing recurring motion events through Sparse Motion Tokens (SMTs). The method supervises an open language model with subtask annotations to interpret token sequences, clearly outperforming baseline segmentation methods on the F1 metric, and can transfer zero-shot to unseen robot embodiments and first-person human motion. Beyond segmentation, KiRoSe can also track task progress and detect action reversals on unseen robots.
arXiv cs.RO ·