Event
Ego-CEVR proposes cross-embodiment visual representation learning framework
1 reports · 1 sources
Keru Zhou and other authors released a paper proposing a hierarchical visual representation learning framework named Ego-CEVR, which divides visual representations into two levels, global and interaction, and sets learning objectives for each respectively. According to the paper, the global module performs future prediction on human first-person videos, learning object relationships, task states, and temporal progress; the interaction module combines global context to interpret wrist observations, and through robot action supervision learns local alignment and contact cues. Two residual pathways feed both representations simultaneously into the action policy. The paper states that the global video module can be independently pretrained on large-scale human first-person videos and videos from various robot embodiments, without requiring robot action labels or paired wrist observations.
Report timeline
- Ego-CEVR: Learning Cross-Embodiment Visual Representations from Human Egocentric Videos
Ego-CEVR proposes a data-native hierarchical visual representation learning framework that assigns distinct learning objectives to global and interaction representations. The global module learns object relations, task state, and temporal progress through future prediction on egocentric video; the interaction module interprets wrist observations in global context and learns local alignment and contact cues through robot action supervision. Two residual paths feed both representations into the action policy simultaneously, and the global video module can be pretrained independently on large-scale human egocentric videos and videos from multiple robot embodiments, without robot action labels or paired wrist observations.
arXiv cs.RO ·