Event
DepState-JEPA Dual-Stream RGB-Depth State Representation Research Released
1 reports · 1 sources
Xiujian Liang released DepState-JEPA, a dual-stream RGB-depth joint-embedding predictive architecture that connects spatial understanding with action-conditioned prediction through a shared latent state, DepState. The framework is built on V-JEPA 2.1 and uses depth both as a complementary observation stream and as an anchor for metric 3D space. Methodologically, a ray-conditioned readout predicts local features without requiring target depth; in-model ablations are used to distinguish the respective contributions of depth content, metric coordinates, and future-state matching, while calibrated forward-kinematics regularization yields an additional control gain. The report does not provide quantitative results from specific manipulation experiments.
Report timeline
- DepState-JEPA: Depth-Grounded State Representation Learning for Visual Perception and Manipulation
The authors propose DepState-JEPA, a dual-stream RGB-D joint-embedding predictive architecture that connects spatial understanding with action-conditioned prediction through a shared latent state, DepState. Built on V-JEPA 2.1, the framework uses depth both as a complementary observation stream and as the grounding for metric 3D space, and constrains state learning with ray-conditioned readouts, object-trajectory supervision, and calibrated forward kinematics.
arXiv cs.RO ·