Article

FELT: Generating Tactile Signals from Vision for Visuo-Tactile Manipulation

FELT is a framework that uses existing paired visuo-tactile data to learn contact-aware representations from RGB observations, providing them to downstream policies through two interfaces: generated tactile images and contact-aware features. It combines a frozen visual encoder with separate query decoders for the left and right fingers, and the two branches exchange information through gated connections, with a convolutional readout producing spatial contact representations. On four contact-intensive real-robot tasks, both interfaces achieved higher success rates than vision-only policies, and directly integrating contact-aware features into policies raised success rates by 15-22.5 percentage points.

Part of the event: FELT uses RGB to generate tactile signals, improving real-robot task success rates by 15-22.5 points

This page provides a summary and source link only.