Event

T-WLA uses shared visual memory to simultaneously generate actions and correct touch

1 reports · 1 sources

Researchers including Kaichen Zhou propose the world-language-action architecture T-WLA, which uses a single shared visual memory to simultaneously guide action generation and tactile correction: the memory is cached to a lightweight tactile editor, which combines the current action chunk and real-time touch to predict residual corrections to unexecuted actions, without rerunning the visual branch. Future image and future pressure predictions serve only as training supervision. On the EgoTouch and Tachin tactile hand motion prediction benchmarks and four real-world robot dexterous manipulation tasks, T-WLA reduces finger joint MPJPE by 30% relative to the strongest reported baseline, and achieves an average success rate of about 50.0% on the real-world manipulation tasks.