Event

PhysCanvas: A Visuo-Tactile World Model for Robot Policy Evaluation and Learning

1 reports · 1 sources

Zebin Xing and other researchers propose PhysCanvas, a state-consistent visuo-tactile world model for contact-rich robotic manipulation, which unifies multi-view RGB, tactile images, marker displacement, task state, and actions into a single visual space, sharing a video VAE and a pretrained generative backbone, for closed-loop policy evaluation and GRPO policy post-training. On four real manipulation tasks, the authors report that when rolling out policies in the world model, State Reward Model supervision raises the mean per-task Pearson correlation coefficient between the world model and real success rates from 0.602 to 0.885; policy post-training in the reward-guided world model raises the real average success rate of π₀.₅ from a 59.1% SFT baseline to 70.8%. Its consistency-aware post-training uses a frozen Scene Reward Model, Modality Reward Model, and State Reward Model to supervise scene preservation, visual-tactile compatibility, and consistency with real task outcomes, respectively.