Event

TAPESTRY: Shared Backbone for Video Generation and Robot Control

1 reports · 1 sources

Jeremy A. Collins and co-authors propose TAPESTRY, a world action model that treats robot actions as a single patch token in the latent space of a video model, adapting a pretrained video diffusion Transformer into a robot policy. The authors state that at inference time the model does not need to generate video; instead, starting from pure noise, it produces a compact sequence of action tokens via single-step DDIM generation, taking 84 milliseconds per action chunk on consumer-grade hardware. The authors report that TAPESTRY performs well on simulation benchmarks and real-robot experiments, with inference cost markedly lower than comparable joint video-action policies.