Event

OxyGen paper proposes unified KV-cache inference design for VLAs

1 reports · 1 sources

Xiangyu Li et al. posted a paper on arXiv proposing OxyGen, a unified-management inference design that treats the KV cache as a resource shared across tasks and across control cycles, for multi-task parallel inference in VLA (vision-language-action) models. The design includes cross-task KV sharing to eliminate redundant prefill of shared observations, and cross-frame continuous batching that advances unfinished language requests within each frame's decoding budget. The paper reports evaluation on π0.5, where OxyGen achieves up to 3.7× speedup compared with isolated execution, and on an RTX 4090 simultaneously reaches over 200 tokens/s language throughput and 70 Hz action frequency; the design is also validated on Xiaomi-Robotics-0 and StarVLA across different backbone and expert configurations, and inference gains are demonstrated on a humanoid robot equipped with Jetson AGX Thor.