Event

Shanghai Jiao Tong University and others propose VaM: value prediction as task progress memory

1 reports · 1 sources

A research team from Shanghai Jiao Tong University and Shanghai Innovation Institute (Shanghai Chuangzhi Institute) and others proposed the Value as Memory (VaM) method, which feeds the value model's recent prediction sequence as "task progress memory" into the action policy, together with an offline-to-online real-robot reinforcement learning pipeline. The team reports that in a cardboard box assembly task, VaM ran autonomously continuously for 4 hours with a task success rate exceeding 95%. The pipeline includes an Online DAgger stage that allows human corrections, and a Learn from Experience stage that no longer collects human corrections and learns solely from sparse task-outcome rewards; the implementation builds an action expert based on π₀.₅ and configures a value model that predicts the return distribution. Ablation results show that compared with the version with the value cache removed, the final task success rate improved by 29.4%; the online RL scheme improved the average success rate across three tasks by 32.8%, and reduced the average real-world time cost per gradient step by 57.8%. The above results are all as reported by the research team.