Reinforcement Learning Enables Scalable Multi-Task Improvement of Generalist Robot Policies

This paper examines whether generalist robot policies can be improved online at deployment time for multiple different tasks simultaneously, and systematically evaluates algorithmic and engineering choices such as value estimation, policy extraction, and resource allocation. The authors build a structured evaluation suite to test generalist improvement capability and find that a suitable recipe not only improves multi-task performance online but also generalizes to tasks not covered in online training, with the training task distribution and value-based reinforcement learning being key factors. The authors describe this as one of the early pieces of evidence for online generalist improvement of robot foundation models.