# New constraint enables offline-to-online reinforcement learning

- Source: Sergey Levine
- Published: 2026-11-05T04:03:40.000Z
- RoboHot: https://robohot.news/en/items/cqvd8z2np3vxptjulq210bdt8
- Original: https://x.com/svlevine/status/2118192498652991753

## Summary

When running reinforcement learning with prior data, a behavior constraint is important for trading off exploration against staying close to known good parts of the data. But choosing this constraint is delicate—we neither want to merely match all prior data nor to deviate too far. In our new paper, we propose a new constraint that combines a support constraint with a continuous penalty to achieve excellent offline-to-online reinforcement learning performance.