A New Trick for RL Training of Flow Matching Policies

Diffusion (or flow) makes for excellent policies, but training them with RL is notoriously hard: BPTT is unstable, and RL over diffusion blows up the horizon. In our new paper, we show how to optimize a flow matching actor with a "weird little trick"—using the identity matrix to "approximate" the Jacobian of the flow denoising process. 👇