Event
AnchorQ Improves Sample Efficiency of Robot Policy Reinforcement Learning with Flow-Inversion Anchors
1 reports · 1 sources
Carl Qi and other researchers proposed the AnchorQ method on arXiv for post-training improvement of frozen flow-matching robot policies: at each decision step, flow inversion is used to map reference policy behavior into state-conditioned latent anchors, particles are sampled near the anchors, and value-guided sampling is used for coarse-to-fine policy improvement. The paper claims that on simulated and real manipulation tasks, AnchorQ exceeds existing post-training baselines in sample efficiency under the same interaction budget, improving real-world success rate by 37% and simulation task throughput by 97%. The paper's analysis points out that anchors reduce identification complexity but introduce support bias; particle refinement trades off between this bias and action-class complexity, and critic accuracy determines the reliability of KL-regularized value-guided selection.
Report timeline
- AnchorQ uses flow-reversed action anchors to improve RL sample efficiency for robot policies
An arXiv paper proposes AnchorQ, which uses flow reversal to map reference policy behaviors into state-conditioned latent anchors, samples particles near the anchors, and applies value-guided sampling to achieve coarse-to-fine policy improvement. The analysis shows that anchors reduce identification complexity but introduce support bias; particle refinement trades this bias against action-class complexity, and critic accuracy determines the reliability of KL-regularized value-guided selection.
arXiv cs.RO ·