Event

PrefO2O: A New Framework for Zero-Shot Preference Reinforcement Learning

1 reports · 1 sources

Ruiqi Wang and co-authors submitted a paper proposing PrefO2O, an offline-to-online latent preference transfer framework, which they claim can achieve zero-shot preference-based reinforcement learning (PbRL) on target tasks without any human feedback. The method learns a task-agnostic latent preference space from a small amount of offline preference data from source tasks, disentangling transferable preference structure from task-specific variation; it then generates pseudo-preference supervision via confidence-weighted optimal transport alignment for online reward adaptation. The authors claim that in robot manipulation and locomotion benchmark experiments, PrefO2O significantly outperforms existing preference transfer baselines, matches or exceeds PbRL methods trained with real human feedback, and generalizes to many-to-one and one-to-one transfer settings as well as different robot morphologies.