Event
Proposing the P3L Pluralistic Alignment Framework under Anonymous Crowdsourced Feedback
1 reports · 1 sources
Akansha Kalra and other researchers propose the Preference-PCA Pluralistic Learning (P3L) framework, which jointly learns trajectory embeddings and a shared basis of preference directions from anonymous heterogeneous crowdsourced feedback via PCA decomposition, enabling personalized reward learning and downstream policy optimization without requiring annotator identities. In experiments on multiple embodied control tasks, the authors report that P3L supports steerable personal alignment, generalizes to out-of-distribution new users beyond the offline dataset, achieves competitive reinforcement learning and test performance, while significantly reducing computational cost and supporting variable adaptation query budgets.
Report timeline
- Preference-PCA Pluralistic Learning (P3L): Steerable Pluralistic Policy Alignment from Anonymous Crowd Feedback
The P3L (Preference-PCA Pluralistic Learning) framework jointly learns trajectory embeddings and a shared basis of preference directions via PCA decomposition from anonymous, heterogeneous crowd feedback, enabling personalized reward learning and downstream policy optimization without requiring annotator identities. In experiments across multiple embodied control tasks, P3L supports steerable personal alignment, generalizes beyond the offline dataset to out-of-distribution new users, and achieves competitive reinforcement learning and test performance while substantially reducing computational cost and supporting variable adaptation query budgets.
arXiv cs.RO ·