Event
Comparison of Preference Aggregation Levels: No Single Approach Is Universally Optimal
1 reports · 1 sources
Ziqian Huang et al. compared four objective aggregation methods—reward-level, advantage-level, loss-level, and PCGrad—in preference-conditioned reinforcement learning across two humanoid robot control frameworks, concluding that no aggregation level is optimal across all environments and metrics. The experiments used Preference Reward Responsiveness (PRR) combined with coverage and utility metrics for evaluation: early aggregation produces a broader range of behavioral responsiveness but introduces more unintended interference with other objectives; late aggregation improves control precision at the cost of some loss in responsiveness.
Report timeline
- One Policy, Many Styles: A Comparative Study of Preference Aggregation Locations in Steerable Humanoid Control
A study compared four preference aggregation methods—reward-level, advantage-level, loss-level, and PCGrad—across two humanoid robot control frameworks, finding that no aggregation level is optimal across all environments and metrics. The experiments used Preference Reward Responsiveness (PRR) combined with coverage and utility metrics; results show that early aggregation produces a wider range of behavioral responses but introduces more unintended objective interference, while later aggregation improves control precision at the cost of some responsiveness.
arXiv cs.RO ·