Event
R²E: A Multi-Role LLM Evolutionary Framework for Automatic Reward Function Design
1 reports · 1 sources
Shouhao Chang and other authors propose R²E, a framework that automatically designs reward functions through explicit role division, aiming to alleviate the problems of insufficient exploration and unstable optimization in reward design based on large language models. The framework sets up three types of roles: a novelty explorer responsible for expanding the search space, a guardian responsible for conservative improvement to maintain training stability, and an evolver that recombines high-performance rewards through crossover and mutation; a dynamic scheduling strategy gradually shifts the search from broad exploration to focused exploitation, and a global elite pool retains the best-performing rewards to guide subsequent iterations. The authors state that extensive experiments on multiple robotic tasks show the method is effective.
Report timeline
- R²E: A multi-role LLM evolutionary framework for automated reward function design
The authors propose R²E, a framework for automated reward function design that uses explicit role specialization to mitigate insufficient exploration and optimization instability in LLM-based reward design. The framework includes a novelty explorer responsible for expanding the search space, a guardian responsible for conservative refinements to improve training stability, and an evolver that recombines high-performing rewards through crossover and mutation, with a dynamic scheduling strategy that gradually shifts the search from broad exploration to focused exploitation. R²E also introduces a global elite pool that retains the best-performing rewards to guide subsequent iterations. Experiments on multiple robotic tasks show the effectiveness of the method.
arXiv cs.RO ·