Event
AnyGoal benchmark released; compositional generalization in world models generally fails
1 reports · 1 sources
A research team released the AnyGoal benchmark and an open-source library for evaluating the compositional generalization ability of world models on goal-shifted task variants. The benchmark is built on commonly used world model environments, and the task design ensures that individual state factors and actions remain within the training support, but their combinations are underrepresented in expert trajectories. The authors state that experiments across multiple environments and world model families show that current models generally fail in these recombination cases, and they further analyze where failures occur at the dynamics and representation levels.
Report timeline
- AnyGoal: Evaluating World Models' Compositional Generalization Beyond Expert Trajectories
The study introduces the AnyGoal benchmark and open-source library to evaluate world models' compositional generalization on goal-shifted task variants. The task design ensures that individual state factors and actions remain within the training support, but their combinations are underrepresented in expert trajectories. Experiments across multiple environments and world model families show that current models consistently fail under these recombination scenarios, and the authors further analyze where failures occur at the dynamics and representation levels.
HuggingFace Daily Papers ·