Event

Study finds backbone representation geometry determines world action model generalization

1 reports · 1 sources

Junha Chun et al. found that the generalization performance of world action models (WAMs) diverges significantly depending on the representation geometry of their generative backbones, and proposed using nuisance orthogonality (the degree to which features are orthogonal to the in-distribution manifold under out-of-distribution visual changes) to quantify this representational prior. Experiments covered a variety of image and video backbones, and the results show that the higher the orthogonality before training, the stronger the robustness of the WAM to visual changes after training, enabling prospective prediction of policy generalization before training; further analysis of robust WAMs shows suppressed feature displacement and robust action generation.