Scaling Study of Continued Pre-Training for Embodied Agents in Virtual Worlds

The study uses 120 controlled experiments to examine the scaling patterns of continued pre-training in virtual worlds, covering four model sizes from 0.8B to 27B, six data budgets from 50 to 1,600 hours, and five data mixtures ranging from Minecraft-only to a multi-world blend.