NVIDIA proposes Cosmos-based interactive world simulation framework

NVIDIA proposes a shared framework that adapts Cosmos world foundation models for real-time interactive world simulation for Physical AI. The framework can stream visual observations in response to actions and scene control flows, supporting both predicting the visual consequences of actions and generating observations from physical states maintained by an external simulator. The authors demonstrate it in camera navigation, robot manipulation, controllable video transfer, and multi-view autonomous driving simulation, and have released the code and model checkpoints.