Code-as-World Represents Videos as Executable Physics Programs

MirroS has released Code-as-World, a novel paradigm that represents physical worlds through executable world representations, moving beyond pixel-based analysis. The core argument posits that pixels in a video are merely evidence of a physical scene, not its fundamental ontology. This means a video model can predict plausible future frames without explicitly understanding concepts like mass, contact, or gravity. Instead of relying on pixels, latent representations, or textual captions, Code-as-World represents a scene as executable code, specifically a `scene.json` file that can be run by the MuJoCo physics simulator. This executable code allows an AI agent to verify the simulation against the source video and enables human users to edit and re-simulate the scene. An agentic loop is employed to recover these physics programs from real-world footage, iterating up to five times to refine the executable representations. The verified worlds then serve as training data, providing exact physical labels that are absent in raw video. This approach aims to imbue AI models with a deeper understanding of physical interactions. When trained on this generated supervision, the Code-as-World-VL-9B model achieved a score of 55.4 on the QuantiPhy validation benchmark. This performance surpasses Gemini-3.1 Flash, which scored 54.8, and is approximately 15 points higher than the strongest available open-weight baseline models. The system is considered deployable at the research and internal-prototype tier. MirroS has made the GitHub repository and two model checkpoints, Code-as-World-VL-4B and Code-as-World-VL-9B, available under the Apache 2.0 license. These models are fine-tuned from Qwen3.5-4B and Qwen3.5-9B respectively. Both checkpoints are in BF16 safetensors format and are served using vLLM through an OpenAI-compatible `/v1` endpoint. The system processes videos by sampling 16 frames per video and utilizes a maximum model length of 4608 tokens. The technical report from MirroS elaborates on the limitations of existing video models, 3D reconstruction techniques, and captioning methods, arguing that while they may recover aspects of a scene, they fail to capture its underlying physical mechanism. Code-as-World aims to bridge this gap by generating explicit physical simulations from visual input, thereby creating richer and more physically grounded training data for AI agents.