Open Dreamer Releases JAX/Flax Reproduction of Dreamer 4

Researchers from Reactor have released Open Dreamer, an open-source implementation of the Dreamer 4 world-model pipeline, built using JAX and Flax NNX. The project aims to reproduce the Dreamer 4 research by providing a complete training pipeline and associated artifacts. Two primary repositories were launched: next-state/open-dreamer, which contains the training pipeline including a causal video tokenizer, an action-conditioned latent dynamics model, rollout generation, and Frechet Video Distance (FVD) scoring; and reactor-team/open-dreamer, which offers a minimal local rollout harness for generating frames from MP4 files and matching action files.

A third component is a browser-based demonstration hosted on the Reactor runtime. This demo streams a generated Minecraft world in real-time and features a "Game ⟷ Dream" toggle that allows users to switch the frame stream between the actual game and the world model. The research team focused on replicating the methods described in the original Dreamer 4 research paper, deliberately avoiding external techniques to maintain a focused experimental scope. The development process began with CoinRun, a 2D platformer designed for single-GPU training, before scaling the pipeline to handle Minecraft/VPT-style gameplay video.

The architecture of Open Dreamer features a single backbone that supports both the tokenizer and the dynamics model. This backbone employs a block-causal transformer that alternates between two types of attention: space layers, which distribute information within a single frame, and causal time layers, which propagate information across frames. The tokenizer is implemented as a transformer-based Masked Autoencoder, distinct from a VAE. The team reported approximately 100x compression and noted that this design does not require KL or adversarial loss, suggesting that masking enhances the diffusibility of the latent space.

The dynamics model is responsible for predicting the next frame and is trained using diffusion forcing, flow matching, and shortcut models. It also predicts the subsequent action. Instead of separating transition and policy modules, the rollout process is integrated into per-timestep blocks. Each block contains the previous action, state, and policy. Spatial attention operates within these blocks, while causal temporal attention connects them across time. This integrated approach allows for efficient processing and prediction within the world model pipeline.