Cursor Open-Sources Mixture-of-Kittens Training Megakernel

Cursor Research has open-sourced Mixture-of-Kittens (MoK), a specialized training megakernel designed for Mixture-of-Experts (MoE) models. This megakernel integrates all communication and computation steps within an MoE layer into a single, deterministic process. The Cursor team reported that MoK achieves up to 2.37 times higher throughput compared to the most robust publicly available alternatives. This technology is already being utilized to power the training of Cursor's Composer models, which collectively span tens of thousands of GPUs.

Deployment of MoK is feasible, but it necessitates a significant hardware investment. The minimum hardware requirement includes NVIDIA Blackwell SM100 or SM103 GPUs, specifically within GB200 NVL72 or GB300 NVL72 rack configurations. Additionally, users must have Python version 3.12 or later, PyTorch version 2.10 or later, and the CUDA toolkit version 13.0 or later installed. The inter-GPU buffer operations within MoK depend on PyTorch's symmetric memory feature, which effectively restricts realistic adoption to organizations that possess or rent NVL72 computing capacity. This hardware prerequisite means that organizations such as large-scale research laboratories, well-funded AI startups, GPU cloud providers, and national computing centers are the primary potential users, while smaller teams with fewer than 8 GPUs are unlikely to benefit. The applications for MoK are specific but high-value, including the pre-training and post-training phases for MoE models similar to DeepSeek-V3. The deterministic nature of MoK also makes it suitable for on-policy reinforcement learning post-training and for conducting internal model ablations. The relevant industries that can leverage MoK include AI model development, cloud GPU infrastructure management, code generation tooling, and quantitative research.

Cursor's prior research focused on the computational aspects of MoE training, including the development of custom MXFP8 and NVFP4 training kernels and a specialized 'warp decode' path for MoE inference. However, these earlier efforts assumed that inter-GPU communication was handled as a separate process. In production environments, it was discovered that communication became the primary bottleneck, consuming over half of the total end-to-end training time for MoE layers. The transition to GB300 NVL72 racks, which feature 72 GPUs within a single NVLink domain enabling fine-grained overlap, presented a new set of challenges. A key issue identified was the relative slowness of the integrated Grace CPUs compared to the GPUs, necessitating aggressive minimization of CPU-GPU synchronization. To address this, MoK incorporates three critical design decisions. One of these decisions involves dynamically selecting the communication direction for each operation, a departure from previous approaches like DeepEP that relied on push-based transfers.