NVIDIA Enables Rust for GPU Kernel Development

NVIDIA has announced CUDA Rust, a significant initiative to establish Rust as a first-class language for writing GPU kernels. While Rust code could previously launch CUDA kernels, the kernel body itself typically required development in other languages. CUDA Rust bridges this gap through two open-source projects from NVlabs: cuda-oxide, designed for the Single Instruction, Multiple Threads (SIMT) model, and cutile-rs, intended for the newer Tile model. Both projects facilitate the native compilation of Rust kernels and leverage Rust's robust ownership rules to identify and prevent aliasing bugs during the compilation phase, thereby enhancing code safety and reliability.

The deployability of these projects is partially realized. The cutile-rs component has been published on crates.io, supports stable Rust versions starting from 1.89, and has already seen adoption in Hugging Face's Grout inference engine and the mistral.rs project. In contrast, cuda-oxide is currently in an early alpha stage. Both projects are classified as alpha, and their suitability for production environments has not yet been confirmed by NVIDIA. The increasing prevalence of Rust in the systems layer of artificial intelligence, encompassing inference engines, drivers, and agent runtimes, underscores the strategic importance of this development. NVIDIA's own Nova Linux driver incorporates Rust, and NVIDIA Dynamo features a core written in Rust, with NVTX also offering Rust bindings. The GPU kernel had remained a notable exception until now.

The two CUDA Rust tracks directly correspond to the established programming models within CUDA. The SIMT model, utilized in CUDA C++ and Numba-cuda, involves describing the operations of a single thread and then launching thousands of such threads concurrently. The Tile model, a more recent addition also accessible via C++ and Python, shifts the focus to describing the operations on a tile of data, with the Tile Intermediate Representation (IR) compiler managing thread mapping and memory layout. NVIDIA recommends the Tile model as the primary approach, while the SIMT model is advised for scenarios requiring explicit control over threads and memory. Future plans for inter-language interoperability are intended to ensure that developers choosing Rust will not be excluded from using C++ or Python.

The SIMT track is supported by cuda-oxide, which functions as a custom Rust compiler (rustc) code generation backend. This backend routes functions annotated with #[kernel] through Rust's Mid-level Intermediate Representation (MIR), the Pliron IR framework utilized by the community, and then to LLVM IR before conversion to PTX. The remaining compilation tasks are handled by the standard backend. NVIDIA has developed the GPU dialects on top of Pliron. The current requirements for using cuda-oxide include a Linux operating system, a GPU with compute capability 8.0 or later, CUDA version 12.x or newer, the clang compiler with libclang, and a pinned nightly toolchain, specifically nightly-2026-04-03. A utility named 'cargo oxide doctor' is available to verify the setup and diagnose any configuration issues.