ByteDance Seed and Tsinghua AIR Release CUDA Agent for GPU Kernel Generation

ByteDance Seed and Tsinghua AIR have introduced CUDA Agent, a large-scale agentic reinforcement learning system designed to generate optimized GPU kernels for CUDA. This system trains a large language model to produce CUDA code that significantly outperforms standard compiler optimizations. The primary challenge CUDA Agent addresses is the generation of efficient, rather than merely correct, CUDA code. Prior to this development, frontier AI models could generate functional CUDA kernels, but these often lagged behind the performance achieved by compiler optimizations.

On the KernelBench benchmark, the base model, Seed1.6, demonstrated a 74.0% task completion rate but only surpassed torch.compile in performance for 27.2% of tasks, exhibiting a geometric-mean speedup of 0.69×, indicating its kernels were generally slower than compiler-generated ones. CUDA Agent was trained within a comprehensive CUDA development environment that included profiling tools, correctness verification, and a permission-locked sandbox. This training utilized Proximal Policy Optimization (PPO) over 150 steps with a substantial 131,072-token context window. The result of this intensive training is a remarkable 98.8% task pass rate and a 96.8% rate of outperforming torch.compile across a 250-task benchmark. The system achieved a geometric-mean speedup of 2.11× over standard compilation, positioning it approximately 40 points ahead of advanced models like Claude Opus 4.5 and Gemini 3 Pro on the most challenging Level-3 tasks.

While the full trained agent has not been released, the research team has made public several components: the CUDA-Agent-Ops-6K dataset, the SKILL.md specification, and the reward and anti-reward-hacking recipes. The underlying model, Seed1.6, is a proprietary Mixture-of-Experts (MoE) model featuring 23 billion active parameters and a total of 230 billion parameters. The development and training process required significant computational resources, with the profiling sandbox alone utilizing 128 NVIDIA H20 GPUs. This level of resource intensity suggests that full replication is feasible primarily within frontier research labs, large GPU cloud providers, and major infrastructure teams. However, mid-sized teams can still leverage key components such as the dataset, milestone reward system, and skill specification, integrating them with open-source base models.

The applications for CUDA Agent span various industries and critical computational domains. These include AI infrastructure and inference serving, GPU cloud computing, autonomous driving systems, quantitative trading platforms, medical imaging analysis, and recommendation systems. Essentially, any area where fused kernels are on a latency-critical path can benefit. Specific use cases involve optimizing operator sequences that torch.compile handles poorly, reducing the cost per token in AI model inference, and re-tuning kernels to adapt to new generations of GPUs. The research also touches upon data synthesis techniques, with the team crawling reference operations to enhance the training data.