Kimi AI and kvcache-ai Open Source AgentENV for RL Training

Moonshot AI's Kimi team and kvcache-ai have jointly open-sourced AgentENV (AENV), a distributed platform engineered to facilitate the execution of agent environments at a large scale. This new platform is specifically designed to power agentic reinforcement learning (RL) training for Kimi K3, Moonshot's advanced 2.8-trillion-parameter Mixture-of-Experts model. The AgentENV code has been released under the permissive MIT license, enabling broader adoption and development within the AI community.

The development of AgentENV addresses a critical bottleneck in agentic RL, which requires AI models to interact within realistic computational environments rather than merely processing text. Each training iteration necessitates an isolated Linux environment equipped with a functional filesystem, a network stack, and active processes. This requirement presents a significant technical challenge: while containerization offers rapid startup times, it shares the host kernel, compromising the isolation needed for model-generated code. Conversely, full virtual machines provide robust isolation but suffer from slow boot times and inefficient memory utilization when idle. AgentENV aims to bridge this gap by making the startup, restart, and branching of these environments cost-effective enough for large-scale training operations.

Within the AgentENV architecture, each isolated sandbox is implemented as a Firecracker microVM. These microVMs feature their own dedicated Linux kernel, filesystem, and network namespace, ensuring strong isolation. Incoming requests are managed via an Axum HTTP API, which then directs them to an orchestrator responsible for overseeing the lifecycle of each sandbox. The system's storage mechanism is particularly noteworthy. The root filesystem is served through a ublk userspace block device, which is backed by overlaybd layered images. This setup allows read-only base layers to be shared across multiple sandboxes, while each individual sandbox maintains its own distinct upper layer for writing data. A daemon named envd operates within each guest environment, handling command execution, file operations, and reporting the health status of the sandbox on port 49983. Furthermore, a reverse proxy is employed to route HTTP and WebSocket traffic from external clients to the services running inside the virtual machines.

AgentENV incorporates two key mechanisms to enhance density and resource utilization. The host's page cache is shared across both storage and memory-snapshot data, optimizing data access. Additionally, memory ballooning is utilized to return reclaimable guest memory to the host system. This feature is crucial for sustaining overcommit capabilities as environments diverge and evolve over time during the training process. By addressing these infrastructure challenges, AgentENV aims to significantly accelerate the development and deployment of sophisticated agentic AI systems.