NVIDIA Releases Switchyard for LLM API Translation

NVIDIA has released Switchyard, an experimental proxy and library written in Rust, designed to address the challenge of integrating different Large Language Model (LLM) APIs within coding agent workflows. Teams developing coding agents often encounter difficulties when different models and services utilize distinct API formats. For instance, Claude Code communicates via the Anthropic Messages API, while tools like Codex CLI interact with the OpenAI API. Furthermore, models served through platforms like vLLM, NVIDIA NIM, or Ollama may have their own specific integration requirements. Rewriting the agent to accommodate each new API is often impractical, necessitating a translation layer that can operate independently. Switchyard aims to fulfill this role by routing requests across various LLM providers and translating between the OpenAI and Anthropic API formats. It also records operational metrics and offers typed, composable routing algorithms. The project is released under the Apache 2.0 license, with comprehensive documentation available at docs.nvidia.com/nemo/switchyard.

While Switchyard is available for evaluation, NVIDIA explicitly labels it as pre-alpha and experimental, warning that it is not intended for production use. The company anticipates significant changes to the API and algorithms before a stable v1.0 release. The binary can be installed from crates.io, and a launcher is available via PyPI. Switchyard can be self-hosted in any environment. The core functionality of Switchyard involves clients maintaining their native API interactions. Switchyard decodes incoming requests into a provider-neutral set of Rust types. It then executes a routing algorithm to select an appropriate backend service. Subsequently, it re-encodes the request into the specific wire format required by the chosen backend, sends the request, and translates the response, including streaming events, back into the format expected by the client. The server component of Switchyard is designed to accept three inbound request formats: OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages. Any of these inbound formats can be directed to any configured route, with each LLM client configured to use a specific upstream format. This decoupling is a key objective, ensuring that the agent's API and the backend LLM's API no longer need to be identical.

Switchyard can be deployed in three primary ways, with the launcher path specifically targeting coding agents. Users can install the tool using the command `uv tool install --python 3.12 "nemo-switchyard[cli]"`. Following installation, users can launch Switchyard with specific configurations, such as `switchyard launch claude` or `switchyard launch codex`, enabling seamless interaction with different LLM backends without modifying the agent's code. This approach simplifies the integration process for developers working with diverse LLM ecosystems, reducing the overhead associated with managing multiple API specifications and formats. The experimental nature of the release suggests that NVIDIA is actively seeking feedback and iterating on the design to build a robust solution for the evolving LLM landscape. The focus on Rust indicates a commitment to performance and safety in handling network traffic and data translation.