NVIDIA Releases Open Full-Duplex Speech Model

NVIDIA has released NemotronLabs VoiceChat 11B, an open-source, 11-billion parameter, end-to-end speech-to-speech model engineered for real-time, full-duplex conversational interactions. This model distinguishes itself by unifying speech understanding and speech generation within a single neural network, thereby eliminating the need for separate Automatic Speech Recognition (ASR), Large Language Model (LLM), and Text-to-Speech (TTS) components. This integrated approach significantly reduces the complexity associated with orchestrating multiple models and managing API handoffs, a common characteristic of cascaded systems. The primary benefit of this unified architecture is a substantial reduction in end-to-end latency. NVIDIA reports a measured smooth turn-taking latency of 448 milliseconds on the Full-Duplex-Bench 1.0 benchmark. A key feature of NemotronLabs VoiceChat 11B is its ability to listen while it speaks, enabling users to interrupt mid-turn. The model demonstrates a take-over rate of 1.00 at 480 milliseconds, meaning it can effectively yield to the user's input with minimal delay. Furthermore, it is the first open full-duplex model to support tool calling capabilities while maintaining continuous conversation flow. This is achieved through a dedicated output channel for <TOOLCALL> scripts, alongside operator-defined "on-hold" lines that fill the silence while an API request is processed. While the model is available with permissive licensing and its weights and container are publicly accessible, NVIDIA specifies that the checkpoint is intended for research purposes only and is not yet production-ready. The documentation highlights several failure modes, including a two-minute audio context ceiling, degradation into unintelligible speech after multiple turns, instances of runaway self-talk, and dropped words in user transcriptions. Deployment requires a single GPU with at least 80 GB of VRAM, such as an NVIDIA A100, H100, RTX 6000 Pro, or B200, running on an x86_64 Linux system. This hardware requirement makes it accessible to AI-native startups, funded scaleups, enterprise R&D departments, GPU cloud providers, and university speech research groups. Currently, there is no hosted API or third-party inference provider offering the model, limiting evaluation for those without direct GPU access. Potential industry applications span contact centers and customer experience platforms, in-cabin automotive assistants, retail and drive-thru ordering systems, modernization of telecom Interactive Voice Response (IVR) systems, dynamic dialogue for games, and accessibility tooling. Specific use cases include barge-in-capable voice agents, voice front-ends for internal APIs, live-lookup assistants for information retrieval, and duplex conversational agents.