Cohere Releases 218B North Small Translate Model

Cohere has released North Small Translate, an open-weight machine translation model developed by Cohere and Cohere Labs. This model is a sparse Mixture-of-Experts (MoE) architecture, featuring a total of 218 billion parameters, with 25 billion active parameters per token. North Small Translate supports translation across 50 languages, ranging from Albanian to Vietnamese. In evaluations conducted by Cohere on the WMT26 benchmark, the model achieved an average score of 83.6 across all supported languages. Cohere claims this performance surpasses that of commercial translation services like DeepL and Google Translate, as well as other open-source models such as GLM 5.2 and Mistral Large 3. The model is accessible through various deployment options: it can be used freely via Cohere's API up to specified rate limits, self-hosted for non-commercial use, or licensed for commercial applications. The development of North Small Translate represents a return to the foundational problem of machine translation, which was a key focus of the original Transformer architecture introduced by Google researchers in 2017 with the paper "Attention Is All You Need." The Transformer's initial breakthroughs were demonstrated on WMT 2014 English-to-German and English-to-French translation tasks. Cohere frames the release of this dedicated translation model as a matter of digital sovereignty, asserting that organizations unable to communicate globally may struggle to maintain their sovereignty. North Small Translate is the inaugural model within Cohere's "North" family of models and follows previous multilingual efforts such as Tiny Aya and Command A Translate. The model's real-world translation quality was shaped through collaboration with RWS, an organization whose Language Weaver scientists and language experts contributed to its development. Architecturally, North Small Translate is a decoder-only sparse MoE Transformer. It incorporates 128 experts, with 8 experts activated per token, alongside shared experts applied universally. The routing mechanism employs a sigmoid function over expert logits, normalized across the selected top-k experts. Its attention mechanism includes interleaved sliding-window layers (with a window size of 4096 and RoPE embeddings) and global layers that do not utilize positional embeddings, in a 3:1 ratio. This specific attention configuration was previously introduced in Cohere's Command A model. The model supports an input and output context window of 16,000 tokens, exclusively for text. It underwent post-training specifically to enhance translation quality. Approximately 11.5% of the model's weights are active for each token processed, and the per-token compute load corresponds to the 25 billion active parameters, although the full 218 billion parameters must be held in memory.