Thinking Machines Lab Releases Inkling-Small Multimodal MoE Model

Thinking Machines Lab has released Inkling-Small, an open-weights Mixture-of-Experts (MoE) model featuring 276 billion total parameters and 12 billion active parameters. This new model is approximately a quarter of the size of its predecessor, Inkling, which boasts 975 billion total parameters and 41 billion active parameters. The training of Inkling-Small was conducted on NVIDIA GB300 NVL72 systems. A key capability of Inkling-Small is its native reasoning over text, images, and audio, expanding its utility beyond text-only models. The model supports an extensive context window of 1 million tokens, and its "thinking effort" is adjustable, allowing users to fine-tune its computational resource usage. The model weights are distributed under the Apache 2.0 license and are available on Hugging Face. Deployment of Inkling-Small is facilitated by its quantized checkpoints. The BF16 checkpoint requires a minimum of 600 GB of aggregated VRAM, achievable with configurations such as 4x NVIDIA B300 GPUs or 8x NVIDIA H200 GPUs. For more constrained environments, the NVFP4 checkpoint reduces this requirement to 180 GB. Specifically, the W4A4 quantization can run on a single B300 GPU, necessitating SM100+ compute capability, while W4A16 quantization requires two H200 GPUs. Supported runtimes for Inkling-Small include SGLang, vLLM, TokenSpeed, Unsloth, and Hugging Face. This single-GPU deployment capability makes a 276 billion parameter model accessible outside of specialized frontier lab environments, enabling startups to self-host on a single rented B300 instance. Mid-size enterprises with existing H200 capacity can deploy the model without acquiring new hardware. Furthermore, regulated sectors such as financial services, healthcare operations, insurance, telecommunications, and the public sector can leverage Inkling-Small for private-weights deployment, ensuring data privacy and compliance. Applicable workloads for Inkling-Small span various domains. In text and image processing, it can power coding agents, terminal automation, and the understanding of documents and charts. The inclusion of audio processing capabilities extends its use to call-center analytics, voice interfaces, and meeting summarization. Architecturally, Inkling-Small is a 42-layer decoder-only transformer model. It utilizes a sparse MoE feed-forward backbone where each token is routed to 6 out of 256 available experts, in addition to 2 shared experts that are active for every token. The attention mechanism is a hybrid of local and global layers, and the model is encoder-free, reinforcing its native multimodal design. For image input, images are segmented into 40x40 pixel patches and processed through a four-layer hMLP. Audio input is handled as dMel spectrograms. Both image and audio data pass through a lightweight embedding layer before being jointly processed with text tokens. The model supports numerical precision formats including BF16, MXFP8, and NVFP4. For audio input, the recommended format is WAV at 16 kHz, ideally with a duration under two minutes.