PrismML Releases Bonsai 27B: Low-Bit Qwen3.6-27B Models

PrismML released Bonsai 27B this week, a series of low-bit representations of the Qwen3.6-27B language model, rather than a new pre-trained model. The architecture of Qwen3.6-27B remains unchanged in these releases, which are distributed under the Apache 2.0 license. Two primary variants are available: Ternary Bonsai 27B, which utilizes weights represented by {−1, 0, +1} and achieves a true bit-per-weight of 1.71, resulting in an ideal file size of 5.9GB. The second variant, 1-bit Bonsai 27B, employs binary weights {−1, +1} and operates at 1.125 bits per weight, with a file size of 3.9GB.

Both Bonsai 27B models are multimodal, with their architecture comprising approximately 24.8 billion language weights, a 0.46 billion vision tower, and 2.5 billion dedicated to embeddings and the language model head. The vision tower is maintained separately at a 4-bit precision using HQQ compression. The models support a context window of 262,000 tokens, a practicality achieved due to roughly 75% of Qwen3.6-27B's attention mechanism being linear. This architectural characteristic influences the compression methodology employed.

The compression technique involves representing each weight as a code, with a single shared FP16 scale factor applied per group of 128 weights. The effective weight is calculated as w_i = s_g · t_i. A ternary weight, carrying log2(3) or approximately 1.585 bits, combined with the FP16 scale (adding 16/128 bits), results in the 1.71 bits per weight for the ternary model. This represents a reduction of approximately 9.4 times compared to FP16. The 1-bit model, costing 1 bit plus the scale factor, achieves 1.125 bits per weight, a reduction of about 14.2 times. This low-bit representation is applied end-to-end across the most computationally intensive components, including embeddings, attention projections, MLP projections, and the LM head, with only normalization and scale parameters retaining higher precision.

PrismML's evaluation of Bonsai 27B involved 15 benchmarks in thinking mode, utilizing EvalScope with vLLM on H100 GPUs. The results indicate that Ternary Bonsai 27B retains 94.6% of the performance of the FP16 baseline, while the 1-bit Bonsai 27B model maintains 89.5% of the baseline performance. These figures demonstrate a significant compression capability while preserving a substantial portion of the original model's accuracy, making it suitable for deployment on devices with limited computational resources such as laptops and smartphones.