PrismML Releases Ternary Bonsai 2 27B Model

PrismML has released Ternary Bonsai 2 27B, a significantly compressed version of the Qwen3.8 27B language model. This new model occupies only 5.93 GB of storage, a substantial reduction from the original 53.80 GB FP16 version. According to PrismML's reported benchmarks, Ternary Bonsai 2 27B maintains 98.2% of its parent model's performance across 20 different evaluation tasks. The model is designed to be multimodal, accepting both text and image inputs, and boasts an extensive 262,000-token context window. PrismML has demonstrated its capability in driving Cline coding agents and performing computer tasks on hardware as modest as an RTX 5090 graphics card.

The release follows the initial Bonsai 27B model two months prior, whose first ternary variant achieved approximately 95% of the original performance. Ternary Bonsai 2 27B is positioned for deployability, with its Apache 2.0 licensed weights capable of running on a 16 GB laptop or a single 24 GB GPU. Users can leverage PrismML’s specialized llama.cpp fork or its MLX runtime for operation. The core architecture of Ternary Bonsai 2 27B remains identical to Qwen3.8 27B, featuring 27.36 billion parameters. These parameters are distributed across a 24.35 billion parameter language backbone, 2.54 billion parameters for embeddings and the language model head, and a 0.47 billion parameter vision tower. The backbone incorporates hybrid attention mechanisms, with approximately 75% of its layers utilizing linear attention and the remaining 25% employing full attention.

The ternary weight format is applied to the embeddings, attention projections, MLP projections, and the language model head. A minimal set of 26.2 million parameters, representing just 0.0976% of the total, are retained in higher precision. These high-precision parameters are specifically for the recurrent state path and normalization weights. For image processing, the vision tower is provided as a separate 0.63 GB file in GGUF format, which is loaded only when image input is required.

The ternary format fundamentally alters how model weights are stored. Each weight can assume one of three values: -1, 0, or +1. To manage these values efficiently, every group of 128 weights shares a single FP16 scale factor. A single ternary value requires log2(3), approximately 1.585 bits, of storage. When combined with the 16 scale bits per 128 weights, this results in an average of 1.71 bits per weight. Including the high-precision tensors further increases this to 1.72 bits per weight. For practical implementation and efficient computation, real-world kernels necessitate a packed layout. PrismML's whitepaper details two GGUF packing methods. PTQ1_0 achieves a dense packing of ternary values (trits), resulting in 1.76 bits per weight and the 5.93 GB file size. PQ2_0 stores each trit within a 2-bit slot, leading to a larger 7.25 GB file but offering easier unpacking. The model also employs a rotated basis for weight storage, utilizing a blockwise Hadamard rotation with a block size of 1,024 before ternary assignment. The runtime then applies a corresponding transform to the activations prior to each matrix multiplication.