Bonsai 2 27B: Near-Lossless Compression in a 9x Smaller Footprint
PrismML has unveiled Ternary Bonsai 2 27B, an AI model that achieves nearly lossless compression, shrinking a powerful 27B-class model by over 9x while retaining 98.2% of its capability. This technical feat allows sophisticated AI to run efficiently on local devices, improving throughput and reducing energy consumption. Hacker News appreciates such advancements for their potential to democratize access to advanced AI, shift deployment paradigms, and make powerful tools more private and performant.
The Lowdown
PrismML has announced Ternary Bonsai 2 27B, a significant advancement in AI model compression that drastically reduces the footprint of a 27B-class multimodal model without meaningfully compromising its performance. This new iteration builds on previous work to enable powerful AI capabilities directly on local devices, addressing key challenges in deployment and efficiency.
- Based on Qwen3.8 27B, the model utilizes ternary weights ({-1, 0, +1}) with FP16 group-wise scaling, resulting in an effective 1.76 bits per weight.
- This compression shrinks the model to a mere 5.9GB, making it over nine times smaller than its full-precision counterpart.
- Despite the dramatic size reduction, Ternary Bonsai 2 27B retains an impressive 98.2% of the aggregate benchmark performance of the original model, excelling in reasoning, coding, vision, and agentic tasks.
- The smaller footprint facilitates high local throughput (e.g., 143 tokens/second on an RTX 5090) and enhanced energy efficiency (40% more efficient than an 8B full-precision model on an RTX 4090).
- It supports a 262K-token context window and multimodal text-and-image input, suitable for sophisticated local applications like coding agents, private document analysis, and hybrid cloud orchestration.
- The model weights are available under the Apache 2.0 license and run on NVIDIA GPUs via CUDA and Apple devices via MLX.
This near-lossless compression signals a practical 'deployment unlock' for AI, pushing the frontier of intelligence density. It suggests that low-bit models are not just viable but potentially optimal for deploying AI across the spectrum—from personal devices to data centers—by optimizing capability within strict memory, compute, and power budgets, thereby reshaping AI's economic and architectural landscape.