Bonsai 27B (1-bit LLM): The First 27B-Class Model to Run on a Phone
PrismML has unveiled Bonsai 27B, a groundbreaking 1-bit large language model based on Qwen3.6 27B, designed to operate efficiently on mobile phones. This innovation allows a 27B-class model, traditionally requiring significant resources, to run locally on devices like an iPhone, promising to democratize powerful AI capabilities. The announcement resonates with Hacker News's interest in optimizing cutting-edge AI for practical, on-device applications, pushing the boundaries of intelligence density and local AI agentic workloads.
The Lowdown
PrismML has announced Bonsai 27B, a significant advancement in the field of large language models, making a 27B-class model runnable on everyday smartphones. This model, based on Qwen3.6 27B, leverages 1-bit and ternary weight quantization to drastically reduce its footprint without substantial loss in intelligence, opening new avenues for on-device AI.
- Bonsai 27B is introduced as the first 27B-class model capable of deployment on a phone, building on PrismML's previous work with 1-bit and ternary models.
- It offers two main variants: Ternary Bonsai 27B (5.9 GB, 1.71 effective bits) for quality on laptops, and 1-bit Bonsai 27B (3.9 GB, 1.125 effective bits) specifically optimized to fit within a phone's memory budget, such as an iPhone 17 Pro.
- Despite the aggressive quantization, Ternary Bonsai 27B retains 95% and 1-bit Bonsai 27B retains 90% of the full-precision baseline performance across a 15-benchmark suite, excelling particularly in math, coding, and tool-calling.
- The model supports multimodal capabilities (vision), a large 262K-token context, and speculative decoding for improved speed.
- This technology is crucial for enabling agentic workloads—multi-step reasoning, tool calls, and complex workflows—to run entirely on-device, offering benefits like zero marginal cost per step, enhanced privacy, and offline functionality.
- Bonsai 27B runs natively on Apple devices via MLX and NVIDIA GPUs via CUDA, with model weights released under the Apache 2.0 License.
This breakthrough represents a paradigm shift by making sophisticated AI accessible directly on consumer devices, moving the frontier of "intelligence density" and allowing for new hybrid deployment architectures where local models handle routine tasks and cloud models are reserved for the most challenging computations.