Summary
Fahd Mirza runs Bonsai 27B — a 27-billion parameter open-weight model from Prism ML based on Gemma 3 27B — on an NVIDIA RTX A6000 GPU, demonstrating how ternary weight quantization shrinks a 54 GB model down to approximately 7 GB while retaining an estimated 95% of full-precision intelligence. The video covers the full installation pipeline, from forking Prism ML’s custom llama.cpp repository and building from source with CMake through model download from Hugging Face and server configuration.
The technical explanation of ternary quantization is a highlight: each weight is rounded to one of three values (−1, 0, or +1), with a shared full-precision scaling factor per block of 128 weights. Because the scale is amortized across the block, real storage lands at approximately 1.71 bits per weight — a roughly 9.5× compression with no hidden high-precision escape bits. A 1-bit sibling variant reaches 3.9 GB and is noted as runnable on an iPhone 17 Pro via MLX, making it the first 27B-class model demonstrated on a phone. Both models are Apache 2.0 licensed.
Two benchmark tasks are shown. In the first, Hermes agent runs Bonsai 27B autonomously against a buggy full-stack call center application (React frontend, Python backend, SQLite database) — the model finds and fixes multiple bugs across all layers without being told their location. In the second, the model generates a self-contained single-file HTML physics simulation with custom water-park slide mechanics, curved paths, projectile arcs, and particle splashes, with no external libraries permitted.
📺 Source: Fahd Mirza · Published July 15, 2026
🏷️ Format: Tutorial Demo
