Summary
Fahd Mirza demonstrates how to use Colibri, an open-source inference engine, to convert and run GLM-5.2 — a 744 billion parameter mixture-of-experts model originally shipped by Zhipu AI as a 756 GB FP8 file on Hugging Face — on a single consumer machine with no GPU required. The video focuses specifically on the conversion pipeline rather than inference, walking through how Colibri’s built-in converter shrinks the original model to approximately 370 GB in a streaming-optimized format.
The converter processes the model in 141 sequential shards: it downloads one chunk at a time (roughly 5.36 GB each), compresses it to approximately half the size (2.68 GB), saves it, then deletes the original chunk before pulling the next. This means the full 756 GB model never needs to be held in memory simultaneously. At inference time, Colibri keeps a small portion in RAM and streams thousands of expert sub-networks off disk as needed — exploiting the MoE property that only a few experts fire per token.
Mirza notes the current converter is purpose-built for GLM-5.2’s exact architecture but that the underlying technique generalizes to any mixture-of-experts model. He predicts Colibri will extend support to DeepSeek and similar open-weight MoE models, making it possible to run frontier-scale models on commodity hardware without a GPU cluster. The full conversion takes several hours on a standard disk but requires only PyTorch 2.1.3, SafeTensors, and NumPy.
📺 Source: Fahd Mirza · Published July 21, 2026
🏷️ Format: Tutorial Demo
