Summary
Fahd Mirza demonstrates how to run Hunyuan3 (Hy3), Tencent’s 295-billion-parameter mixture-of-experts model, on local hardware using the open-source Colibri inference tool. The setup pairs a single Nvidia H100 GPU (80GB VRAM) with system RAM and NVMe storage, with the 142GB model—less than half the size of the full weights due to pre-conversion—streamed intelligently across all three memory tiers.
Mirza explains Colibri’s three-tier routing architecture in concrete terms: cold experts that haven’t been accessed recently stream from disk (the slow path), recently-used experts cache in RAM (faster), and the most frequently-hit experts get pinned directly into GPU VRAM for the fastest possible access. The system learns dynamically, so the first few tokens are slow while the cache warms, but performance improves as Colibri identifies which of the model’s 192 expert sets are most commonly needed. The build in this video also compiles with CUDA support and enables speculative decoding automatically—a speed technique where a small draft model predicts multiple tokens ahead and the main model verifies them in a single forward pass.
Token throughput is low, as Mirza openly acknowledges, but he frames the comparison correctly: the point is not speed against an optimized inference endpoint, but demonstrating that a 295B model can produce coherent output on hardware that has no business running it. The video follows his earlier demonstration of GLM 5.2 using the same disk-streaming approach.
📺 Source: Fahd Mirza · Published July 22, 2026
🏷️ Format: Tutorial Demo
