Summary
NVIDIA has shipped a 2-bit model that challenges the long-held rule of thumb that local models should not go below 4-bit. This video from Sam Witteveen examines Nemotron 3.5 Bolt 74B A9B, a mixture-of-experts model with 74 billion total and about 9 billion active parameters, built on NVIDIA’s hybrid Mamba-Transformer design. It handles images and video, supports up to 256K context, and fits in roughly 24 GB. Microsoft previewed it at its RTX Spark laptop event.
The core argument is that quantization is only the last step. The video traces how NVIDIA went from the 120-billion-parameter Nemotron 3.5 Super to a roughly 20 GB file using pruning, distillation, post-training with SFT and reinforcement learning for agent tasks, and finally quantization-aware distillation, where the quantized model is the student of the unquantized one.
It also explains why post-training quantization breaks down below 3 bits, citing Meta’s ParetoQ research, and places NVIDIA’s approach alongside quantization-aware training from Google’s Gemma 3 and Gemma 4, OpenAI’s GPT-OSS and Moonshot’s Kimi. Viewers get a clear map of the techniques behind modern small models, plus demos of the model’s output.
📺 Source: Sam Witteveen · Published November 09, 2026
🏷️ Format: Deep Dive
