Summary
Fahd Mirza covers two new quantization techniques — GSQ (Gumbel-Softmax Quantization) and RCO (Riemannian Constrained Optimization) — developed by the DAS Lab at the Institute of Science and Technology Austria, the same group responsible for GPTQ. Applied to the Qwen3 27B model, these methods compress it to just 11.8GB while maintaining what the researchers call “task lossless” performance — meaning the quantized model scores nearly identically to the full BF16 original on real-world benchmarks.
Mirza explains the mechanics clearly before touching the terminal: GSQ examines each individual weight and learns the optimal bit-depth (2, 3, or 4 bits) for that weight based on its sensitivity, while RCO operates at the full model level, allocating a fixed storage budget across all tensors to minimize total accuracy loss. Together, GSQ finds per-weight optimal precision and RCO decides how that precision budget is distributed globally.
The hands-on portion runs the IQ3S variant (11.8GB) via llama.cpp on Ubuntu with an NVIDIA RTX GPU (48GB VRAM). Tests push the model across three domains: generating a physics-accurate 2D canvas JavaScript animation of a rotating döner kebab spit (first attempt, no libraries); detecting a subtle ascending-vs-descending logic error buried inside a complex nested SQL CTE; and a multilingual translation challenge spanning 75-plus languages including archaic runic script and two constructed fictional languages. The quantized model handles all three tasks on par with the full-precision original, validating the task-lossless claim in practice.
📺 Source: Fahd Mirza · Published September 07, 2026
🏷️ Format: Tutorial Demo
