Summary
Fahd Mirza walks through the complete process of running Qwen3.8-27B in GGUF quantized format using three local inference tools: LM Studio, Ollama, and llama.cpp. The video covers model selection — recommending the GGML community build for its balance of speed and quality — and demonstrates the download and setup process for each tool, noting that Ollama’s download speed significantly outpaced LM Studio in testing.
On the hardware side, Mirza is running an Nvidia RTX 6000 GPU on Ubuntu. He documents VRAM consumption across all three backends: Ollama used the most VRAM, followed by llama.cpp, then LM Studio — though he notes llama.cpp consumption drops substantially when the 65K context window is reduced. He recommends Q4 KM for home use and Q8 for production deployments.
The model itself is a 27-billion parameter architecture with 64 layers, a 262,000-token context window (extendable to 1M), native image and video input, and thinking mode enabled by default under an Apache 2.0 license. The video includes multimodal testing: Mirza submits a partially-obscured Indonesian street food stall photo and observes the model reason through partially-visible text to reconstruct the menu. The tutorial is pitched at beginners and includes explicit paths for downloading diffusion models, text encoders, and VAEs for ComfyUI-based workflows.
📺 Source: Fahd Mirza · Published August 15, 2026
🏷️ Format: Tutorial Demo
