Summary
Fahd Mirza covers DwarfStar, a brand-new inference engine built specifically for DeepSeek V4 Flash (DS4) by the creator of Radius. Unlike llama.cpp wrappers or generic GGUF runners, DwarfStar is a self-contained system with its own CUDA and Metal backends, a dual KV cache that spills active sessions to disk for persistence across restarts, a built-in HTTP server that speaks the OpenAI, Anthropic, and Responses APIs simultaneously, and a native in-process coding agent.
The video walks through building DwarfStar from source on Ubuntu with an Nvidia H100 (80GB VRAM), downloading the Q2 imatrix quantization — the only variant small enough to potentially fit on a single 80GB card at approximately 81GB on disk. Mirza explains what Q2 imatrix means in practice: routed experts are compressed to two bits while attention projections, shared experts, and output layers stay at higher precision, with calibration from real activation data to preserve quality-critical weights.
The honest highlight of the video is a live out-of-memory failure: despite the H100’s 80GB, the model marginally exceeds available VRAM and crashes. Mirza notes that DwarfStar’s primary target is Apple Silicon (Metal), CUDA is supported but secondary, and the engine only works with the specific GGUF files published by the project author. For anyone evaluating local DeepSeek V4 Flash deployment, this video provides a clear-eyed look at hardware requirements and current limitations.
📺 Source: Fahd Mirza · Published May 28, 2026
🏷️ Format: Tutorial Demo
