Descriptions:
Fahd Mirza demonstrates a full local stack for running Google’s Gemma 4 E2B instruction-tuned model through the Hermes agentic framework, powered by vLLM as the inference backend. The tutorial targets practitioners with access to GPU hardware — in this case an NVIDIA RTX 6000 with 48GB of VRAM — and covers the complete setup: upgrading vLLM to the version that first introduced Gemma 4 support, downloading the model from Hugging Face, serving it locally on port 8000, and configuring Hermes to point at that custom endpoint rather than a commercial API.
With the stack running, Mirza shows Hermes launching with its 70+ preloaded skills and successfully fielding a general knowledge query through the Gemma 4 model, which draws around 43GB of VRAM during active inference. The video then shifts to testing Gemma 4 E2B’s native audio capabilities directly via Python API calls — a modality Hermes doesn’t natively pass through — running transcription tasks across a wide range of languages to probe the model’s claimed support for over 100 languages.
For developers interested in running multimodal open-weight models locally with agentic tooling, this tutorial provides concrete installation commands, realistic VRAM expectations, and honest notes about bleeding-edge rough edges in vLLM’s Gemma 4 integration that are expected to smooth out with future updates.
📺 Source: Fahd Mirza · Published April 03, 2026
🏷️ Format: Tutorial Demo
