Summary
Google’s Gemma 4 12B model contained a subtle but impactful bug in its official Jinja chat template that was silently breaking multi-turn and agentic performance. The issue: the model’s chain-of-thought reasoning was being discarded at the end of every turn. Because Gemma 4 reasons before answering, losing that context meant that by the third or fourth turn of a conversation, the model would effectively start from scratch — tool call arguments would collapse to empty objects, and complex multi-step tasks would fail in ways that appeared to be model quality problems rather than a template configuration issue.
Fahd Mirza breaks down the root cause and demonstrates the fix in detail. The correction is a single flag in the Jinja template — `preserved_thinking`, now defaulting to `true` — which ensures the reasoning chain persists across turns. To validate the fix, Mirza serves Gemma 4 with llama.cpp on a system equipped with an NVIDIA RTX 6000 (48GB VRAM), explicitly pointing llama.cpp to the corrected template. He then runs the Hermes agent on a multi-file task requiring the model to read Python and HTML files, infer relationships between them, and apply consistent edits across multiple turns — a task designed to fail without working cross-turn memory.
The results are visible in the terminal: the model generates a substantial chain of reasoning tokens before acting, something that was not happening before the fix. For anyone running Gemma 4 locally for agent workflows, applying this template update is a practical and immediately actionable improvement. The video also explains the broader principle of how Jinja chat templates shape model behavior independently of the weights themselves.
📺 Source: Fahd Mirza · Published June 09, 2026
🏷️ Format: Tutorial Demo
