Descriptions:
Danilo Campos, an engineer at PostHog, delivers a frank breakdown of where AI coding agents fail in production — and the concrete fixes his team built to stop them. His talk centers on the PostHog Wizard, an agentic tool that runs 15,000 times per month to help developers integrate PostHog analytics into their projects in about 8 minutes instead of the usual two hours.
The talk walks through three core failure patterns. First, model rot: language models trained months ago don’t know the current state of fast-moving projects, so the PostHog team solved this by injecting fresh markdown documentation directly into the agent’s context window at runtime. Second, poor architectural judgment from training on low-quality codebases, addressed by maintaining “model airplanes” — lightweight, framework-specific reference implementations with working PostHog integrations the agent can study. Third, human error in the agent instructions themselves: contradictory or missing MCP tool definitions that caused hundreds of failed runs before the team caught the issue.
Campos also walks through the PostHog Wizard’s multi-step prompting strategy — starting by identifying files with business logic, then enumerating meaningful analytics events to track, then finally writing the integration — which avoids the failure mode of asking an agent to do everything in a single pass. This is a practical, battle-tested talk for anyone shipping AI coding agents to real end users at scale.
📺 Source: AI Engineer · Published April 30, 2026
🏷️ Format: Deep Dive
