Summary
Will, an engineer on Anthropic’s Applied AI team, presents a structured workshop on one of the most common failure modes in production agent development: the agent that worked great at launch but degraded as requirements piled on. The case study centers on StockPilot, a fictional inventory management agent for a mid-size retailer that grew from a focused tool into a 400-line system prompt with 12 tools and multiple bolted-on subagents — and whose eval scores dropped from 83% to 62% as a result.
The session introduces Anthropic’s framework for choosing the right agentic primitive at the right time: plain tools for discrete actions, skills for packaged composable knowledge that Claude loads on demand (enabling progressive disclosure instead of bloated system prompts), and subagents for genuinely isolated workstreams with separate context windows. Will walks through using Claude Code itself to triage the failing evals, identify root causes — tool gaps, output structure mismatches, system prompt contradictions — and then systematically refactor the architecture.
The workshop is hands-on, with a live repo and 12 eval tasks across five grader types. Engineers dealing with prompt sprawl, mysterious regressions, or the question of when a tool should become a subagent will find direct, reproducible guidance here, grounded in the same methodology Anthropic uses internally.
📺 Source: Claude · Published May 23, 2026
🏷️ Format: Deep Dive
