Summary
Draško Profirović, staff engineer at Pinterest, presents Medic — a production agentic diagnostics tool built to automatically troubleshoot failing Apache Spark jobs. The talk traces the system’s evolution from an early prototype using Model Context Protocol (MCP) to expose data resources to LLMs, through a single ReAct agent with a monolithic prompt, to a more mature multi-agent architecture that addresses the specific failure modes discovered along the way.
Two engineering investments stand out. First, an exception classifier pipeline that fingerprints and clusters Spark log exceptions, learns which exceptions commonly appear in successful jobs (likely red herrings), and surfaces only the most content-relevant and temporally proximate exceptions to the LLM — dramatically improving signal-to-noise ratio. Second, a metrics analysis sub-agent that converts raw time-series data into annotated image collages resembling Grafana dashboards, allowing the model to reason about executor behavior, bottlenecks, and failure patterns with a predictable token budget regardless of job duration. Pinterest also built an end-to-end test harness using OpenTelemetry and LangFuse that snapshots production state as fixtures, enabling offline evaluations that catch regressions before deployment.
The talk is notable for its frank account of what failed — single monolithic prompts that degraded when extended, manual production testing that was anecdotal and non-reproducible, and raw log ingestion that quickly hit context limits. Engineers building LLM-powered observability tools or production diagnostic agents will find actionable architectural lessons throughout.
📺 Source: AI Engineer · Published July 20, 2026
🏷️ Format: Workflow Case Study
