Summary
Raphael Kalandadze, founder of Wandero AI, addresses what he calls “the missing layer” in AI agent development: the operational infrastructure required after an agent ships to production. Most conference talks end at deployment; this one starts there, arguing that the real engineering challenge begins the moment real users interact with a live system at scale.
Kalandadze walks through the unique difficulties of monitoring non-deterministic, long-running agents. Unlike traditional software with defined flows and testable button paths, agents produce endless conversational trajectories, silent failures masked by lucky mid-task recoveries, and complex tool-calling chains spanning hundreds of calls. He describes failures referenced in Anthropic blog posts — agents marking tasks complete without verifying results — and explains why standard safety nets like unit tests, regex checks, and scripted simulations capture only a thin slice of production reality.
His solution is a two-loop observability system built at Wandero. A fast loop runs every 15–60 minutes: a log monitoring agent reads production trajectories, identifies stuck users and hidden errors, and automatically opens pull requests with structured descriptions, Mermaid diagrams, and metadata summaries. A separate review agent critiques those PRs from an independent angle before routing them to a human. A slower second loop provides high-level system health monitoring. For engineering teams scaling agentic products beyond the demo stage, this talk offers a practical, battle-tested playbook for closing the production feedback loop.
📺 Source: AI Engineer · Published July 05, 2026
🏷️ Format: Workflow Case Study
