Descriptions:
Pratik Verma presents a practical framework for moving AI agents from impressive demos to production-ready systems at AI Dev 26 San Francisco. His core finding: agents rarely fail because of logical coding errors — they fail in real-world deployment due to ambiguous inputs, missing context, and edge cases that are difficult to anticipate during development, making conventional software testing approaches largely insufficient.
The centerpiece of the talk is Monle, an open-source observability framework designed specifically for agentic traces. Standard OpenTelemetry instrumentation struggles to capture failures inside framework-internal code — such as LangGraph or LangChain internals — where the most consequential agent behavior actually occurs. Monle solves this with structured telemetry that minimizes post-collection cleanup. The talk also announces a new Monle capability enabling billing instrumentation for outcome-based pricing, so the same library that helps find failures can facilitate payment when agents succeed.
Verma demonstrates the Okahoo IDE extension for VS Code and Cursor, which surfaces relevant traces directly in the editor and passes them as context to coding agents like GitHub Copilot or Claude CLI for targeted fixes. A second demo tackles quality failures — agents that respond politely but incorrectly, generating no exceptions and leaving no trace in standard logs — using LLM-as-judge evaluations applied at the trace level. The resulting testing framework supports assertions on tool call inputs and outputs, not just final responses, enabling a continuous deploy-observe-evaluate-fix reliability loop. The Okahoo extension is available free via QR code at the talk.
📺 Source: DeepLearningAI · Published May 20, 2026
🏷️ Format: Tutorial Demo
