Summary
Sonam Pankaj, CEO and co-founder of Starlight Search, presents Agent RX (Runtime Experience) — a memory layer designed to make AI agents learn from their own task outcomes without retraining or manual prompt engineering. The talk opens with sobering industry data: Gartner has reported 85% of AI agents fail to gain meaningful traction, and Starlight’s own analysis attributes 73% of pipeline failures to retrieval problems rather than generation quality.
The core problem Pankaj identifies is that current memory systems — including LangChain and Mem0 — store user preferences and conversation history but don’t incorporate whether a retrieved memory actually helped or hurt the agent’s execution. Agent RX introduces a “utility score” that weights semantic similarity by historical task outcome, transforming retrieval from pure embedding similarity into outcome-informed re-ranking. The system also distills accumulated memories into “skills” — updatable reasoning patterns baked into the agent’s behavior — allowing it to automatically deprecate stale context like obsolete database columns.
Benchmark results presented include TaulBench improvements from 66% baseline to 76% without skills and 80% with skills enabled. On agentic task benchmarks, Agent RX moves from a 35.7% baseline to 61.3%, outperforming competing memory systems that reach 58.2%. The talk acknowledges limitations including cold start behavior and utility drift at scale, and positions the approach as the missing layer between observability traces and agent action.
📺 Source: AI Engineer · Published June 28, 2026
🏷️ Format: Deep Dive
