Descriptions:
Mehedi Hassan, a product engineer at Granola (an AI-powered meeting notes app), presents a candid account of what it actually takes to ship production AI features beyond the demo stage. His core argument: one-shotting an LLM into a generic chatbot is straightforward, but making it genuinely useful for a diverse user base requires engineering depth that most tutorials overlook. The talk uses Granola’s meeting Q&A chat feature as a running example, walking through the failure modes that appeared once real users got access — inappropriate web search triggers, token costs rising to around 10 pence per chat at scale, and a silent upstream model update that degraded web search quality with no warning from the provider.
A central theme is the impossibility of a single prompt serving all users. Someone in sales expects deal-focused summaries; an engineer wants action items and Linear ticket suggestions; HR needs something different again. Hassan explains how Granola began building custom internal tracing tools — wrapping the AI SDK, persisting traces to a database, and building a purpose-built UI — so that the entire team (product, data, CX, and leadership) can follow the agent loop step by step and pinpoint exactly where outputs go wrong, without needing to run complex CloudWatch queries.
The talk is a valuable on-the-ground perspective for anyone building LLM-powered products at scale, covering observability, cost management, persona-aware prompting, and the practical case for investing in internal tooling over off-the-shelf SaaS eval providers.
📺 Source: AI Engineer · Published May 10, 2026
🏷️ Format: Deep Dive
