Summary
Paula Dozsa, an iOS engineer at Tolan (the AI companion app with over 4 million hours of logged voice conversation), walks through four hard-won engineering principles for building voice-first AI products that feel genuinely relational rather than merely functional.
The talk is grounded in production numbers. A latency regression from 2.0 to 2.5 seconds measurably tanked every retention metric — the 500ms difference made users feel their companion was “too slow.” The biggest quality jump came from migrating to GPT-5.1 on the Responses API, which cut time-to-first-speech by a reported significant margin. On the model routing side, Tolan runs a “tone router” that directs emotionally significant turns to frontier models (roughly 5x the cost of smaller models) while routing casual back-and-forth and background tasks to cheaper, faster alternatives — and AB tests showed that routing a third of turns to smaller models had almost no measurable effect on retention, making the economics viable.
The memory architecture is equally detailed: rather than replaying raw conversation transcripts, Tolan embeds extracted facts, preferences, and emotional signals into a vector store with sub-50ms lookup, runs nightly compression to merge duplicates and drop noise, and maintains a two-tier system of volatile and stable memory. Context is fully reassembled each turn from parts (summary, persona card, retrieved memories, tone guidance, real-time app state) rather than reused — a deliberate choice to handle the topic volatility inherent in voice. The character design rationale — an alien to avoid real-world anchoring — also gets a brief but interesting treatment.
📺 Source: AI Engineer · Published September 15, 2026
🏷️ Format: Workflow Case Study
