Summary
Microsoft engineers Ornella Bahidika and Joel Allou present the architecture behind Ace, a live AI voice tutor they built to run reliably within a strict 950-millisecond response budget. Their central argument: frontier models like Claude Opus 4.7 are the wrong tool for real-time voice applications not because they are unintelligent, but because their reasoning latency exceeds what human perception tolerates in conversation.
The solution they demonstrate is offloading all decision-making — lesson sequencing, student mastery tracking, branching logic — into a state machine built in code. The model (Claude Haiku 4.5) receives only a pre-computed summary each turn and is asked to do exactly one thing: speak. Side-by-side demos show Opus 4.7 taking several seconds to respond versus Haiku 4.5 returning an answer in approximately 900 milliseconds after scaffolding is applied.
The talk offers a practical heuristic for voice and high-volume real-time systems: pick the fastest model your latency budget allows, then invest the engineering time in scaffolding rather than model capability. The tradeoff is upfront cost — small models without structure tend to drift on long or complex sessions — but the scaffolding is paid once in code, not on every inference call. Applicable patterns extend beyond voice to coding agents, ops runbooks, and any multi-step workflow where reliability matters more than open-ended reasoning.
📺 Source: AI Engineer · Published July 20, 2026
🏷️ Format: Keynote Launch
