Summary
Nidhi Kaushik Vyas, a product manager at Google DeepMind, presents a framework for building multimodal collaborative agents that work with fuzzy, underspecified user intent — a problem she argues current search-wrapper agents handle poorly by assuming users already know what they want. Her framework, grounded in commerce use cases but designed to generalize across finance, education, and other verticals, describes a three-phase loop: discovery, research, and adaptive response.
In the discovery phase, the agent synthesizes contextual signals — past conversations, user queries, personal context, and visual references — to form a collaborative strategy. Rather than asking all clarifying questions upfront, the agent identifies which single unknown variable provides maximum information gain and prioritizes that, avoiding the over-asking failure mode that Vyas flags as a common evaluation criterion. The research phase introduces multimodal elicitation: using visual inspiration boards rather than text prompts to surface preferences users cannot yet articulate in words, then doing background work to compare, trade off, and summarize options.
The response phase focuses on adaptive presentation — choosing comparison tables, bulleted lists, or visual boards based on query type rather than defaulting to text-heavy output. Vyas shares specific evaluation metrics for each phase including blocker identification, question utility scoring, and optimal-move selection logic. The talk offers a concrete engineering blueprint for teams building agents designed to genuinely guide users toward goals rather than simply wrapping a search interface.
📺 Source: AI Engineer · Published September 01, 2026
🏷️ Format: Deep Dive
