Descriptions:
Hannah Lichtenberg and Amir, co-founder and agentic search lead at Mixedbread AI, present research on what they call the “knowledge gap” — the widening disparity between rapidly improving LLM reasoning and the comparatively stagnant quality of retrieval systems that feed those models. Using two benchmarks, BrowseComp Plus and OfficeQA Pro (created by OpenAI and Databricks respectively), they show that GPT-5 with optimal document access achieves near-oracle performance of 93% and 64%. When forced to retrieve from a noisy 100,000-document corpus using default tools, performance drops by 9 and 8 points respectively. Swapping in Mixedbread’s search reduces those gaps to just 3 points on BrowseComp.
The root cause, the presenters argue, is that LLMs are trained on coding and web-search tasks that reward BM25-style keyword queries, which perform poorly against modern semantic search systems. Agents effectively guess keywords to maximize BM25 overlap rather than composing the rich semantic queries that would return the right documents.
Mixedbread’s solution is a purpose-built knowledge agent with four specialized tools: a wide overview search returning up to 50 chunk summaries, a deep semantic search returning full payloads for the top 10 results, a metadata filter tool for faceted lookups, and a grep tool for exact keyword matching. The agent runs up to four parallel search rounds per query and is designed to force the model to articulate what evidence it needs before writing any query — a framing trick that steers the model away from BM25-style outputs and toward queries that leverage semantic search properly.
📺 Source: AI Engineer · Published July 07, 2026
🏷️ Format: Deep Dive
