Descriptions:
The Pragmatic Engineer host Gergely Orosz interviews Simon Ericson, founder and CEO of Turbopuffer, a vector database engineered on top of object storage rather than traditional in-memory infrastructure. The conversation traces Ericson’s path from competitive programming and an unconventional high-school recruitment by Shopify to the founding insight behind Turbopuffer: while building a recommendation engine for ReadWise, he calculated that storing vectors for all users would cost $30,000 per month — roughly six times that bootstrapped company’s entire infrastructure budget. That gap sent him looking for a cheaper architecture.
Ericson explains the core technical bet: store everything on S3, then engineer around its latency characteristics. S3’s P99 on a 256–512 KB object runs around 200ms, which would be disqualifying for interactive search — but with careful file clustering and query design, Turbopuffer achieves workable latency at dramatically lower cost. The interview gets into why navigating a tree on S3 amplifies P99 exposure with every hop, how the team tuned around that, and why S3’s durability properties make it an attractive foundation despite the latency challenge.
Developers building RAG pipelines or evaluating vector store infrastructure will find Ericson’s engineering rationale particularly useful, and the early-LLM context — when 4–8 KB context windows made retrieval non-negotiable — helps explain why the vector database market grew so fast alongside the rise of large language models.
📺 Source: The Pragmatic Engineer · Published July 21, 2026
🏷️ Format: Interview
