Descriptions:
At the AI Engineer conference, Thiyagarajan Maruthavanan of Kalmantic Labs delivers a candid talk on the hidden costs and control risks of relying on rented AI inference from providers like Anthropic and OpenAI. He opens with a striking data point: a major U.S. retailer spent approximately $200 million on Anthropic inference before deciding to build its own infrastructure, while Uber’s engineering team exhausted an entire year’s token budget by month four.
Drawing on personal experience building Ulta Sono — a reverse-prompt generator for Suno.com — Maruthavanan describes costs ballooning to hundreds of thousands of dollars and a stolen API key that nearly caused $100,000 in damage before being caught at $10,000. He profiles three enterprise archetypes that hit walls with rented inference: an investment fund needing rate-limit control, a hospital whose third-party vendor dependency was flagged in a compliance audit, and a tax practice requiring reproducible model outputs for regulatory accountability.
His build-vs-rent framework centers on product-market fit: pre-PMF startups can safely rent, but post-PMF companies and enterprises with budgeted AI projects cannot afford the cost unpredictability, auditability gaps, and rate-limit constraints of shared inference. Maruthavanan describes his own transition to a Nvidia DGX box and frames AI inference capacity as “cognitive infrastructure” — an asset that mature AI organizations need to own rather than perpetually lease.
📺 Source: AI Engineer · Published July 18, 2026
🏷️ Format: Keynote Launch
