Summary
Mingsheng Hong, VP of Engineering focused on AI at legal contracting platform Ironclad, addresses one of the most pressing questions facing engineering leaders today: how do you measure and control AI token spend without accidentally creating incentives that harm productivity? The talk draws on Ironclad’s recent experience navigating the transition from AI adoption to AI cost governance.
Hong introduces the concept of ‘trusted throughput’ as a proxy for AI ROI — arguing that raw token usage, like lines of code, is a metric worth monitoring but dangerous to optimize directly. He traces Ironclad’s metric evolution from tracking open PRs to merged PRs to complexity-weighted merged PRs, where AI assigns each pull request a t-shirt-size complexity score to weight its contribution to engineering velocity. Usage dashboards, he argues, should function as smoke detectors (flagging unusually low AI adoption) rather than leaderboards that reward maximization.
The talk also addresses the human side of AI adoption, including how to respond to engineers who feel that AI has replaced craftsmanship with reviewing ‘AI slop code.’ Ironclad’s approach involves a self-learning feedback loop: reviewing AI-assisted work to extract institutional best practices, then feeding those back into the system. The session is a practical guide for engineering leaders who have moved past the early adoption phase and are now grappling seriously with demonstrating and governing AI value at scale.
📺 Source: AI Engineer · Published August 29, 2026
🏷️ Format: Deep Dive
