Summary
Two Minute Papers host Dr. Károly Zsolnai-Fehér breaks down Kimi K3, a new open-weights AI model from Moonshot AI that he argues fundamentally disrupts the economics of frontier AI access.
The headline stat: Kimi K3 is a 2.8 trillion parameter model — too large for most consumer hardware to run locally, but available free via the web and dramatically cheaper than competing closed models via API. The video focuses on two architectural innovations the Kimi team has disclosed. The first is Kimmy Delta Attention (KDA), a mechanism that replaces full context re-reads with a maintained and gradually-fading “notebook,” enabling efficient handling of very long sequences. The second is attention residuals, which give each transformer layer access not just to the most recent document state but to a version history across earlier layers. Together, these two techniques produce what the paper claims is a 2.5x improvement in scaling efficiency over Kimi K2 — meaning roughly two and a half times more learning progress from the same amount of training compute.
The broader argument made in the video is that open-weights releases at this scale push API token prices down industry-wide and accelerate distillation into smaller, more accessible models. Dr. Zsolnai-Fehér frames this as part of a wider “golden age of open science” in AI, noting that the team published a full technical report alongside the weights release.
📺 Source: Two Minute Papers · Published July 29, 2026
🏷️ Format: News Analysis
