Descriptions:
Nick Saraev breaks down a landmark Anthropic study examining what happens to coding skill when developers use AI assistance. The study used a randomized controlled trial design: participants completed a warm-up task without AI, then a main task where the treatment group could use AI and the control group could not, followed by a quiz and survey taken by both groups without AI.
The results were clear. Developers who used AI assistance scored 17 percentage points lower on a quiz covering concepts from the task they just completed — roughly two letter grades — compared to those who coded by hand. The speed advantage was real but statistically insignificant: AI users finished in about 23 minutes versus 25 minutes for the control group. Quiz averages were approximately 50% for AI users versus 65–66% for manual coders. Anthropic noted this contrasts with prior research showing AI can accelerate productivity by up to 80% on tasks where workers already possess the relevant skills.
Saraev uses the findings to articulate a distinction between horizontal productivity (producing more of what you already know how to do) and vertical growth (developing deeper, novel understanding). His argument is that AI is a powerful multiplier for existing expertise but may impede skill acquisition when used as a crutch during learning. The video is a grounded look at a real research result with specific numbers, relevant to anyone deciding how and when to integrate AI tools into their development workflow.
📺 Source: Nick Saraev · Published January 31, 2026
🏷️ Format: News Analysis
