Claude and GPT Get Nerfed After Launch (They Admitted It)

Claude and GPT Get Nerfed After Launch (They Admitted It)

More

Summary

Every new AI model launches to viral demos, and a week later users start asking whether it got worse. Nate Herk looks into whether models really are nerfed after release and finds the answer is more nuanced than the rumors suggest.

He points to Anthropic’s two published postmortems, which acknowledged that Claude degraded for weeks and that the company’s own evaluations missed what users were reporting. He also notes that Sam Altman said GPT-5 seemed dumber after launch because of a routing problem. Launch-page figures can mislead too: Opus 5.5’s coding score of 57.8 percent was measured at max effort, while the default experience scores closer to 52 percent. Herk finds no hard proof that companies swap in quantized models after launch. The Live Nerf tracker, which runs the same 78 questions daily, detected no drop in its first 50 days, while Margin Lab recorded a fall from about 65 to 50 percent that traced to a Claude Code app update rather than the model itself.

Herk frames the problem as three dials that labs can turn without notice: reasoning effort, system prompts and instructions, and model routing. He explains how each can change results, and gives viewers practical advice on checking which settings sit behind any demo or benchmark.


📺 Source: Nate Herk | AI Automation · Published November 18, 2026
🏷️ Format: Opinion Editorial

1 Item

Channels

2 Items

Companies