Summary
Fahd Mirza explores Talkie, a 13 billion parameter language model built by Alec Radford — the researcher behind GPT-2 — that was trained exclusively on English text published before December 31, 1930. The result is a model that genuinely has no knowledge of World War II, television, computers, the internet, or any event after that date, behaving instead like a highly educated person frozen in time 95 years ago.
The training corpus spans 260 billion tokens drawn from pre-1931 books, newspapers, patents, scientific journals, and case law sourced via the Internet Archive and the Institutional Data Initiative. A key challenge was data quality: because all source material was OCR-scanned from physical documents, the team had to build a custom anachronism classifier to filter out modern text contamination introduced through bad metadata. Even the instruction tuning used period-accurate sources — etiquette manuals, letter-writing guides, and encyclopedias from the era — rather than modern chat data.
Mirza runs Talkie locally on an NVIDIA RTX 6000 (48GB VRAM), where the model occupies roughly 26GB. He tests it by roleplaying as Herbert Hoover during the 1929 crash, Sigmund Freud in 1930, Nikola Tesla, and Gandhi during the Salt March, assessing historical accuracy against what he knows of each figure’s actual views. The Hoover and Freud responses come across as impressively authentic; the Tesla response draws mild criticism for being too generic. It’s an unusual model with clear appeal for historians, educators, and AI researchers curious about the limits of deliberate temporal knowledge constraints.
📺 Source: Fahd Mirza · Published May 14, 2026
🏷️ Format: Tutorial Demo
