Summary
What happens when an AI can read and write the raw weights of another neural network? David Shapiro breaks down “Sparks of Weight Space Programming,” a November 2nd blog post from MIT researchers using OpenAI’s GPT-6 Astra.
The video starts with a plain-language explanation of neural network weights, then covers the reading experiments. Given all 17,253 weights of small networks, Astra correctly identified which of four games (Breakout, Pong, Pac-Man, Freeway) each was trained on. In language models of about 7 million weights, with the first and last layers removed, it identified whether each had learned Python, English, or Romaji, and found every copying head without a false positive.
On the writing side, Astra produced network weights from zero for three Atari-style games with no training data or tools, beating networks trained for 100 million steps. In Breakout, it outscored one trained for 500 million steps by 20 times. The researchers also had it edit an 800-million-weight open-weight Alibaba model to reduce looping and improve math scores, and a Hugging Face engineer reproduced most of the gain, scoring 51%.
Shapiro closes with his own take, comparing weight storage to a hologram and discussing what direct weight editing could mean for recursive self-improvement.
📺 Source: David Shapiro · Published November 06, 2026
🏷️ Format: Deep Dive
