Descriptions:
Jacob Kahn, a researcher at Meta FAIR, presents the Code World Model (CWM)—a research project aimed at teaching AI systems to reason about and plan through code by explicitly modeling program execution rather than treating code as pure syntax.
The core insight of CWM is that standard autoregressive language models see only tokens, but a world model can predict execution traces: step-by-step state transitions showing local variables, memory, and control flow as a program runs. By framing code execution as a transition function (state + action → next state), CWM creates a structured chain-of-thought that models can both learn from and generate, enabling them to simulate execution without running code in a real environment. This simulation capability is designed to make agentic code workflows significantly more efficient.
Kahn also walks through CWM’s training pipeline—pre-training on trillions of tokens, domain-specific mid-training, long-context extensions, instruction fine-tuning, and an asynchronous RL loop with samplers, trajectory scorers, and a bash-heavy tool environment. A deliberate design choice to minimize available tools forces the model to master terminal commands, closely mirroring how a real software engineer operates. The talk is particularly valuable for researchers and practitioners interested in how execution-aware training, world models, and RL post-training intersect to produce stronger coding agents.
📺 Source: AI Engineer · Published December 17, 2025
🏷️ Format: Deep Dive
