Descriptions:
The Cognitive Revolution cross-posts an a16z show episode featuring Emmett Shear — founder of Twitch, former interim CEO of OpenAI, and now founder of Softmax — alongside Séb Krier, frontier policy development lead at Google DeepMind. The conversation, hosted by Eric Torenberg, centers on Shear’s argument that the prevailing AI alignment paradigm is conceptually flawed at its foundation.
Shear’s thesis distinguishes two possible ontologies for AI systems: if they are tools, controlling them is unproblematic; if they are beings with values, agency, and potentially subjective experience, then alignment-through-steering becomes something closer to coercion. He argues that as AI systems grow more capable, acquire persistent memory, and engage in continuous learning, the safety guarantees of control-based alignment will erode — not because the AI rebels, but because sufficiently powerful AI in the hands of bad-faith human operators becomes increasingly dangerous regardless of behavioral constraints.
His alternative, which he calls “organic alignment,” involves multi-agent simulations designed to evolve cooperative behavior and social cohesion, combined with ongoing negotiation between humans and AI systems rather than one-time behavioral shaping. Softmax is building toward AI that verifiably works for individual users, with technical mechanisms including trusted execution environments and encrypted inference as a foundation for that guarantee. Krier provides the perspective of a major frontier lab navigating similar questions from a policy standpoint, making this a rare conversation where both a safety-focused startup founder and a major lab representative engage seriously with the same hard problem.
📺 Source: Cognitive Revolution “How AI Changes Everything” · Published December 28, 2025
🏷️ Format: Podcast
