Summary
At the AI Engineer conference, Hugging Face co-founder and chief science officer Thomas Wolf interviews Olive Song, a MiniMax researcher and former NYU PhD student from Jan LeCun’s lab. Their conversation focuses on MiniMax M3, an open-source model with roughly 400 billion total parameters (20 billion activated), a 1-million-token functional context window, and native support for text, image, and video understanding.
Song details the MiniMax Sparse Attention (MSA) architecture that makes the long context tractable at scale: an index branch identifies what matters most in the context, then a sparse attention branch performs calculations only on the selected blocks. She traces MiniMax’s context ambitions back to M1 and MiniMax-01, which could handle up to 10 million tokens but lacked agentic capability — M3 combines long context with coding performance and multimodality specifically to serve complex agentic tasks where tool responses and multi-turn interactions can overwhelm shorter windows.
A particularly detailed segment covers MiniMax’s choice to train multimodality from the very first pre-training step — what they call “native multimodality” — rather than bolting on vision adapters after text pre-training is complete. Song argues that late-stage adapter training hurts both text performance and vision convergence. Wolf contextualizes MiniMax within the competitive field of Chinese AI labs — alongside DeepSeek, Moonshot (Kimi), and GLM — all contending for the top position on open-source leaderboards.
📺 Source: AI Engineer · Published September 04, 2026
🏷️ Format: Interview
