Summary
Marah Abdin and Robert McHardy from poolside share detailed lessons from scaling their foundation model lineup at the AI Engineer conference. The company recently released two open-weight models on Hugging Face — Laguna M and Laguna XS — and Robert McHardy previews a newer, soon-to-be-released successor during the talk. Their tech report accompanies the release for those wanting full architectural detail.
Abdin focuses on synthetic data strategy, framing it not as a replacement for organic data but as a mechanism to surface implicit structure, planning, and rationale that organic web data contains but does not expose in a training-friendly form. For Laguna XS, synthetic data constitutes 13% of the pre-training mix, drawn from a corpus now exceeding 6 trillion tokens. Three core techniques drive their approach: an auto-mixer that enables cheap data mix sweeps before committing to expensive training runs; improved web data sampling tuned for higher recall; and specialized synthetic pipelines for STEM rephrasing and code. Their internal infrastructure, called Hive, manages these pipelines through configurable queued agents with orchestrators and supervisors that dynamically route and police generation quality.
McHardy then addresses pre-training engineering from a zero-trust perspective, arguing that data quality and training code correctness must be treated as inseparable concerns when running at the scale of billions of parameters across thousands of GPUs. He describes model replica hash checks as a key invariant verification step, and highlights how subtle bugs in distributed training code can silently corrupt model quality in ways that only surface much later in evaluation.
📺 Source: AI Engineer · Published July 26, 2026
🏷️ Format: Deep Dive
