Descriptions:
Joseph Wang and co-founder Sid of Emulated present their approach to one of the hardest open problems in AI engineering: training agents to perform the full scope of work done by a software infrastructure engineer, not just isolated code diffs. Existing benchmarks like SWE-Bench Pro, Terminal Bench, and Frontier Code evaluate agents on tasks contained within a single codebase — producing a PR over 50–100 turns. Real infrastructure work involves owning systems across years, responding to live incidents, managing distributed clusters, and reasoning about operational blast radius.
Emulated’s solution is to containerize entire software companies as training environments. These sandboxes include organizational context — tickets, postmortems, customer conversations — alongside infrastructure-level complexity such as multi-node distributed clusters, network failures between nodes, data corruption events, and clock skew. An example environment shown is an etcd consensus cluster of the kind a production service might depend on, where the agent must reason through rolling deployments, version management, and real-time troubleshooting while live traffic is running.
The talk addresses why single-sandbox simulation is insufficient for infrastructure-scale work: provisioning resources like EC2 instances or Cloud Run environments, managing VPCs, subnets, and authentication systems, and handling gradual rollouts cannot be replicated in a single containerized node. Wang and Sid’s backgrounds in distributed databases and network infrastructure directly informed this design. The post-training implications are also covered: standard homogeneous pipelines don’t reflect real multi-node system behavior, and the richer sandbox environments are beginning to feed into novel post-training data pipelines for more capable infrastructure agents.
📺 Source: AI Engineer · Published July 31, 2026
🏷️ Format: Deep Dive
