Descriptions:
Milan Shetti, VP of Data and Analytics at AWS and the engineer who has run S3 for 13 years, joins The Pragmatic Engineer to share rarely disclosed details about the engineering behind the world’s largest cloud storage service. The scale figures set the stage: S3 currently holds over 500 trillion objects, stores hundreds of exabytes of data, processes hundreds of millions of transactions per second, and handles over a quadrillion requests annually. Underneath that sits tens of millions of hard drives across millions of servers, distributed across 120 availability zones in 38 regions worldwide. Individual customers now store exabytes of data in what Sony’s group CEO has taken to calling a “data ocean” rather than a data lake.
A major portion of the conversation focuses on S3’s transition to strong consistency, which required proving correctness across every possible request path rather than simply testing it. AWS achieved this using formal methods — mathematical proofs integrated directly into the CI/CD pipeline that verify the consistency model has not regressed whenever engineers modify the index subsystem. The same approach is applied to cross-region replication and API correctness. Shetti explains that formal methods, where computer science and mathematics intersect, are used extensively across S3 in ways that most engineering organizations never encounter.
The episode also covers the concepts of correlated failure and crash consistency — disciplines central to how S3 engineers reason about reliability at this scale — and the original design philosophy that S3 launched with in 2006 when it was the first AWS service. For engineers interested in distributed systems, fault tolerance, and how one of the most relied-upon infrastructure services in the world maintains its guarantees, this episode provides technical depth rarely available outside AWS engineering publications.
📺 Source: The Pragmatic Engineer · Published January 21, 2026
🏷️ Format: Podcast
