Descriptions:
Emma McGrattan, Chief Technology Officer at Actian and a 30-year veteran of the data industry, delivered a talk at AI Dev 26 x SF focused on the architectural challenge she calls the “context layer” — the data infrastructure that grounds enterprise LLMs in business reality rather than generic internet knowledge. McGrattan’s central thesis is that getting this layer right is now the defining engineering challenge for organizations moving AI from pilot to production.
She identified three forces that complicate clean AI data architectures: regulatory pressure (including EU data sovereignty requirements and the US Patriot Act’s reach into US-hosted platforms), industry-specific compliance mandates in financial services and healthcare that prohibit data from leaving on-premises data centers, and data gravity — citing Gartner’s figure of 400 average data sources per enterprise, distributed across mainframes, clouds, SaaS platforms, and APIs. She noted that latency requirements for autonomous vehicles and fraud detection (sub-millisecond in some cases) further rule out cloud-only approaches.
McGrattan then walked through the tradeoffs of three deployment topologies for the context layer: cloud (elastic scale, global reach, but 20–200ms latency and egress costs), on-premises (required for regulated industries, but capital-intensive), and edge (sub-millisecond decisions, but constrained resources). The talk concluded with a practical discussion of RAG architecture — using a car insurance pricing example — and how the physical location of the vector search and retrieval components directly determines response quality and speed for end users.
📺 Source: DeepLearningAI · Published May 19, 2026
🏷️ Format: Deep Dive
