Descriptions:
Steve Ruiz, founder of tldraw, delivered a talk at AI Engineer Europe 2026 tracing three years of AI integration into the tldraw canvas — from the viral “Make Real” prototype in 2023, which converted whiteboard sketches into working HTML using early vision models, to a fully spatial multi-agent system where AI assistants exist as visible, interactive entities living directly on the canvas itself.
The talk walks through several generations of the idea. An early agent could draw shapes, complete diagrams, and fill out forms using structured text outputs — but still felt like a sidebar assistant rather than a true collaborator. A subsequent agent loop added iteration and self-review, which proved useful in education contexts (tutoring, form completion, D&D character sheets) but still placed the AI outside the user’s spatial workspace. The newest implementation, nicknamed “fairies,” places multiple agents as distinct animated sprites on the canvas. Each agent can observe what the others are doing in real time, receive coordinated instructions as a group, and act on shared objects simultaneously — making the state of a multi-agent system legible through spatial position and motion rather than text logs.
The tldraw SDK already underpins Replit’s agent canvas, Lubu AI’s canvas, and portions of Google Stitch’s annotation mode. Ruiz’s central argument is that spatial representation surfaces agent state in ways that chat sidebars fundamentally cannot — users see not just output but where on a shared document an agent is acting and how multiple agents relate to each other’s work. The talk is a compelling prototype for what human-in-the-loop multi-agent collaboration might look like as a design paradigm.
📺 Source: AI Engineer · Published May 01, 2026
🏷️ Format: Showcase
