Summary
Fahd Mirza tests the Ornith 1.5 9B model — the smallest and densest member of the Ornith 1.5 family — on a single NVIDIA A100 with 80GB VRAM, served via vLLM. The model is notable for its self-improving training loop: rather than following a fixed human-designed harness, it jointly generates its own tasks, scaffolds, and solutions through reinforcement learning. On paper, its benchmark numbers are striking for a 9B dense model, reportedly closing in on Qwen’s 35B mixture-of-experts on several agentic and reasoning tasks.
To stress-test real-world agentic capability, Mirza gives the model genuine AWS credentials with a scoped IAM user (EC2 permissions only) and asks it to provision a free-tier Ubuntu EC2 instance autonomously via CLI — a task simple for a human but demanding for an agent. The model struggles significantly: it consumes over 43,000 tokens without completing the task, makes repeated AMI ID selection errors across thousands of public images, generates CLI typos, and ultimately requires human steering to proceed. Switching to browser-based interaction is attempted as a fallback.
Mirza’s takeaway is deliberately grounded: benchmark performance and reliable autonomous execution in real infrastructure tasks remain meaningfully different. The video also covers VRAM consumption details (just over 72GB at default context) and explains how reducing context length lowers KV cache pressure — useful context for practitioners evaluating single-GPU deployment feasibility.
📺 Source: Fahd Mirza · Published August 20, 2026
🏷️ Format: Benchmark Test
