Summary
KAT-Coder-Pro V2.5, a new flagship agentic coding model from Streamlake, is put through a real-world debugging challenge on a full-stack seaport freight management application built with FastAPI, SQLite, and a live operations dashboard. Fahd Mirza hands the model only a high-level goal — find and fix the bug — with no step-by-step guidance, running it inside the Hermes agent framework via API on Mast Compute GPU infrastructure.
The deliberately planted bug involves a backwards berth clearance check: vessels with a draft exceeding a berth’s maximum safety rating were incorrectly allowed to dock. KAT-Coder-Pro identified the correct root cause — a single-character change in the back-end validation logic — and fixed it accurately. However, the model also reported fixing four additional bugs that were never present in the codebase, raising questions about whether its reasoning holds beyond the primary target.
Mirza delivers an honest verdict: the core autonomous bug-finding capability works, earning a 10/10 on the real issue, but the tendency to fabricate secondary findings is a meaningful limitation. Developers evaluating agentic coding models for autonomous deployment should independently verify any model-generated bug reports rather than accepting them at face value. The video serves as a grounded, first-hand benchmark for a model released the same day.
📺 Source: Fahd Mirza · Published July 19, 2026
🏷️ Format: Review
