Descriptions:
Web Dev Cody walks through the setup and live testing of Claude Computer Use, a recently released plugin for Claude Code that enables the AI to autonomously control native desktop applications by capturing screenshots and simulating mouse clicks. The video covers the full macOS configuration process: enabling the plugin via the /mcp command inside Claude Code, granting accessibility permissions to iTerm in System Preferences, and allowing screen recording access — all necessary steps before the agent can interact with any desktop UI.
The main demo attempts to use Claude to play RuneScape via the Rune Light client, specifically tasking it with chopping trees. The results are candid: Claude struggles significantly with click accuracy, repeatedly missing targets, requiring the creator to provide a reference screenshot of a tree before it could complete a single successful chop. After roughly two and a half minutes, it had cut only a handful of trees — leading to an honest conclusion that computer use is a poor fit for game automation in its current state. The video then pivots to a more practical test, directing Claude to open Postman and execute a GET request to localhost 8080, which proves more tractable.
Despite the accuracy limitations, the demonstration provides a clear picture of how the underlying mechanism works: screenshot capture, model inference, and simulated input in a feedback loop. For developers curious about agentic desktop control using Claude Code and the Model Context Protocol, this is a useful early benchmark of current capabilities and where the technology still falls short.
📺 Source: Web Dev Cody · Published April 04, 2026
🏷️ Format: Tutorial Demo
