Computer use for any coding agent, from the agent-browser team at Vercel Labs. A Rust CLI that reads and operates desktop apps on macOS, Windows and Linux through the OS accessibility tree instead of pixels.
The same loop as agent-browser, applied to the whole desktop: agent-desktop snapshot --app Finder -i returns JSON with a ref for every element the agent can act on, the agent clicks or types on a ref, then snapshots again to check. No screenshots or pixel matching, and the loop stays in your own agent.
Refs belong to their snapshot, so a stale ref fails with STALE_REF instead of hitting the wrong element. Ref actions are headless by default: they go through the accessibility API without moving the pointer or taking focus, so it runs in the background while you keep working. Every command prints one JSON envelope, and errors say whether a retry is safe.
Install with npm install -g agent-desktop. The binary ships its own agent docs (agent-desktop skills get desktop), and there is a skill on skills.sh. macOS needs the Accessibility permission, Linux needs at-spi2-core. Apache-2.0.