Agent Desktop connects AI agents to desktop apps

Vercel Labs’ repository provides a command-line tool and programming library for macOS, Windows and Linux. The agent receives structured observations with identifiers for available controls, acts on an identifier, then observes again to check the result. The coordinating loop remains in the agent. Setup differs: macOS requires accessibility permission; Windows needs no separate grant for applications at the terminal’s integrity level; Linux requires the user’s graphical session and accessibility service.

The concepts document explains that this structure comes from operating-system accessibility interfaces: a representation of an application’s controls. An identifier belongs to its observation and is not stable across interface changes. Resolution rejects missing, outdated or ambiguous targets instead of guessing. The documentation also separates input delivery from verification: an action may have reached the application without its effect being confirmed. Default interactions use accessibility operations. If those cannot perform the requested operation, they fail rather than silently switching to physical input. These distinctions matter when interpreting an automation’s reported outcome.

MarkTechPost’s coverage introduces the release to developers and discusses reduced token use. It explicitly identifies the reported savings as the builder’s own figures, without an independent computer-use benchmark.