AI product Open source · Apache-2.0

Agent Desktop

Agent Desktop is a Rust-based CLI and C library for letting AI agents inspect and operate desktop applications through operating-system accessibility trees rather than pixel matching. It returns JSON snapshots containing stable, snapshot-scoped references for actionable elements; agents act on those references and take another snapshot to verify the result, while stale or ambiguous references produce structured errors. It supports macOS, Windows, and Linux, with commands for application and window management, keyboard and mouse input, clipboard access, notifications, screenshots, sessions, and batch operations. It can be installed from npm with platform-native binaries or used through release archives and its FFI library. macOS requires Accessibility permission, Linux requires a graphical session with at-spi2-core, and the project is licensed under Apache-2.0.

View repository Visit site Mentioned in 1 video ↓

What Agent Desktop is used for

2 uses taken from transcripts — each links to the moment in the video.

  • A command-line tool and C library that lets AI agents read and operate desktop apps on macOS, Windows and Linux using the OS accessibility tree instead of pixel matching, returning JSON snapshots with element references.

  • CLI and C library letting AI agents read and operate desktop apps via the OS accessibility tree

Videos mentioning Agent Desktop

1 in the library.