AI product Open source
Agent S is an open-source computer-use agent framework from Simular. It accepts natural-language tasks, observes a real desktop through screen and accessibility signals, and operates ordinary applications by clicking, typing, scrolling, and executing optional local code rather than relying on per-application APIs or scripts.
The project has progressed through three generations: accessibility-tree grounding with an LLM, a vision-based manager-worker architecture using multiple foundation models, and Agent S3, which scales computer-use attempts through parallel rollouts and selection. Its workflow can combine model-driven exploration with reusable automation code, replay saved skills, and recover when the interface changes. The framework supports macOS, Windows, and Linux, works with models from OpenAI, Anthropic, Gemini, OpenRouter, vLLM, and open-weight providers, and is distributed through the `gui-agents` Python package. An optional local coding environment can execute Python and Bash with the current user's permissions, so the repository warns that this capability should be used only in trusted environments.
1 use taken from transcripts — each links to the moment in the video.
Simular's open-source, model-agnostic computer use agent harness. It has gone through three generations: accessibility-tree-based grounding with an LLM, a vision-only manager-worker architecture with two foundation models, and a test-time-scaling version running multiple parallel agents. It broke the human-performance record on OSWorld in December.
1 in the library.