An open-source computer-use agent framework developed by Simular AI. Instead of relying on APIs or dedicated integrations, it looks at screenshots and moves the mouse and keyboard to operate GUIs the way a person does. The first paper and code were released in October 2024, and the latest version, Agent S3, scored 72.60% on the desktop-operation benchmark OSWorld — the first time an agent has exceeded the human level. It is licensed under Apache-2.0 and can be run locally with pip install gui-agents.
Key Features
- Operates GUIs directly: It does not matter whether the target application exposes an API. Because the agent decides where to click and what to type by looking at the screen, existing business applications and legacy in-house systems can be automated as they are
- Published benchmark results: Agent S3 records 72.60% on OSWorld (with Best-of-N), 56.6% on WindowsAgentArena, and 71.6% on AndroidWorld (both with 3 rollouts), with scores published for desktop and mobile alike
- Windows / macOS / Linux support: Runs on the major desktop operating systems (single monitor assumed). The paper also reports zero-shot generalization across platforms
- Model-agnostic design: The reasoning model can be chosen from providers such as OpenAI, Anthropic, and Google, and it works together with a grounding model that locates coordinates on screen (for example UI-TARS-1.5-7B). You are not locked to a single vendor
- Code execution plus reflection: Beyond GUI operation, it can run Python and Bash locally, so work that would be a detour through the GUI can be handled in code. It also includes a reflection mechanism that reviews execution results
- A path to the commercial version: The framework is also built into Simular AI’s commercial products (Sai / Simular Cloud), so running it as a managed workflow without setting things up yourself is an option
Pricing
| Plan | Price | What’s included |
|---|---|---|
| Agent S (OSS) | Free | Apache-2.0 license. API usage fees for the models it calls are your own cost |
| Plus | $20/month (per computer) | 10,000 credits (reloadable), 1 developer agent, macOS/Windows, workflow editor |
| Pro | $500/month (per computer) | Unlimited credits, 1 deployment agent, virtual machine support, team workflow sharing, priority support |
| Enterprise | Contact for pricing | All platforms, unlimited agents, audit logs, SOC2 / HIPAA / SSO / RBAC |
The official site states that annual billing is not offered at this time.
Pricing is as of August 2026. Please check the official site for the latest information.
Pros & Cons
✅ Pros
- The framework itself is free and the implementation is fully published, so you can read the code and verify its behavior
- Because it works by looking at the screen, applications without an integration API can still be automated
- The reasoning model can be swapped, making it easier to tune for cost and accuracy requirements
- Benchmark scores and the conditions used to reproduce them are public, which helps when comparing options before adoption
- You can start small and move to the commercial version once you reach the operational stage
⚠️ Cons
- Running it requires API keys for the models used, and those usage fees fall on the user. Longer tasks consume more tokens
- Getting started assumes pip installation and environment variable setup, which makes it hard for non-engineers to adopt on their own
- Multi-monitor environments are not supported
- The high scores are measured under conditions that include multiple attempts (Best-of-N or 3 rollouts), so a single run will not always reach the same accuracy
- Since it actually operates the screen, there is a risk of misclicks and unintended submissions. Limiting the scope and permissions is up to the user
Comparison with Similar Services
| Criteria | Agent S | Anthropic Computer Use | OpenAI CUA | UI-TARS |
|---|---|---|---|---|
| Provider | Simular AI | Anthropic | OpenAI | ByteDance |
| Distribution | Open source (Apache-2.0) | Offered as an API capability | Offered via API and products | Open source (model) |
| Main target | Desktop in general | Desktop in general | Mainly browser operation | Desktop and browser |
| Model choice | Multiple providers | Uses Claude | Uses OpenAI models | Own model |
| Cost model | Free framework + model API fees | Pay-as-you-go API | Via API or an eligible plan | The model itself is free |
Who Is It For
- Engineers who want to automate business applications that offer no API
- People who want to understand the internals of a computer-use agent by reading the code
- Teams that want to evaluate several models without depending on one vendor
- Researchers and evaluators looking for an implementation with clearly stated benchmark conditions
- Anyone who wants to try the open-source version first and consider commercial operation later
Summary
Agent S is a framework that publishes, implementation and all, an agent that operates GUIs the way a person does. The 72.60% figure on OSWorld comes from conditions that include multiple attempts, but it still marks a milestone for this field. It is a strong option when you need to automate an application without an API, or when you want to inspect how a computer-use agent works for yourself. At the same time, model API costs and the risk of misoperation remain the user’s responsibility, so starting with low-impact tasks is the practical approach.