Awesome-CUA

A source-backed collection of papers, models, datasets, benchmarks and tools for agents that use computers.

English · 简体中文

Search the website · Papers · Topics · Data · Agent guide · Contribute

If this collection is useful, a star on GitHub helps others find it.

Curated entries were reviewed against a first-party source snapshot; records awaiting review have their bibliographic identity checked; candidates are discovered identifiers awaiting resolution. What “verified” means · Coverage ledger

Featured benchmarks

Website reconstruction	Browser agents
Can AI agents rebuild websites through browser exploration?★ Star on GitHub	Open-source benchmark for browser AI agents on daily tasks.★ Star on GitHub

Featured: Awesome Reverse Engineering Anything — a collection of tools, case studies and research for reverse-engineering websites, games, apps, native binaries, firmware and protocols.

Find by task

Your task	Start here	What you can find
Find papers	Research library · Topics · Paper–asset connections	Reviewed papers by year and topic, with original sources and the code, datasets and models linked to them through reviewed primary evidence.
Choose a benchmark	Benchmark guide	Platforms, tasks, evaluators, and versions to compare.
Find training data	Training-data guide	Supervision, splits, count units, and data licenses.
Build an agent	Agents · Frameworks	Implementation entry points, supported platforms, and papers.
Use RAG or an agent	Agent guide · Data hub	Complete records or retrieval documents with review states and sources.

cd Awesome-CUA
python3 scripts/catalog.py search "grounding" --kind paper --status curated --limit 0 --json
python3 scripts/catalog.py search --kind benchmark --environment web --status curated --limit 0 --json
python3 scripts/catalog.py search --kind dataset --environment android --status curated --limit 0 --json
python3 scripts/catalog.py search --kind framework --status curated --limit 0 --json
python3 scripts/export_rag.py --output dist/rag-documents.jsonl
python3 scripts/catalog.py get arxiv:2404.07972 --related --json

4,404 curated resources · 1,690 reviewed papers and surveys · 104 discovery documents

Evidence through 2026-11-09T05:26:45Z. 451 records awaiting review have verified bibliographic metadata, and 337 discovered candidates await resolution. 452 candidates have an out-of-scope review. 1,308 scope exclusions retain their original metadata in the archive.

Explore

Browse	What you find	Entries
Research by topic	Reviewed papers grouped by topic	1,661
Full bibliography	All paper and survey records, including those awaiting review	2,141
Surveys	Surveys of computer-use agent research	29
Learning resources	Courses and tutorials	12
Datasets	Training and evaluation data for GUI agents	185
Benchmarks	Evaluation environments and task suites	254
Models and checkpoint releases	Checkpoints and model families	271
Agents	Open implementations that act on a computer	390
Frameworks	Code for building and evaluating agents	607
Infrastructure	Sandboxes, VMs, device farms and environments	74
Tools	Evaluation harnesses, recorders and other utilities	836
Products	Computer-use products from vendors and projects	66
Publishers and brands	Publishers and brands connected to cataloged resources	19

By platform: Web 2,254 · Desktop 1,508 · Windows 471 · macOS 461 · Linux 425 · Android 617 · iOS 64 · HarmonyOS 4 · Cross-platform 628

Coverage spans web, desktop, Android, iOS, Windows, macOS, Linux, and cross-platform work. Topics include grounding, screen understanding, planning, memory, long-horizon workflows, reinforcement learning, data generation, safety, robustness, and evaluation.

Use the data

Download the catalog, feeds and RAG documents; the data hub also lists sources, review states and schemas.

• llms.txt - Navigation file for language models.
• llms-full.txt - Curated annotations in Markdown.
• Data hub - Catalog, RAG documents, citations, topics, comparisons, sources and schemas in six groups.
• Catalog JSONL - Complete catalog with one resource per line.
• Download manifest - Export counts, byte sizes and SHA-256 checksums.
• Atom feed - Research updates as an Atom feed.
• Hugging Face dataset - Read-only mirror; metadata_only and github_candidates are unreviewed.
• Agent and RAG guide - Guide for agents and RAG systems.

Clone the complete catalog:

git clone https://github.com/reacher-z/Awesome-CUA.git

Agent instructions · Retrieval documents

Discovery is scheduled at 00:07, 08:07 and 16:07 UTC. Actual scan results · Schedule and safeguards.

Read the research survey — Methods, evaluation, human agency, and reusable research assets. English working manuscript with a versioned bibliography and reading register.

Highlights

The most-mentioned curated resources per category: the count is how many other cataloged resources mention each one in their reviewed first-party source (README, model or dataset card). It reflects connectedness inside this catalog. Every category page lists all reviewed entries. Surveys come first, newest first.

Start with the newest surveys · all 29 →

Date	Survey	Environments
2026‑10‑21	Multimodal Embodied Agents: A Unified Capability-centric Perspective from Computer-Use to Robot-Use	unspecified
2026‑10‑14	Seeing, Maintaining, and Learning: The Evolution of Multimodal Agent Memory	unspecified
2026‑09‑02	Efficient GUI Agents: A Systems Survey of Observation, Memory, Action, and Runtime Optimization	cross-platform
2026‑08‑30	A Systematic Survey of Agentic Skills: Architecture, Lifecycle, and Security	cross-platform
2026‑08‑10	Software Engineering for and with GUI Agent	web

Benchmarks · all 254 →

Resource	Mentions	Environments	Paper	License	Description
OSWorld	54	desktop, linux, windows, macos	2404.07972 (2024)	code Apache-2.0	Execution-based evaluation of agents performing tasks in real desktop environments.
WebArena	22	web	2307.13854 (2023)	code Apache-2.0	Self-hosted web environments and tasks for evaluating autonomous web agents.
AndroidWorld	17	android	2405.14573 (2024)	code Apache-2.0	Android environment with programmatic task evaluation for mobile agents.
BrowserGym	14	web	2412.05467 (2024)	code Apache-2.0	Unified interfaces for browser-agent tasks and benchmark environments.
ScreenSpot Pro	13	desktop, windows, macos, linux	—	code MIT	GUI grounding benchmark focused on professional high-resolution interfaces.
Mind2Web	10	web	2306.06070 (2023)	code MIT	Human demonstrations and tasks for generalist web navigation research.

Models · all 271 →

Resource	Mentions	Environments	Paper	License	Description
UI-TARS	22	cross-platform	2501.12326 (2025)	code Apache-2.0	Native GUI agent models; UI-TARS-1.5 adds reinforcement-learned reasoning before acting and an open 7B checkpoint, and UI-TARS-2 extends to games, code and tool use.
SeeClick	16	cross-platform	2401.10935 (2024)	code Apache-2.0	GUI grounding model trained to locate visual interface elements.
Qwen2-VL-7B-Instruct	11	unspecified	—	weights apache-2.0	An instruction-tuned 7B multimodal model for image and video understanding; its card describes visually guided operation of mobile devices.
Aguvis	7	cross-platform	2412.04454 (2024)	—	Vision-based GUI agent models with unified action representations.
Holo4 27B BF16	7	web, desktop, cross-platform	—	weights CC-BY-NC-4.0	Dense 27B computer-use checkpoint from H Company built on Qwen3.8-27B; an external harness sends it screenshots and tool results and executes the clicks, typing, code and tool calls it requests.
Holo4 35B-A3B BF16	7	desktop, web	—	weights apache-2.0	A Qwen3.6-based computer-use checkpoint designed to request GUI actions, code execution and tool calls through an external agent harness.

Datasets · all 185 →

Resource	Mentions	Environments	Paper	License	Description
OS-Genesis	11	cross-platform	2412.19723 (2024)	—	Interaction-driven pipeline that synthesizes mobile and web GUI agent trajectories without human supervision or predefined tasks, via reverse task synthesis and a trajectory reward model.
ScreenSpot-Pro data	11	desktop, windows, macos, linux	2504.07981 (2025)	data mit	High-resolution professional interface screenshots and target annotations for evaluating instruction-based GUI localization.
AMEX dataset release	7	android	2407.17490 (2024)	data cc-by-4.0	Official Hub release of AMEX: Android app screenshots annotated for interactive-element grounding, screen and element functionality, and instructions with stepwise GUI-action chains.
Holo4 Trajectories	6	web, desktop, android, cross-platform	—	data apache-2.0	Dataset of 7,366 computer-use agent execution trajectories from Holo4 27B and Holo4 35B-A3B, recording step-level reasoning, actions, tool results, and screenshots.
GroundCUA dataset	5	desktop, windows, macos, linux	2511.07332 (2025)	data mit	Desktop GUI-grounding data from the GroundCUA project: PNG screenshots of 87 applications paired by hash with JSON lists of element boxes, visible text and optional coarse categories.
GUI-Actor data	5	unspecified	2506.03143 (2025)	—	A processed screenshot and bounding-box collection released for training GUI-Actor visual grounding models.

Frameworks · all 607 →

Resource	Mentions	Environments	Paper	License	Description
Browser Use	52	web	—	code MIT	Open-source browser agent library that runs locally with any LLM against a local or cloud browser, and also serves as a CLI for coding agents and as a driver for Claude's browser toolset.
Cua	35	desktop	—	code MIT	Monorepo behind Cua Spaces, which gives agents full macOS or Linux desktops, plus Cua Driver for operating native apps, Lume for Apple Silicon VMs, Cua Bench and the CUA-S1 decision models.
SCALE-CUA (THUDM)	12	desktop	2607.11185 (2026)	—	Computer-use training framework with VeriGen task synthesis, adaptive rollout sampling, and visual-context segmentation.
AgentLab	7	web	2412.05467 (2024)	code Apache-2.0	Experiment framework for developing and evaluating browser agents.
Stagehand	7	web	—	code MIT	Browser automation framework combining code and natural-language actions.
hai-agents	6	cross-platform, web, desktop	—	code MIT	hai-agents is a Python SDK and CLI for launching browser and desktop agents through H Company's Agents API and monitoring their sessions.

Agents · all 390 →

Resource	Mentions	Environments	Paper	License	Description
Mobile-Agent	27	android, cross-platform	2401.16158 (2024), 2406.01014 (2024), 2508.15144 (2025), 2602.16855 (2026)	code MIT	Mobile device agent family using visual interface interaction.
Hermes Agent	20	cross-platform, desktop, web, linux, macos, windows	—	code MIT	Hermes Agent is a self-improving autonomous agent by Nous Research with procedural skill synthesis, browser control, and multi-platform sandboxes.
OpenAI Codex CLI	14	cross-platform, desktop, macos, linux, windows	—	code Apache-2.0	OpenAI Codex CLI is a local terminal and desktop coding agent that executes commands and edits files inside OS-enforced sandboxes.
OpenClaw	13	cross-platform, desktop, web, macos, windows, linux	—	code MIT	OpenClaw is an open-source personal computer-use assistant that executes browser, desktop, terminal, and messaging workflows across local devices.
Gemini CLI	9	cross-platform, desktop, macos, linux, windows	—	code Apache-2.0	Gemini CLI is Google's open-source terminal agent providing shell execution, file manipulation, web search, and MCP tool integration.
Agent S	8	desktop	2410.08164 (2024), 2504.00906 (2025), 2510.02250 (2025)	code Apache-2.0	Simular's open-source framework for computer-use agents that click, type and scroll through desktop and web apps on macOS, Windows and Linux; its third generation, Agent S3, reports 72.6% on OSWorld.

Recently reviewed

Newest editorial approvals of models, datasets, benchmarks, systems, and tools (newest first). New papers appear by date in the topic hubs.

Reviewed	Resource	Kind	Environments	Description
2026‑11‑09	Qwen2.5-VL-3B-Instruct	model	desktop, android, web, cross-platform	An instruction-tuned vision-language checkpoint that reads visual layouts, localizes points or boxes, and supports computer and phone use. Its card reports screen and mobile-agent evaluations.
2026‑11‑09	Dreamer-7B-Shopping	model	web	A shopping-domain Dreamer-7B checkpoint for WebDreamer, which uses screenshot-conditioned predictions of candidate web actions to support agent planning.
2026‑11‑09	AndroidWorld-SeeAct-V	agent	android	AndroidWorld-SeeAct-V runs a visual Android agent that plans the next action and grounds coordinate actions in screenshots. The repository includes code for AndroidWorld evaluation.
2026‑11‑09	Process Discovery	product	cross-platform, desktop, windows, web	An enterprise process-discovery product that captures activity across desktops and web apps, then maps workflows to identify patterns and automation opportunities.
2026‑11‑09	Muse-Glimmer-30B-GUI-Grounding-Fast	model	cross-platform, desktop, windows, macos, android, ios, web	A LoRA adapter for Muse-Glimmer-30B that returns a UI element's bounding box from a screenshot and instruction, with examples spanning desktop, mobile, and web screens.
2026‑11‑09	Jedi-3B-1080p	model	unspecified	A Qwen2.5-VL-3B-based computer-use grounding checkpoint with an example that predicts mouse-action coordinates from an interface image.
2026‑11‑09	HobbyLM-1B Computer Use	model	desktop, macos	An on-device LoRA adapter that turns English requests and Mac state into single actions executed by a companion app, with training code and evaluation records.
2026‑11‑09	GUI-Actor-Verifier-2B	model	unspecified	A GUI grounding verifier that checks whether a marked screenshot position matches an instruction and provides a score for candidate action positions.
2026‑11‑09	OpenCUA-7B	model	desktop, linux, windows, macos	A Qwen2.5-VL-based computer-use checkpoint that produces desktop actions from screenshots and instructions, using recent screen history for context.
2026‑11‑09	Hronaut	infrastructure	web	A local browser workspace for agents, exposed through MCP. It supports isolated task sessions, visible browser use and reconnection to existing tabs.
2026‑11‑09	UI-Ins-7B	model	android	UI-Ins-7B is a GUI-grounding checkpoint trained on varied instruction formulations and reinforcement learning to locate interface elements for agent actions.
2026‑11‑09	Holo4-27B-NVFP4	model	cross-platform, web, desktop	A quantized Holo4 vision-language checkpoint for computer use across web, desktop, and mobile interfaces, with a harness that executes requested GUI and tool actions.

Data files and local queries

Browse the interactive website to search and filter all resources, inspect source evidence, switch between English and Chinese, or download the complete catalog. Website development and publishing.

The English data hub and 中文数据中心 organize public downloads into six groups: complete catalog, RAG documents, citations, topics and comparisons, sources and review status, and schemas. Cloning still gives you the complete source catalog; the hub provides exports from its published snapshot.

The download manifest records each listed export's actual count and unit, scope, byte size, and SHA-256 fingerprint. Compare catalog_sha256 across manifests before combining files from different downloads.

Reviewed research · Computer-use agent guides · Static resource directory · Search and AI discovery

• Catalog JSONL: one resource per line, with stable IDs, typed fields, provenance, scoped licenses, and review status.
• Reference graph: outbound references from primary sources.
• Discovery registry and candidates: source snapshots and where each identifier was found.
• Review queue: unresolved evidence, unindexed discoveries, and proposals requiring editorial work. Discovered links reviewed as non-resources or as other addresses of indexed resources stay listed with a resolution and their reason.
• JSON Schema and coverage JSON: machine-checkable contracts and coverage accounting.
• BibTeX and CSL JSON: reviewed-paper citations with recorded authors, dates, and versions. Each resource page also provides its own record.json.

# Python 3.11+. Search and export require only the standard library.
cd Awesome-CUA
python3 scripts/catalog.py search "grounding" --kind model
python3 scripts/catalog.py search --kind dataset --environment android
python3 scripts/catalog.py search --kind product --status curated --json
python3 scripts/catalog.py get arxiv:2404.07972 --related --json
python3 scripts/catalog.py build
# Creates dist/catalog.sqlite and dist/catalog.json, with no network calls.
python3 scripts/export_rag.py --output dist/rag-documents.jsonl
python3 scripts/export_citations.py --format bibtex --output dist/citations.bib
# Add --include-metadata to include every active record, with its review state intact.

Use SQLite to join resources, environment/topic facets, and reviewed relations. See query examples. data/catalog.jsonl is committed; SQLite and JSON array exports are reproducible local build products.

What “verified” means

State	What has been checked
curated	AI-assisted editorial review against a first-party source snapshot, with name, scope, summary, and taxonomy
metadata_verified	Bibliographic identity, title, authors, and recorded dates checked against retained arXiv or publication sources
candidate	A normalized URL or identifier was discovered

A paper, repository, dataset release, model checkpoint, product, and publisher can be separate related resources. Model families and individual checkpoints are explicitly linked where reviewed.

Code, data, weights, and document licenses are separate. Missing values are null.

Maintain and verify

python3 -m venv .venv
.venv/bin/python -m pip install --require-hashes -r requirements-dev.txt
.venv/bin/python scripts/catalog.py validate
.venv/bin/python scripts/catalog.py build --check
.venv/bin/python -m unittest discover -s tests -v
.venv/bin/python scripts/catalog.py build
node --test tests/web.test.mjs
.venv/bin/python scripts/build_site.py

These commands match the checks that Catalog quality runs on every push and pull request, in the same order. They need Python 3.11+ and Node.js 24+, the engines version in package.json (CI uses Python 3.12 and Node.js 24). The web tests use only the built-in Node.js test runner, so nothing needs to be installed with npm. npm run test:web and npm run build:web are shortcuts for the last two commands.

Network discovery and verification are explicit commands; ordinary builds are offline. Changed primary snapshots require another editorial review before publication. Maintenance guide · Methodology · Data dictionary

Acknowledgments and reuse

The source registry credits the community collections and surveys used for discovery. Bibliographic records use retained arXiv, publisher, and author sources; publication observations preserve edition-specific titles, authors, and date precision. Selected assets are checked against publisher repositories, documentation, and model/dataset cards. Upstream READMEs, papers, model weights, and datasets are not redistributed here.

Original repository code and annotations use the MIT license. Linked resources retain their own terms.

To cite the catalog, use CITATION.cff or GitHub's "Cite this repository" button. The data changes with every scan, so pin the exact version you used. Each v* tag has a data release with versioned downloads built from the tagged commit: the catalog as JSONL, JSON and SQLite, the coverage ledger, topic and profile exports, and their SHA256SUMS. Cite the release tag, or record the commit from git rev-parse HEAD in your clone, or link files at that commit, such as https://github.com/reacher-z/Awesome-CUA/blob/<commit>/data/catalog.jsonl. Cutting a release.

Star History

View Awesome-CUA on Star History