UI-Venus

🌟 What's New in UI-Venus-2

Compared with UI-Venus-1.5, we introduce:

• 📱 Scaled multilingual mobile environments: A substantially expanded executable mobile pool covering Chinese and English app ecosystems, paired with a deep-research-driven query-generation strategy that grounds task queries in real application functionality — improving the accuracy, validity, and executability of generated instructions.
• 🖥️ Computer use, built from the ground up: Dedicated desktop operating-system capabilities constructed from scratch through computer-use data collection and task-specific training, extending the UI-Venus family to mobile, web, and OS interaction in one unified end-to-end agent.
• 🎯 Keypoint-grounded verification: Task completion is judged on task-relevant visual keypoints rather than a coarse holistic look at the final screen, with multi-model voting aggregating heterogeneous judges — reducing single-judge bias and making the reward signal robust to reward hacking.
• 🔄 Verification-augmented reflection: Verified feedback is distilled back into training as reflection supervision, so the agent can distinguish partial progress from true completion, avoid premature termination caused by observation misinterpretation, and recover during long-horizon interaction.

---

Performances of UI-Venus-2 on GUI-agent benchmarks. Each panel compares UI-Venus-2-27B and UI-Venus-2-9B with some selected strong baselines. We favor standalone end-to-end systems evaluated on the closest available task subset and step budget; source-reported action scaffolds may still differ. MobileWorld uses GUI-only success rate on 117 tasks with 50 steps, WebVoyager uses the refreshed 595-task split, Odysseys uses average rubric score over 200 tasks, VenusBench-CAPTCHA uses micro Pass@1 over all 219 examples, and VenusBench-GD uses English-instruction micro-average accuracy. “*” denotes results reproduced by us.

---

📰 News

• [2026/10] We release LBS (Learning to Look Beyond Screens), which learns transferable visual observation using only GUI grounding training data and generalizes to general-domain visual question answering. See branch LBS.
• [2026/09] We release VenusBench-CAPTCHA, a comprehensive benchmark for evaluating GUI agents on diverse CAPTCHA tasks. See branch VenusBench-CAPTCHA.
• [2026/08] We release UI-Venus-2, a 9B/27B general-purpose foundation GUI agent that unifies mobile, web, and desktop interaction with scaled multilingual environments, keypoint-grounded verification, and verification-augmented reflection.
• [2026/02] We release UI-Venus-1.5, an end-to-end GUI Agent designed for robust real-world applications.
• [2026/02] We release VenusBench-Mobile, a challenging online benchmark for mobile GUI agents. See branch VenusBench-Mobile.
• [2025/12] We release VenusBench-GD, a comprehensive multi-platform GUI grounding benchmark. See branch VenusBench-GD.
• [2025/8] We release UI-Venus, the first version of our UI agent model.

---

🧭 Overview

• Demo
• Venus Framework
• Quick Start
• Benchmark Results
• Contact
• Citation

---

✨ Demo

See more demos

---

🛠 Venus Framework

We provide two frameworks for running agents in real environments:

Framework	Description	Documentation
Mobile Framework	Android/ADB agent framework for single-task execution, multi-device batch execution, trajectory recording, and reflection.	English · 中文
Browser Plugin	Chrome Side Panel extension that connects UI-Venus to the active browser tab and executes browser tasks interactively.	English · 中文

See the Venus Framework overview for the directory layout and entry points. The lightweight domain examples below can be used without either framework unless their individual requirements state otherwise.

---

🚀 Quick Start

Installation

conda create -n ui-venus-2 python=3.11 -y
conda activate ui-venus-2
pip install -r requirements.txt

Python 3.10 or newer is required. All commands below are executed from the repository root. Configure the OpenAI-compatible model service through MODEL_URL, MODEL_NAME, and either API_KEY or MODEL_API_KEY; the same values can also be edited at the top of each domain script. API_KEY takes precedence when both key variables are set. Set MODEL_NAME to the served name of your UI-Venus-2 9B or 27B model.

Mobile

Run multi-turn inference over the included prerecorded screenshot sequence. N_IMG controls how many recent historical screenshots are retained:

MODEL_URL=http://127.0.0.1:8000/v1 \
MODEL_NAME=UI-Venus-2 \
N_IMG=2 \
bash scripts/mobile.sh

This example performs model inference only and does not execute actions on a device. Use the Mobile Framework above for real-device ADB automation.

Mobile multi-turn example and input/output format

Computer

Run multi-turn Computer inference over a prerecorded desktop screenshot sequence:

MODEL_URL=http://127.0.0.1:8000/v1 \
MODEL_NAME=UI-Venus-2 \
N_IMG=2 \
bash scripts/computer.sh

The default command uses the included desktop screenshot sample. The standalone example validates and normalizes model actions but does not execute them on the host. It has no runtime dependency on OSWorld.

Computer multi-turn example and action format

Browser

Start Chrome with a CDP port as described in the domain documentation, then run one natural-language browser task:

MODEL_URL=http://127.0.0.1:8000/v1 \
MODEL_NAME=UI-Venus-2 \
bash scripts/browser.sh "Open https://example.com and report the page title"

Browser usage and CDP setup

Grounding

Run the direct grounding evaluation on the three samples included with the repository:

MODEL_URL=http://127.0.0.1:8000/v1 \
MODEL_NAME=UI-Venus-2 \
bash scripts/grounding.sh

Grounding evaluation, smoke test, and benchmark configuration

CAPTCHA

Run inference on the included CAPTCHA image and save the parsed JSON and visualization under results/captcha/:

MODEL_URL=http://127.0.0.1:8000/v1 \
MODEL_NAME=UI-Venus-2 \
bash scripts/captcha.sh

CAPTCHA usage, prompt, action format, and visualization

---

📊 Benchmark Results

Mobile

Performance comparison on various mobile GUI benchmarks. VenusBench-Mobile reports success rate on its 149-task primary pool. For MobileWorld, we report GUI-only success rate on 117 tasks under the 50-step setting; values in parentheses, when available, use 100 steps. MemGUI reports Main Results Pass@1. * denotes baseline results evaluated or reproduced by us.

Computer

Performance comparison on computer-use agent benchmarks: OSWorld-Verified and DeskCraft (left), and OSWorld 2.0 (right) under the official 150-step budget with 108 tasks. Reported OSWorld-Verified baselines use the 361-task setting in their cited source and may use model-specific action scaffolds. For DeskCraft, we report an author-evaluated aggregate over the 538-task union of the Standard and Interactive splits, which differs from the benchmark's official split-level reporting. OSWorld 2.0 results report the official Binary Accuracy and Partial Score metrics; baselines are taken from the official leaderboard, possibly with model-specific tool settings, and the reasoning-effort setting is labeled in parentheses for models with multiple official entries. * indicates baseline results evaluated by us.

Browser

Performance comparison on four live-web benchmarks: WebVoyager, Online-Mind2Web, REAL, and Odysseys. The Fara1.5 and GPT-5 (SoM) WebVoyager entries use the refreshed 595-task, 100-step robust protocol and are averaged over three runs; live-site states may vary by evaluation date. For Odysseys, we report both the averaged rubric score (Avg.) and the perfect rubric score (Perfect). * indicates our reproduced results.

Grounding

Performance comparison on various Grounding Benchmarks. VenusBench-GD reports English-instruction micro-average point-in-box accuracy. * indicates baselines evaluated or reproduced by us.

CAPTCHA

Performance Comparison across CAPTCHA Benchmarks. All results are Pass@1 percentages, and higher is better. We evaluate on VenusBench-CAPTCHA and four public benchmarks: MCA-Bench, Spatial-CAPTCHA-Bench, NextGen-CAPTCHAs, and Open CaptchaWorld. We use 1,000 sampled MCA-Bench examples, 15 NextGen-CAPTCHAs task types, and 16 Open CaptchaWorld task types.

---

📬 Contact

For any questions or collaboration, please contact the maintainers.

---

📚 Citation

# UI-Venus-2
@misc{venusteam2026uivenus2technicalreport,
      title={UI-Venus-2 Technical Report}, 
      author={Venus Team and Zhuohan Cai and Haoxing Chen and Jiaxuan Chen and Weizhi Chen and Changlong Gao and Zhangxuan Gu and Yuan Guo and Yusong Hu and Jianrong Jiang and Jianguo Li and Runze Li and Jinzhen Lin and Zhenyu Ma and Changhua Meng and Han Peng and Xinyu Qiu and Shuheng Shen and Zhongyi Shui and Weiqiang Wang and Ming Wen and Zhuoer Xu and Hang Yan and Kaiwen Yang and Ruilin Yao and Nanjun Yu and Zhengwen Zeng and Lianrui Zhang and Yunzhu Zhang and Zhe Zhao and Beitong Zhou},
      year={2026},
      eprint={2609.00028},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2609.00028},
}

# UI-Venus-1.5
@misc{venusteam2026uivenus15technicalreport,
      title={UI-Venus-1.5 Technical Report}, 
      author={Venus-Team and Changlong Gao and Zhangxuan Gu and Yulin Liu and Xinyu Qiu and Shuheng Shen and Yue Wen and Tianyu Xia and Zhenyu Xu and Zhengwen Zeng and Beitong Zhou and Xingran Zhou and Weizhi Chen and Sunhao Dai and Jingya Dou and Yichen Gong and Yuan Guo and Zhenlin Guo and Feng Li and Qian Li and Jinzhen Lin and Yuqi Zhou and Linchao Zhu and Liang Chen and Zhenyu Guo and Changhua Meng and Weiqiang Wang},
      year={2026},
      eprint={2602.09082},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2602.09082}, 
}

# UI-Venus
@misc{gu2025uivenustechnicalreportbuilding,
      title={UI-Venus Technical Report: Building High-performance UI Agents with RFT}, 
      author={Zhangxuan Gu and Zhengwen Zeng and Zhenyu Xu and others},
      year={2025},
      eprint={2508.10833},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2508.10833}, 
}

---

⚖️ License

This project is for research and educational purposes only.