游戏库不在主仓库里，需单独 clone GameWorld-Games 到 games/gameworld-games 才能跑任务；调用闭源模型要先 export 对应厂商的 API key，本地模型则需自行用 vLLM 起服务；监控面板默认监听 127.0.0.1:8787，端口被占用时需手动改 --port。
conda create -n gameworld python=3.12
conda activate gameworld
pip install -r requirements.txt
playwright install chromium
git clone https://github.com/gameworld-project/GameWorld-Games.git games/gameworld-games 硬件要求按 README 说明
README 未说明显存或 GPU 型号要求；本地自托管模型需用 vLLM 起服务，浏览器端依赖 Playwright 的 Chromium。
在浏览器里跑 34 款游戏、170 个任务，用可验证的游戏状态给多模态游戏智能体打分
面向多模态游戏智能体的标准化评测基准，提供可验证的评测流程，用来衡量智能体在游戏环境中的真实表现。
核心能力
- 覆盖 34 款游戏、170 个任务，分 Puzzle/Platformer/Simulation/Arcade/Runner 五类
- 采用基于结果的状态可验证评测，而非只看截图或人工打分
- 提供 Fine-Control Protocol 分支，单局 400 步、更高频统一交互
- 内置并行跑批与本地监控面板，实时查看多任务运行进度
- 支持导出交互轨迹为 MP4，并公开基准轨迹数据集
容易踩的坑
内容由模型根据 README 生成，可能不完整或不准确，以官方文档为准。看原文 README
暂无 release 数据（可能没有发版，或尚未同步）。
用户点评
登录用户的真实使用记录，可按版本和硬件筛选共 0 条
还没有人点评。第一个说出真实体验的人最有价值。
GameWorld benchmarks multimodal game agents across 34 games and 170 tasks in a browser-based environment, using outcome-based, state-verifiable evaluation.
📢 Updates
[!IMPORTANT] ✨ New release: We have released the Fine-Control Protocol (branch: fine-control-400-steps-202608) and recommend it for higher-frequency, unified interaction with a 400-step limit per episode. Its finer control granularity better reveals models’ decision-making and control capabilities.
- 2026.10: 🔭 VISTA improved GameWorld's success rate from 40.0% to 63.3% with GPT-5.6 Sol. Thanks to the VISTA team for sharing their visual harness!
- 2026.09: GameWorld received the Best Paper Runner-Up Award and was invited to give a spotlight talk at the MDA Workshop @ ECCV 2026.
- 2026.06: GameWorld was used as a unified digital agent benchmark in Seed-2.1 Model Card.
- 2026.06: GameWorld was accepted to ECCV 2026.
- 2026.04: The full game library for benchmarking is available at gameworld-project/GameWorld-Games.
- 2026.04: GameWorld launched with its Technical Report and Project Page.
🏆 GameWorld (Fine-Control Protocol, 400-Step Limit) Leaderboard
Each model was evaluated on 170 tasks with a 400-step limit per run. Qwen scores average three runs per task; Gemini and GPT scores use one run per task. SR is the success rate, and PG is mean final task progress.
| Model | Runs/task | CUA SR | CUA PG | General SR | General PG |
|---|---|---|---|---|---|
| Qwen3.5-9B | 3 | 14.51% | 35.09% | 17.65% | 39.32% |
| Qwen3.5-27B | 3 | 21.96% | 41.79% | 16.27% | 39.59% |
| Qwen3.5-35B-A3B | 3 | 22.94% | 43.82% | 19.41% | 39.38% |
| Qwen3.5-122B-A10B | 3 | 23.14% | 45.23% | 19.02% | 41.87% |
| Qwen3.6-27B | 3 | 21.18% | 43.60% | 18.82% | 43.25% |
| Qwen3.6-35B-A3B | 3 | 19.22% | 40.57% | 15.69% | 38.84% |
| Qwen3.8-27B | 3 | 29.22% | 54.64% | 24.12% | 49.34% |
| Qwen3.8-Flash-Next [Trajectory data] | 3 | 36.86% | 61.48% | 29.80% | 55.86% |
| Gemini-3.8-Flash [Trajectory data] | 1 | 36.47% | 60.11% | 42.35% | 63.52% |
| GPT-6.1-Terra [Trajectory data] | 1 | 55.29% | 72.53% | 61.18% | 76.79% |
📦 Installation
Python and browser environment:
conda create -n gameworld python=3.12
conda activate gameworld
pip install -r requirements.txt
playwright install chromium
Set the API keys for the providers you plan to use:
export GOOGLE_API_KEY=...
export OPENAI_API_KEY=...
export ANTHROPIC_API_KEY=...
Or host your own models locally with vLLM.
vllm serve Qwen/Qwen3.5-122B-A10B --port 8088
Clone the full game library from gameworld-project/GameWorld-Games.
git clone https://github.com/gameworld-project/GameWorld-Games.git games/gameworld-games
More setup notes: docs/install/INSTALLATION.md.
🚀 Quick Start
First, validate that the browser and runtime are set up correctly:
python human_play.py --game 10_doodle-jump
Run a single preset:
python main.py --config 10_doodle-jump+10_01+gpt-5.2 --headed
Run a benchmark experiment:
python run_benchmark.py --experiment benchmark/experiments/quick-start.yaml --max-parallel 5
🖥️ Results and Monitoring
Results are saved to results/run_<session>_<game>_<task>_<model>/. The
canonical run data is stored in run_meta.json,
agent_N/interactions.jsonl, and agent_0/evaluation/evaluation.json.
We recommend using the dashboard to monitor the parallel runs. To launch the dashboard, run:
python -m tools.monitor.server --results-dir results --host 127.0.0.1 --port 8787 --open-browser
MP4 export is an explicit offline operation:
python tools/export_video.py --logs-dir results --session <run_id>
📚 Documentation
See docs/ for full documentation.
💬 Game Agent Community
🎙️ Join our Discord to discuss GameWorld, ask questions, and share your thoughts on multimodal game agents. GLHF!
📆 TODO
- Release GameWorld leaderboard.
📖 BibTeX
If you find GameWorld useful for your research, please kindly cite:
@article{ouyang2026gameworld,
title={GameWorld: Towards Standardized and Verifiable Evaluation of Multimodal Game Agents},
author={Ouyang, Mingyu and Hu, Siyuan and Lin, Kevin Qinghong and Ng, Hwee Tou and Shou, Mike Zheng},
journal={arXiv preprint arXiv:2604.07429},
year={2026},
}