开发环境需 uv 0.11.8 或更高版本；uv 默认只解析发布满七天的版本，模型厂商 SDK 与 Inspect 自身包除外；改前端需先初始化 src/inspect_ai/_view/ts-mono/ 子模块，纯 Python 开发可跳过
git clone https://github.com/UKGovernmentBEIS/inspect_ai.git
cd inspect_ai
uv sync 硬件要求按 README 说明
README 未说明
英国 AI 安全研究所开源的大模型评测框架，内置 200+ 现成评测可直接跑
Inspect 是一个用 Python 编写的大语言模型评测框架，帮助研究者系统化地设计、运行和分析模型评估实验。
核心能力
- 内置提示工程、工具调用、多轮对话与模型打分评测组件
- 提供 200 多个预置评测，可直接在任意模型上运行
- 支持通过独立 Python 包扩展新的诱导与评分方法
- 附带 Web UI 查看评测结果，前端为 TypeScript 子模块
- 文档提供 llms.txt 等结构化索引，便于编码代理读取
容易踩的坑
内容由模型根据 README 生成，可能不完整或不准确，以官方文档为准。看原文 README
- 0.3.288v0.3.288最新正式版2026-11-07 · 2026-11-07 01:13 UTC
0.3.288 (2026-11-06)
Breaking Changes
- environments with
google-genai2.xfail thevalidate_google_clientversion check. Installgoogle-genai>=3.0.0. - Evaluations using the automatically generated Docker sandbox must provide an explicit image if they need the
nonrootaccount,pipx, or Playwright browser dependencies. - Replace calls to
web_browser()or itsweb_browser_tools()alias withweb_search()for retrieval, orcomputer()/ sandbox MCP tools for browser interaction; see #5497 for migration options.
Features
- a runtime can keep a react() or deepagent() loop going after it answers (#6212) (cea079d)
- custom tools inside a sandbox identity scope run as the scope's user (#6316) (572e288)
- experimental transcript events for agent communication and agent swarm evidence (#6164) (cba2b16)
- inspect ctl sample agent lists a sample's named agents and messages, pauses, resumes or stops them (#6297) (0a56f69)
- remove deprecated web browser tools (#6264) (de4b309)
- run each agent's sandbox tools as its own user (#6190) (f59945e)
- sentinel: replay logs that do not record model-call origins, with turns inferred from the timeline (#6098) (7cf09c9)
- use the official Python image for default Docker sandboxes (#6238) (cb7645f)
Bug Fixes
inspect viewno longer terminates an unrelated process that reused a crashed server's pid (#6247) (689a92e)- a bridged agent's retries after a limit no longer pay for a compaction each (#6304) (3c75374)
- a sample limit reached by overlapping bridged generations ends the sample after an inner limit (#6263) (642ee4b)
- aborting one task while a sample fails no longer stops the whole run (#6272) (1ddc34d)
- agent-bridge: Gemini tool calls through the chat completions bridge keep their thought signatures (#5892) (00d9882)
- an interr
- environments with
- 0.3.287v0.3.2872026-10-31 · 2026-10-31 16:33 UTC
查看更新说明
0.3.287 (2026-10-31)
Features
- a runtime can hold and rewrite a react() conversation at turn boundaries, and compaction handlers gain reset() (#6201) (7340817)
- reserve the range_goal registry type for inspect_ranges goal types (#6224) (3dcbf00)
- sentinel: the sentinel host gives the input budget and the token count of a monitor model (#6132) (f1074f9)
Bug Fixes
- an agent limit reached by a sandbox-bridged agent ends that agent, not the sample (#6123) (c0b94c3)
- anthropic: unrecognised Claude models fail with "Invalid signature in thinking block" when an agent's tools change (#6206) (18216c3)
- bridge: a limit raised in a bridged host tool no longer ends the sample (#6129) (3896cba)
- bridge: approval policies now see a bridged tool call under the tool it runs, whatever name the agent declared (#6192) (7937b03)
- Docker file operations report unknown users as unsupported on Docker Engine 28 (#6152) (c300b90)
- exclude dataclass non-init fields from tool parameters (#5283) (f862a6c)
- exec_remote keeps all command output, and a change to any sandbox-tools build input needs a version bump (v37) (#6157) (19a1177)
- google: Gemini long decode responses no longer stop at the per-request output cap (#6205) (aee6aaa)
- inspect score no longer imports a file a log names as a scorer, and prints what it loads from the log (#6085) (c07fcbf)
- model_info() no longer constructs a model provider for a model named in a log (#6084) (96fc9a4)
- parse HTTP(S) URL path when selecting reader in precomputed_scores (#6078) (19d120f)
- reject blank choices that shift answer letters ([#5766](https://github.com/UKGovernmentBEIS/inspect_ai/
- 0.3.286v0.3.2862026-10-28 · 2026-10-28 14:22 UTC
查看更新说明
0.3.286 (2026-10-28)
Breaking Changes
- Docker read_file() now uses the container's default execution user instead of Docker's privileged copy API. Callers needing privileged reads must request user="root"; callers acting for a different sandbox user must pass that user. Docker reads require tar with --no-recursion, /bin/sh and /usr/bin/timeout, and no longer retry the former compose-cp host-timeout path. Sandbox providers must update read_file()/write_file() override signatures; explicit-user calls are rejected until supported. In particular, human_cli(user=...) cannot collect terminal recordings on older providers, including Kubernetes: collection warns and omits those logs. Set record_session=False until support is added, or use the sandbox's default identity if appropriate. This is deliberate security tightening: dropping an explicit user could read a protected file through a sandbox-controlled log symlink with the reader's greater privileges.
- recomputing, editing or recovering a log no longer imports files or builds models it names (#6007)
Features
- openai: support GPT-6.1 Terra (#6100) (f7f3681)
- sentinel: a live sentinel follows the span a bridged agent's event sink gives each generate (#6004) (1692fd8)
- sentinel: rebuild a sample's sentinel steps from its log, with one step builder for live and replay (#5991) (5d8cc44)
- sentinel: the log records where each model call was made, and generate steps keep their ids in replay (#5990) (397d6b8)
- Terminals tab shows all of a sample's terminals together on one shared clock (#6149) (89d8bba)
Bug Fixes
- caches set never to expire no longer read back from logs as expiring after a week (#6088) (05fc2c6)
- enforce user permissions for sandbox file reads and writes (#6107) (c136d29)
- eval-set cache overrides set never to expire no longer run with a one-week expiry (#6097) (eaeeab4)
- prevent agent commands from bypassing transcript recording and output checks (#6148) (ba0748f)
- read_eval_log_samples() missing samples added at runtime by a SampleSource (#6076) (71e00d2)
- recomputing, editing or recovering a log no longer imports files or builds models it names (#6007) ([
- 0.3.285v0.3.2852026-10-25 · 2026-10-25 14:27 UTC
查看更新说明
- 0.3.284v0.3.2842026-10-25 · 2026-10-25 01:19 UTC
查看更新说明
0.3.284 (2026-10-25)
Features
- moonshot: support Kimi K3.1 Pro and K3.1 Pro Highspeed (#6072) (eade928)
- sentinel: step.agent_messages is the running agent's own message list at every stage (#5986) (4744d5f)
- update fireworks and together model info (d6bbd08)
- viewer plays asciicasts stored in the sample store under asciinema:* keys (#6069) (473fac0)
Bug Fixes
- collect() drops the cause of a task's error (#6000) (2521656)
- computer: reject out-of-bounds click coordinates instead of clicking at the screen edge (#5927) (1529d0d)
- concurrent download() calls for the same file no longer fail (#5825) (db24cde)
- content_only no longer modifies the messages it filters (#6024) (ef3bc31)
- dataclass tool parameters losing their types under
from __future__ import annotations(#5746) (7c2dccc) - duplicated task prompt when using CompactionAuto with native compaction (#4804) (1de26ac)
- fail samples on unexpected background subagent errors (#5962) (13cfcad)
- floats in .json log headers saved as strings after tag edits or streaming convert and score (#6034) (b6f7af7)
- keep timezone-less dataframe timestamps on the UTC clock (#5767) (d7c8e2a)
- legacy bridge filters with postponed annotations (#5734) (35565c3)
- local_path no longer drops the Windows drive from file:// URIs (#6035) (901bc92)
- logged model inputs and outputs no longer change when an agent later edits a message in place (#5883) ([ae59272](https://github.com/UKGovernmentBEIS/inspect_ai/commit/ae59272ed3d6649
用户点评
登录用户的真实使用记录，可按版本和硬件筛选共 0 条
还没有人点评。第一个说出真实体验的人最有价值。
Welcome to Inspect, a framework for large language model evaluations created by the UK AI Security Institute.
Inspect provides many built-in components, including facilities for prompt engineering, tool usage, multi-turn dialog, and model graded evaluations. Extensions to Inspect (e.g. to support new elicitation and scoring techniques) can be provided by other Python packages.
To get started with Inspect, please see the documentation at https://inspect.aisi.org.uk/.
Inspect also includes a collection of over 200 pre-built evaluations ready to run on any model (learn more at https://inspect.aisi.org.uk/evals/).
Coding agents: a structured index of the docs is published at https://inspect.aisi.org.uk/llms.txt. The user guide is concatenated as Markdown at https://inspect.aisi.org.uk/llms-guide.txt, and https://inspect.aisi.org.uk/llms-full.txt additionally bundles the API and CLI reference. Individual pages are available as Markdown by appending
.mdto the.htmlpath (e.g./extensions/index.html.md).
To work on development of Inspect, install uv (0.11.8 or later), clone the repository, and sync the development environment from the checked-in lockfile:
git clone https://github.com/UKGovernmentBEIS/inspect_ai.git
cd inspect_ai
uv sync
This installs Inspect in editable mode with its development dependencies at the versions recorded in uv.lock. Project dependencies are declared in pyproject.toml (dependencies and the optional-dependencies extras). When changing dependencies, edit them there and run uv lock. uv only resolves releases that are at least seven days old, except for model provider SDKs and Inspect's own packages; see CONTRIBUTING.md for how to adopt a newer release.
Optionally install pre-commit hooks via
make hooks
Run linting, formatting, and tests via
make check
make test
Prefix those commands with uv run (for example, uv run make check), or activate the environment in .venv first.
If you use VS Code, you should be sure to have installed the recommended extensions (Python, Ruff, and MyPy). Note that you'll be prompted to install these when you open the project in VS Code.
Frontend development (TypeScript)
The web UI lives in a git submodule at src/inspect_ai/_view/ts-mono/. These steps are only needed if you plan to work on the TypeScript/React frontend — Python-only contributors can skip this entirely.
Initialize the submodule and install dependencies — see the one-time setup guide.
Documentation
To work on the Inspect documentation, sync the optional doc dependencies and build the docs:
uv sync --extra doc
cd docs
uv run quarto render # or 'uv run quarto preview'
If you intend to work on the docs iteratively, you'll want to install the Quarto extension in VS Code.