需先安装 bencher CLI 并注册账号获取 API 令牌才能上报结果；CI 共享 runner 噪声大，建议用 Bencher Bare Metal runner 才能获得可信对比；自托管部署需参考单独的 Self-Hosted Quickstart。
README 未给出具体安装命令，仅指向 Install CLI 文档：https://bencher.dev/docs/how-to/install-cli/ 硬件要求按 README 说明
README 未说明具体硬件要求；Bencher Bare Metal 强调在裸金属机器上运行基准以获得低方差，未列出 GPU 或显存规格。
持续基准测试平台，在合并前捕获性能回归，避免噪声 CI 掩盖真实退化
持续基准测试平台，在 CI 中自动追踪代码性能变化，防止性能回归，适合关注代码质量的团队。
核心能力
- 用 bencher CLI 在裸金属上运行并上报基准结果
- 按分支、测试床、基准和度量追踪历史结果并绘图
- 用可定制分析在本地或 CI 中检测性能回归
- 提供 GitHub Action 自动在 PR 中评论基准报告
- 裸金属 runner 将结果方差控制在 2% 以内
容易踩的坑
内容由模型根据 README 生成，可能不完整或不准确，以官方文档为准。看原文 README
- v0.6.16最新正式版2026-10-31 · 2026-10-31 03:52 UTC
- Link a GitHub repository to a Project in its Settings, and a detached Bare Metal run on GitHub Actions no longer needs
--ci-callback-token
- Link a GitHub repository to a Project in its Settings, and a detached Bare Metal run on GitHub Actions no longer needs
- v0.6.152026-10-30 · 2026-10-30 01:40 UTC
查看更新说明
- Add Bencher to the ChatGPT and Codex plugins directory
- List exact matches first, then the most recently used, in the Console search menus for Branches, Testbeds, Benchmarks, Measures, Parameters, and Metrics (Thank you thomaseizinger)
- A project key can now create and update plots with
POST /v0/projects/{project}/plotsandPATCH /v0/projects/{project}/plots/{plot}, and Bencher MCP adds thecreate_plotandupdate_plottools - BREAKING CHANGE Remove the perf image endpoint (
GET /v0/projects/{project}/perf/img) and the MCPperf_imagetool; embed the perf plot from a public project's perf page instead - With
--github-actions, the jobs of a Project on a pull request now share one comment with a numbered Report row per job, and--ci-idlabels the job's Report
- v0.6.142026-10-28 · 2026-10-28 01:44 UTC
查看更新说明
- Add
--start-point-idempotency-key(orBENCHER_START_POINT_IDEMPOTENCY_KEY) tobencher run,bencher report create, andbencher branch update, so parallel jobs that reset a Branch with the same key share one Head; in GitHub Actions,bencher rundefaults the key to the workflow run attempt - Fix concurrent runs on a Branch failing when they create the same Thresholds, such as parallel runs that clone Thresholds from a start point; the first run creates them and the rest use them
- Add
GET /v0/server/health, which checks that the API server runs and reaches its database - BREAKING CHANGE Remove
GET /v0/projects. List an organization's projects withGET /v0/organizations/{organization}/projects, whichbencher project listnow calls with a required organization argument, and the MCPlist_projectstool now takes anorganizationand asks an anonymous caller to sign in - Add
--image-cache-sizetorunner up(orBENCHER_IMAGE_CACHE_SIZE), the bytes of Image config and layer blobs a Bare Metal Runner keeps across Jobs inblobsunder its--state-dir, so a Job downloads only the blobs the Runner does not already hold, while its manifest still comes from the registry. Between Jobs the least recently used blobs are removed until the rest fit, a Runner given a size exits at startup if it cannot make the store its own, and without one the Runner keeps none - A Bare Metal Job whose image cannot be pulled or prepared now fails, even with
--allow-failure, instead of completing with no results - A Bare Metal build time now covers the image's command alone, timed where the command runs, and no longer includes the image pull, unpack, and boot; only an iteration whose command succeeds reports one
- Support Hyperfine 2 JSON output in the
shell_hyperfineadapter (Thank you mrazauskas) - Fix a very large
pagequery parameter returning the wrong page - BREAKING CHANGE Cap perf queries at 8 branches, testbeds, benchmarks, and measures each, and at 64 permutations and 64 lines; extra entries are ignored rather than refused
- Add a
plus_limitserver config option for the daily metrics limit of organizations with a Bencher Plus plan - BREAKING CHANGE Cap a plot at 8 branches, testbeds, benchmarks, and measures each
- Cap active Thresholds at 64 per Branch, Testbed, and Measure, and skip the Thresholds a report cannot create (over a cap or over the daily creation limit) instead of failing the report
- BREAKING CHANGE The
plus.litestreamserver config key is removed; useplus.disaster_recovery. A config that still usesplus.litestreamis ignored and disaster recovery does not run at all: no restore at startup and no replication - BREAKING CHANGE The
sftpdisaster recovery replica scheme is removed;fileands3remain - BREAKING CHANGE The on demand database backup is removed:
POST /v0/server/backup,bencher server backup, and thedatabase.data_storeserver config section - BREAKING CHANGE The container registry storage config moves from
plus.registry.data_store(service: aws_s3, an S3 access point ARN) toplus.registry.storage(scheme: file | s3, a bucket plus optionalendpoint,region, andpath), and any S3-compatible object store now works. Stored keys and the local directory layout are unchanged for a plain prefix, so existing data is picked up as is; apaththat began or ended with a slash is now normalized, and one containing a character the store percent-encodes is now encoded, so see the config reference before upgrading. A config that still carriesdata_storeis ignored and the registry falls back to local files - BREAKING CHANGE A Bare Metal Runner that executes sandboxed Jobs must now run as
root, where before it did not. Confining the Firecracker VMM requires privileges that enabling KVM does not grant:mknodto create the sandbox's/dev/kvm,chownto hand the guest images to the san
- Add
- v0.6.132026-09-28 · 2026-09-28 06:56 UTC
查看更新说明
- Fix slow alert lists and perf queries on large projects
- Alert responses now name the
variantthe alert fired on - BREAKING CHANGE Mark a Bare Metal job
unknowninstead offailedwhen the server loses contact with its runner - Fix rate limit windows counting up to twice their length
- Write API server logs to stderr as JSON
- Add callbacks for detached Bare Metal jobs (Thank you jkaczman)
- BREAKING CHANGE
--detachwith--github-actionsnow requires--ci-callback-token, and no longer posts a comment or completes the GitHub Check before the job has run - BREAKING CHANGE An empty
--github-actionstoken is now refused
- v0.6.122026-08-22 · 2026-08-22 15:39 UTC
查看更新说明
- Support Gungraun JSON format (Thank you gamma0987)
用户点评
登录用户的真实使用记录，可按版本和硬件筛选共 0 条
还没有人点评。第一个说出真实体验的人最有价值。
Bencher is a continuous benchmarking platform that helps software engineers and agents catch performance regressions before they merge. Bencher Bare Metal runs your benchmarks on the exact same bare metal, whether launched from your local terminal, an agent sandbox, or in CI.
Built for teams where performance matters:
- Databases
- Compilers
- Browsers
- Runtimes
- Networking stacks
- Cryptographic libraries
- Operating Systems
Most teams start on GitHub Actions runners with a benchmark comparison script. That approach breaks down when noisy shared CI runners hide real performance regressions.
- Typical CI runners: >30% variance
- Bencher Bare Metal runners: <2% variance
When a number moves, it means something.
The Problem
Shipping a performance regression is expensive:
- A database regression on a hot path pages someone at 2am
- A compiler regression silently makes every downstream build slower
- A browser engine ships a 10% paint regression and users notice
- A crypto library that adds a microsecond to a handshake breaks an SLA
Without trustworthy benchmarks in the PR workflow, you find out when users do.
Local benchmarks aren't reproducible. Every check means stopping work to pull the baseline branch and wait on a comparison. Most engineers skip it.
CI runners are shared and noisy. Noisy benchmarks train engineers to ignore alerts. Performance regressions silently ship.
If you can't tell a real regression from noise, the results are worthless. So teams stop looking.
How Bencher Works
- Run: Run your benchmarks locally or in CI using the exact same bare metal runners and your favorite benchmarking tools. The
bencherCLI orchestrates running your benchmarks on bare metal and stores the results. - Track: Track the results of your benchmarks over time. Monitor, query, and graph the results using the Bencher web console based on the source branch, testbed, benchmark, and measure.
- Catch: Catch performance regressions locally or in CI using the exact same bare metal hardware. Bencher uses state of the art, customizable analytics to detect performance regressions before they merge.
For the same reasons that unit tests are run to prevent feature regressions, benchmarks should be run with Bencher to prevent performance regressions. Performance bugs are bugs!
Example PR Comment
The Bencher Suite
Bencher is a suite of bare metal continuous benchmarking tools:
bencherCLI: run benchmarks and publish results- Bencher API Server: store, query, and alert on results
- Bencher Console: web UI for tracking and graphing
- Bencher Bare Metal
runner: bare metal benchmark execution
The best place to start is the Bare Metal Quickstart.
For on-prem deployments, check out the Bencher Self-Hosted Quickstart.
Documentation
- Tutorial
- How To
- Explanation
- Reference
🌐 Also available in:
Supported Benchmark Harnesses
- {...} JSON
- #️⃣ C#
- ➕ C++
- 🎯 Dart
- 🕳 Go
- ☕️ Java
- 🕸 JavaScript
- 🐍 Python
- ♦️ Ruby
- 🦀 Rust
- libtest bench
- Criterion
- Iai
- Gungraun (formerly Iai-Callgrind)
- ❯_ Shell
👉 For more details see the explanation of benchmark harness adapters.
Don't see your harness? Open an issue →
Showcase
👉 Checkout all public projects.
What Engineers Say
Hosting
- Bencher Cloud: Zero infrastructure to manage. On-demand bare metal runners, billed by the minute. Pay for your benchmark runs, not idle servers. Benchmark for free →
- Bencher Self-Hosted: Deploy Bencher on your own infrastructure. Bare metal, Docker, or Kubernetes. Full control, no data leaving your environment. Deploy in 60 seconds →
GitHub Actions
Install the Bencher CLI using the GitHub Action, and use it for continuous benchmarking in your project.
name: Continuous Benchmarking with Bencher
on:
push:
branches: main
jobs:
benchmark_with_bencher:
name: Benchmark with Bencher
runs-on: ubuntu-latest
env:
BENCHER_PROJECT: my-project-slug
BENCHER_API_TOKEN: ${{ secrets.BENCHER_API_TOKEN }}
steps:
- uses: actions/checkout@v6
- uses: bencherdev/bencher@main
- run: bencher run "bencher mock"
Supported Operating Systems:
- Linux (x86_64 & ARM64)
- macOS (x86_64 & ARM64)
- Windows (x86_64 & ARM64)
👉 For more details see the explanation of how to use GitHub Actions.
Repository Secrets
Add BENCHER_API_TOKEN to you Repository secrets (ex: Repo -> Settings -> Secrets and variables -> Actions -> New repository secret). You can find your API tokens by running bencher token list my-user-slug or view them in the Bencher Console.
Error on Alert
You can set the bencher run CLI subcommand to error
if an Alert is generated with the --error-on-alert flag.
bencher run --error-on-alert "bencher mock"
👉 For more details see the explanation of bencher run.
Comment on PRs
You can set the bencher run CLI subcommand to comment on a PR with the --github-actions argument.
bencher run --github-actions "${{ secrets.GITHUB_TOKEN }}" "bencher mock"
👉 For more details see the explanation of bencher run.
👉 See the example PR comment above.
Specify CLI Version
There is also an optional version argument to specify an exact version of the Bencher CLI to use.
Otherwise, it will default to using the latest CLI version.
- uses: bencherdev/bencher@main
with:
version: 0.6.16
Specify an exact version if using Bencher Self-Hosted. Do not specify an exact version if using Bencher Cloud as there are still occasional breaking changes.
Contributing
To set up a development environment, see the Development Getting Started guide.
🐰 All pull requests should target the devel branch
License
All content that resides under any directory or feature named "plus" is licensed under the Bencher Plus License.
All other content is licensed under the Apache License, Version 2.0 or MIT License at your discretion.
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in Bencher by you, as defined in the Apache-2.0 license, shall be licensed as above, without any additional terms or conditions.