需要先在 app.athina.ai 注册获取 Athina API key；使用 CodeExecutionV2 需额外安装 e2b-code-interpreter；README 未给出具体 pip 包名，安装命令需以官方文档为准
athina-evals
athina-ai长期未更新用 Python 给 LLM 生成的回答跑评测的 SDK，内置多种评测指标，方便在 LLMOps 流程里量化模型输出质量。
Python SDK for running evaluations on LLM generated responses
v1.7.39发布于 2025-05-19 03:30 UTC302star23fork同步于 2026-11-07 07:29 UTC
pip install athina-evals
pip install e2b-code-interpreter 硬件要求按 README 说明
README 未说明
对 LLM 生成结果跑评测的 Python SDK，内置 50+ 预设评测，也可自定义
用 Python 给 LLM 生成的回答跑评测的 SDK，内置多种评测指标，方便在 LLMOps 流程里量化模型输出质量。
核心能力
- 内置 50+ 预设评测，开箱即用
- 支持自定义评测，覆盖自有指标
- 评测结果可同步到 Athina IDE 做实验与数据集对比
- 提供 notebook 快速上手示例
- CodeExecutionV2 通过 e2b 沙箱执行代码
容易踩的坑
内容由模型根据 README 生成，可能不完整或不准确，以官方文档为准。看原文 README
- v1.7.39最新正式版2025-05-19 · 2025-05-19 03:30 UTC
What's Changed
- Minor bump by @vivek-athina in https://github.com/athina-ai/athina-evals/pull/200
Full Changelog: https://github.com/athina-ai/athina-evals/compare/v1.7.38...v1.7.39
- v1.7.382025-04-28 · 2025-04-28 15:29 UTC
查看更新说明
What's Changed
- bumped litellm again by @vivek-athina in https://github.com/athina-ai/athina-evals/pull/199
Full Changelog: https://github.com/athina-ai/athina-evals/compare/v1.7.37...v1.7.38
- v1.7.372025-04-28 · 2025-04-28 14:58 UTC
查看更新说明
What's Changed
- Bumped version of litellm by @vivek-athina in https://github.com/athina-ai/athina-evals/pull/198
Full Changelog: https://github.com/athina-ai/athina-evals/compare/v1.7.36...v1.7.37
- v1.7.362025-04-24 · 2025-04-24 15:54 UTC
查看更新说明
What's Changed
- Litellm version bumped by @vivek-athina in https://github.com/athina-ai/athina-evals/pull/195
- Update pyproject.toml by @vivek-athina in https://github.com/athina-ai/athina-evals/pull/196
Full Changelog: https://github.com/athina-ai/athina-evals/compare/v1.7.34...v1.7.36
What's Changed
- [doximity] add more info for error response type mismatch by @codyathina in https://github.com/athina-ai/athina-evals/pull/197
Full Changelog: https://github.com/athina-ai/athina-evals/compare/v1.7.35...v1.7.36
- v1.7.352025-04-15 · 2025-04-15 15:21 UTC
查看更新说明
What's Changed
- Litellm version bumped by @vivek-athina in https://github.com/athina-ai/athina-evals/pull/195
- Update pyproject.toml by @vivek-athina in https://github.com/athina-ai/athina-evals/pull/196
Full Changelog: https://github.com/athina-ai/athina-evals/compare/v1.7.34...v1.7.35
用户点评
登录用户的真实使用记录，可按版本和硬件筛选共 0 条
还没有人点评。第一个说出真实体验的人最有价值。
Overview
Athina is an Observability and Experimentation platform for AI teams.
This SDK is an open-source repository of 50+ preset evals. You can also use custom evals.
This SDK also serves as a companion to Athina IDE where you can prototype pipelines, run experiments and evaluations, and compare datasets.
Quick Start
Follow this notebook for a quick start guide.
To get an Athina API key, sign up at https://app.athina.ai
Run Evals
These evals can be run programmatically, or via the UI on Athina IDE.
Compare datasets side-by-side (Docs)
Once a dataset is logged to Athina IDE, you can also compare it against another dataset.
Once you run evals using Athina, they will be visible in Athina IDE where you can run experiments, evals, and compare datasets side-by-side.
Preset Evals
Athina Steps
To use CodeExecutionV2, you need to install e2b.
pip install e2b-code-interpreter