各语言包名与仓库名不一致,Python 包名是 liter-llm、npm 包是 @xberg-io/liter-llm,装错会找不到;代理需自备 liter-llm-proxy.toml 配置并设置 LITER_LLM_MASTER_KEY;调用云端提供商仍需各自的 API Key,云厂商鉴权由库自动刷新。
liter-llm
xberg-ioMIT活跃维护统一调用 170+ 家 LLM 提供商的单一 API 客户端,Rust 内核并提供各主流语言原生绑定,省去为每家模型单独对接的麻烦。
One API for every LLM: 170+ providers through one client, with a Rust core and native bindings for every major language.
pip install liter-llm
brew install xberg-io/tap/liter-llm
liter-llm api --config liter-llm-proxy.tomlREADME 未说明显存或 GPU 要求;客户端为 35 MB 单二进制,无解释器依赖,本地模型可经 Ollama 原生 API 或 OpenAI 兼容接口(LM Studio、vLLM、llama.cpp 等)接入。
一个客户端统一调用 174 家 LLM 提供商,Rust 内核加 14 种语言原生绑定,换模型只改名字
统一调用 170+ 家 LLM 提供商的单一 API 客户端,Rust 内核并提供各主流语言原生绑定,省去为每家模型单独对接的麻烦。
核心能力
- 用统一 API 接入 174 家提供商,切换模型只改模型名
- Rust 内核编译出 14 种语言绑定,跨语言行为一致
- 内置 OpenAI 兼容代理与 MCP 工具服务器,单二进制 35 MB
- Tower 中间件组合限流、缓存、成本追踪与自动回退
- 支持 SSE 与 AWS EventStream 流式,含并行工具调用与结构化输出
容易踩的坑
- v2.7.0最新正式版2026-11-06 · 2026-11-06 20:01 UTC
Fixed
- Anthropic:
reasoning_efforton Claude Opus 4.6+, Sonnet 4.6+, Haiku 5.5, Fable and Mythos now sends adaptive thinking withoutput_config.effort(minimalbecomeslow, andxhighbecomesmaxon Opus and Sonnet 4.6), instead of athinking.budget_tokensthat Claude Opus 5.5, Sonnet 5.5 and Haiku 5.5 reject. When the request sets nomax_tokens, it is raised to at least 16,000 (64,000 atxhighandmax), since adaptive thinking spends it on thinking and answer together. Older models keep the token budget, withxhighbetweenhighandmax, and only they still send thethinking-2025-04-14beta header.noneon Claude Sonnet 5.5 sendsthinking: {type: "between_tools"}, its documented way to turn thinking off (#473). - Anthropic: thinking blocks survive a tool loop. A reply keeps each
thinkingblock, with itssignature, and eachredacted_thinkingblock in the newAssistantMessage.thinking_blocks(ThinkingBlock, with the block's position among the turn's text and tool calls), and sending the message back returns them to Anthropic unchanged at that position, as Anthropic requires alongside tool results. A block without itssignature(ordata) is rejected withBadRequestnaming the field. Streams carry them asStreamDelta.thinking_blocksfragments keyed by block index, including the signature that was dropped before; in Rust,ThinkingBlockAccumulatormerges a stream's fragments into the blocks a reply without streaming returns. Other providers never receive the field. Rust code that buildsAssistantMessage,StreamDeltaorThinkingBlockwith a struct literal and without..Default::default()must add the new fields (#474).
Swift Package Manager
The Swift binding is distributed as a pre-built artifact bundle. No Rust toolchain required.
Recommended — depend on the package by version:
.package(url: "https://github.com/xberg-io/liter-llm", from: "2.7.0")The
v2.7.0tag'sPackage.swiftpins the artifact bundle checksum. Therelease/swift/2.7.0branch points at the same commit for manifests that use.package(url: "https://github.com/xberg-io/liter-llm", branch: "release/swift/2.7.0").Alternative — use
.binaryTargetdirectly in yourPackage.swift:.binaryTarget( name: "LiterLlm", url: "https://github.com/xberg-io/liter-llm/releases/download/v2.7.0/LiterLlm-rs.artifactbundle.zip", checksum: "6ec5d59b7d415850863c8f7aac889a00c5e6c5017ae12baf5daefc863f54c260" )Artifact bundle: LiterLlm-rs.artifactbundle.zip Checksum:
6ec5d59b7d415850863c8f7aac889a00c5e6c5017ae12baf5daefc863f54c260Zig
Add to your
build.zig.zon:.dependencies = .{ .@"liter-llm-zig" = .{ .url = "https://github.com/xberg-io/liter-llm/releases/download/v2.7.0/liter-llm-zig-v2.7.0.tar.gz", .hash = "liter_llm-2.7.0-Jfgk_PeTvj1iGeD9bxTmAqSnlJWCrGQHGNsKQ4gb0vCj", }, }, - Anthropic:
- v2.6.12026-11-06 · 2026-11-06 12:26 UTC
查看更新说明
Changed
- macOS 15 is the minimum for every macOS artifact (2.6.0 required macOS 26): the FFI libraries, language extensions, CLI, Swift bundle, Python wheels (now tagged
macosx_15_0) and the Swift package (.macOS(.v15)). The floor is the oldest macOS release Apple still supports (#470).
Swift Package Manager
The Swift binding is distributed as a pre-built artifact bundle. No Rust toolchain required.
Recommended — depend on the package by version:
.package(url: "https://github.com/xberg-io/liter-llm", from: "2.6.1")The
v2.6.1tag'sPackage.swiftpins the artifact bundle checksum. Therelease/swift/2.6.1branch points at the same commit for manifests that use.package(url: "https://github.com/xberg-io/liter-llm", branch: "release/swift/2.6.1").Alternative — use
.binaryTargetdirectly in yourPackage.swift:.binaryTarget( name: "LiterLlm", url: "https://github.com/xberg-io/liter-llm/releases/download/v2.6.1/LiterLlm-rs.artifactbundle.zip", checksum: "58515dc274bb7d0d954462149bdba36a1c4b1c978623295d0573e88be32be4f1" )Artifact bundle: LiterLlm-rs.artifactbundle.zip Checksum:
58515dc274bb7d0d954462149bdba36a1c4b1c978623295d0573e88be32be4f1Zig
Add to your
build.zig.zon:.dependencies = .{ .@"liter-llm-zig" = .{ .url = "https://github.com/xberg-io/liter-llm/releases/download/v2.6.1/liter-llm-zig-v2.6.1.tar.gz", .hash = "liter_llm-2.6.1-Jfgk_JivgD1z0Fwcsfq0QQXDf9cY2GXCW1IhniM28bYb", }, }, - macOS 15 is the minimum for every macOS artifact (2.6.0 required macOS 26): the FFI libraries, language extensions, CLI, Swift bundle, Python wheels (now tagged
- v2.6.02026-11-05 · 2026-11-05 15:10 UTC
查看更新说明
Changed
- Breaking for macOS 25 and earlier: every macOS binary the release ships (FFI libraries, language extensions, CLI, Swift bundle and Python wheels) and the Swift package now require macOS 26, instead of macOS 11 or 13 for the binaries and macOS 14 for the 2.5.0
Package.swift. The macOS Python wheels are taggedmacosx_26_0(#470). - Regenerate the bindings with Alef 0.124.2. The published Python package now declares
requires-python = ">=3.11"and buildsabi3-py311wheels, matching the Python 3.11 minimum. Python option enums subclassStrEnum.
Fixed
- Swift: the core
vX.Y.Ztag'sPackage.swiftnow pins the artifact bundle checksum, sofrom:andexact:consumers resolve a working manifest. A release is now two steps: dispatch Prepare Swift release on main, which calls the shared xberg-io/actions workflow to build the bundle once, push the checksum pin with the tag and attach the exact ZIP and receipt to a draft release; then dispatch Publish Release on the tag, which verifies that state, never rebuilds the bundle and createsrelease/swift/<version>at the tagged commit (procedure). Publish Release verifies the tag and the attached Swift bundle before any registry publishes, rechecks them before it publishes the draft, and rejectsrepublishandforce_republishoutside dry runs. Theswift-<version>tags are no longer created (#460).
Swift Package Manager
The Swift binding is distributed as a pre-built artifact bundle. No Rust toolchain required.
Recommended — depend on the package by version:
.package(url: "https://github.com/xberg-io/liter-llm", from: "2.6.0")The
v2.6.0tag'sPackage.swiftpins the artifact bundle checksum. Therelease/swift/2.6.0branch points at the same commit for manifests that use.package(url: "https://github.com/xberg-io/liter-llm", branch: "release/swift/2.6.0").Alternative — use
.binaryTargetdirectly in yourPackage.swift:.binaryTarget( name: "LiterLlm", url: "https://github.com/xberg-io/liter-llm/releases/download/v2.6.0/LiterLlm-rs.artifactbundle.zip", checksum: "2eb3bccc68f9c6f375dec607e6c8b1ced95af033035e180154bde1bab9a20cc2" )Artifact bundle: LiterLlm-rs.artifactbundle.zip Checksum:
2eb3bccc68f9c6f375dec607e6c8b1ced95af033035e180154bde1bab9a20cc2Zig
Add to your
build.zig.zon:.dependencies = .{ .@"liter-llm-zig" = .{ .url = "https://github.com/xberg-io/liter-llm/releases/download/v2.6.0/liter-llm-zig-v2.6.0.tar.gz", .hash = "liter_llm-2.6.0-Jfgk_MAmgD0-KrS-Ht09CyVbv016btOHTbpVIqu-HDJF", }, }, - Breaking for macOS 25 and earlier: every macOS binary the release ships (FFI libraries, language extensions, CLI, Swift bundle and Python wheels) and the Swift package now require macOS 26, instead of macOS 11 or 13 for the binaries and macOS 14 for the 2.5.0
- v2.5.02026-11-04 · 2026-11-04 04:46 UTC
查看更新说明
Added
blocked_networks: a per-client list of CIDR networks the client never connects to. A host name is refused when any address it resolves to is in a listed network, and an IP-literal URL is refused before the request is sent. Set it onClientConfigBuilder,ClientBuilder,ClientOptions(every binding) or the config file. A refused request fails withOutboundForbidden.wasm32refuses a list that is not empty with aBadRequesterror.
Changed
- Minimum supported versions raised: Node.js 22, PHP 8.3, Elixir 1.17, Go 1.27, Android API 24, Dart 3.13, Zig 0.17, macOS 14.
- Python 3.11 is now the minimum supported version; Python 3.10 reached end of life.
Fixed
- Refresh served skill lock digests after regenerating coding-agent plugin artifacts.
- Keep macOS extension linker flags out of iOS checks and XCFramework builds.
- Keep CI Python imports sorted and use the builtin exception group in the Hermes bridge.
- Regenerate Rust E2E suites with compatible Alef 0.115.15 and per-call client trait imports, preserving stream checks without unused bindings under strict warnings.
- Refresh generated error-family interfaces and required collection JSON serializers while preserving the existing Dart bindings.
- Align the generated Python package README with the Python 3.11 minimum requirement.
- Mistral: preserve parallel tool-call control and presence/frequency penalties, including false and zero values (#458).
- Outbound policy: refuse IPv6 forms that carry a blocked IPv4 address, such as IPv4-mapped, IPv4-compatible, SIIT, NAT64, 6to4 and Teredo addresses, and every ISATAP address (#462).
Swift Package Manager
The Swift binding is distributed as a pre-built artifact bundle. No Rust toolchain required.
Recommended — use
.binaryTargetdirectly in yourPackage.swift:.binaryTarget( name: "LiterLlm", url: "https://github.com/xberg-io/liter-llm/releases/download/v2.5.0/LiterLlm-rs.artifactbundle.zip", checksum: "9cdb64fab6994ac331f7734c74bccdbaa5be9f2e443e24fbcad9ea4c410fa7ea" )Alternative — pin via tag-specific Swift package release:
.package(url: "https://github.com/xberg-io/liter-llm", from: "swift-2.5.0")The main repository's
Package.swiftuses placeholder values. Swift consumers pin to a tag-specific release in theswift-namespace (e.g.from: "swift-2.5.0"), which includes the real artifact bundle checksum. Other languages continue to pin tofrom: "2.5.0".Artifact bundle: LiterLlm-rs.artifactbundle.zip Checksum:
9cdb64fab6994ac331f7734c74bccdbaa5be9f2e443e24fbcad9ea4c410fa7eaZig
Add to your
build.zig.zon:.dependencies = .{ .@"liter-llm-zig" = .{ .url = "https://github.com/xberg-io/liter-llm/releases/download/v2.5.0/liter-llm-zig-v2.5.0.tar.gz", .hash = "liter_llm-2.5.0-Jfgk_FgzhD0_SRedRwqWS1DN3W9_-AF9WVgd66tp6yMp", }, }, - v2.4.42026-10-31 · 2026-10-31 22:08 UTC
查看更新说明
Fixed
- Refresh stale coding-agent plugin provenance that blocked OpenCode and Pi package publication.
Changed
- Regenerate bindings with Alef 0.115.10, including private Swift
CodingKeysand Java Maven Enforcer 3.7.0.
Swift Package Manager
The Swift binding is distributed as a pre-built artifact bundle. No Rust toolchain required.
Recommended — use
.binaryTargetdirectly in yourPackage.swift:.binaryTarget( name: "LiterLlm", url: "https://github.com/xberg-io/liter-llm/releases/download/v2.4.4/LiterLlm-rs.artifactbundle.zip", checksum: "7d51ab4b47cffac4a7294ad2953c84d4994f30ebdc19db5ca5e9a3197ee08291" )Alternative — pin via tag-specific Swift package release:
.package(url: "https://github.com/xberg-io/liter-llm", from: "swift-2.4.4")The main repository's
Package.swiftuses placeholder values. Swift consumers pin to a tag-specific release in theswift-namespace (e.g.from: "swift-2.4.4"), which includes the real artifact bundle checksum. Other languages continue to pin tofrom: "2.4.4".Artifact bundle: LiterLlm-rs.artifactbundle.zip Checksum:
7d51ab4b47cffac4a7294ad2953c84d4994f30ebdc19db5ca5e9a3197ee08291Zig
Add to your
build.zig.zon:.dependencies = .{ .@"liter-llm-zig" = .{ .url = "https://github.com/xberg-io/liter-llm/releases/download/v2.4.4/liter-llm-zig-v2.4.4.tar.gz", .hash = "liter_llm-2.4.4-Jfgk_IT_eT1CwwVK0JQ-Kbi56eB4v16NXi01JwbWFYSr", }, },
用户点评
登录用户的真实使用记录,可按版本和硬件筛选还没有人点评。第一个说出真实体验的人最有价值。
liter-llm
One API for every LLM — in your language, without the rewrites.
What and Why?
liter-llm gives you a single, consistent way to call any large language model. Reach 174 providers — OpenAI, Anthropic, Google, Bedrock, and more — through one client, and switch models by changing a name instead of your code. The same client ships to 14 languages, all built on one Rust core, so you get identical behavior everywhere. Need a gateway? Point your existing OpenAI SDK at the built-in, drop-in proxy — no rewrites, no separate service to run.
- Work with any provider without rewrites — 174 providers behind one API; change the model name to switch, your code stays the same.
- Use it from your language — Python, TypeScript, Go, Java, Kotlin, C#, Ruby, PHP, Elixir, Dart, Swift, Zig, and WebAssembly, all built on the same Rust core, plus a C/FFI surface for everything else.
- Secure by default — API keys are wrapped, redacted from logs, and never serialized; cloud auth for Azure, AWS, and Vertex refreshes and rotates for you.
- One small binary — the client, a drop-in OpenAI-compatible proxy, and an MCP server for AI agents ship in a single 35 MB binary with no interpreter and no dependency tree to compromise.
- Stay up when a provider fails — route by cost, latency, or weight and fall back automatically, with built-in rate limiting, caching, and cost tracking.
Deeper details — observability, middleware, and the full endpoint list — live in the documentation.
We credit litellm for proving the category; our provider registry was bootstrapped from theirs. See ATTRIBUTIONS.md.
Features
| Feature | Description |
|---|---|
| 174 providers | OpenAI, Anthropic, Google, AWS Bedrock, Groq, Mistral, Together, Fireworks, DeepSeek, Cohere, and 130+ more — compiled at build time |
| 14 native bindings | Rust, Python, Node.js, Go, Java, Kotlin, C#, Ruby, PHP, Elixir, Dart, Swift, Zig, WebAssembly — plus a shared C/FFI surface |
| First-class streaming | SSE and AWS EventStream binary protocol with zero-copy buffers |
| Proxy & MCP server | Drop-in OpenAI-compatible proxy (22 endpoints) and MCP tool server in a 35 MB Docker image |
| Tower middleware | Rate limiting, caching (40+ OpenDAL backends), cost tracking, budget enforcement, health checks, and fallback — all composable |
| Observability | OpenTelemetry with GenAI semantic conventions, cost-tracking spans, and HTTP-level tracing |
| Tool calling | Parallel tools, structured outputs, and JSON-schema validation |
| Search & OCR | Web search across 12 providers, document OCR across 4 |
| TOML configuration | liter-llm.toml auto-discovery, custom providers, cache backends, and middleware config |
| Local LLM support | Ollama through its native API; LM Studio, vLLM, llama.cpp, LocalAI, and llamafile through OpenAI-compatible APIs |
Quick Start
Language Packages
pip install liter-llm
See Python README for full documentation.
pnpm add @xberg-io/liter-llm
See Node.js README for full documentation.
cargo add liter-llm
See Rust crate for full documentation.
go get github.com/xberg-io/liter-llm/packages/go/v2
See Go README for full documentation.
Available on Maven Central as io.xberg.literllm:liter-llm. See Java README for the dependency snippet and current version.
gem install liter_llm
See Ruby README for full documentation.
composer require xberg-io/liter-llm
See PHP README for full documentation.
dotnet add package LiterLlm
See .NET README for full documentation.
Add {:liter_llm, "~> 1.6"} to your mix.exs dependencies. See Elixir README for full documentation.
dart pub add liter_llm
See Dart README for full documentation.
Add via Swift Package Manager. See Swift README for full documentation.
Available on Maven Central as io.xberg.literllm:liter-llm-android. See Kotlin README for the dependency snippet and current version.
See Zig README for installation and usage.
pnpm add @xberg-io/liter-llm-wasm
See WebAssembly README for full documentation.
Build from source as part of this workspace. See FFI crate for full documentation.
The liter-llm CLI ships both the OpenAI-compatible proxy and the MCP tool server. Install it any of these ways:
brew install xberg-io/tap/liter-llm
cargo install liter-llm-cli # from crates.io
cargo binstall liter-llm-cli # prebuilt binary from GitHub Releases
pnpm dlx @xberg-io/liter-llm-cli --help # npm (self-installs the binary)
docker run -p 4000:4000 -e LITER_LLM_MASTER_KEY=sk-your-key ghcr.io/xberg-io/liter-llm
Windows users can install the same binary through Scoop:
scoop bucket add xberg https://github.com/xberg-io/scoop-bucket
scoop install liter-llm
Then run the proxy or the MCP server:
liter-llm api --config liter-llm-proxy.toml # OpenAI-compatible proxy (22 endpoints)
liter-llm mcp --transport stdio # MCP tool server (stdio)
liter-llm mcp --transport http --port 3001 # MCP tool server (Streamable HTTP)
See the MCP server guide and the proxy guide for transports, routing, virtual keys, and budgets. To use the MCP server inside a coding agent, install the liter-llm plugin (below) — it registers the server for your MCP client. The CLI release compatibility requirements below also apply to auto-installed binaries.
AI Coding Assistants
Install the liter-llm plugin from xberg-io/liter-llm. It ships the liter-llm agent skills (chat, streaming, tools, embeddings across 174 providers) — select your runtime below.
The bundle targets Claude Code, Codex, Cursor, Gemini CLI, Kimi, OpenCode, Factory Droid, Hermes, Pi, Agent Plugins and GitHub Copilot CLI. Read the plugin guide for runtime resources, CLI prerequisites and configuration. MCP-capable adapters expose the product's tools; Hermes uses a local stdio bridge and explicitly selected skills. Pi loads skills and prompt templates and requires the product CLI for executable operations.
Install only the public Agent Skills, without a plugin or MCP registration, using the skills CLI:
npx skills add xberg-io/liter-llm --list
npx skills add xberg-io/liter-llm
Choose the skills and target agents interactively. Installing instructions does not install the product CLI or configure provider credentials.
Published CLI 2.2.3 predates the local-stdio trust and deferred-provider-credential fixes. Until a CLI release includes them, use a source build for startup without an extra auth config:
cargo install --git https://github.com/xberg-io/liter-llm \
--rev f14fb8862f9cc73e2c4ab38a2876efe77fab8bf4 --locked liter-llm-cli
Put the installed binary on the assistant's PATH. If a plugin has already cached an older binary,
remove that plugin's bin/liter-llm (or bin/liter-llm.exe) before retrying so it selects the source build.
The released 2.2.3 binary requires mcp.stdio_trust_local = true or a configured mcp.stdio_key_id;
configured providers may also require credentials at startup. The source build defers missing provider
credentials until a request uses that provider. HTTP authentication remains configured separately.
The source-built stdio MCP server trusts its local host by default and starts without a provider key.
Set provider credentials such as OPENAI_API_KEY or ANTHROPIC_API_KEY before sending
requests to that provider; missing credentials fail the request. HTTP MCP access retains
its configured authentication. Use liter-llm api for the proxy and liter-llm mcp for MCP;
chat and streaming are SDK, proxy or MCP operations rather than CLI subcommands.
/plugin marketplace add xberg-io/liter-llm
/plugin install liter-llm@liter-llm
/plugins add https://github.com/xberg-io/liter-llm
Then search for liter-llm and select Install Plugin.
Settings → Plugins → Add from URL → https://github.com/xberg-io/liter-llm, then select liter-llm.
From a checkout of this repository:
gemini extensions install ./plugin
The manifest is inside plugin/. See the Gemini extension reference
for local installation and updates.
droid plugin marketplace add https://github.com/xberg-io/liter-llm
droid plugin install liter-llm@liter-llm
copilot plugin marketplace add https://github.com/xberg-io/liter-llm
copilot plugin install liter-llm@liter-llm
Install from a local checkout now:
pi install ./plugin
After the next plugin release, install the same package used by OpenCode:
pi install npm:@xberg-io/opencode-liter-llm
Pi loads the bundled skills and prompt templates. Use /liter-llm-chat and /liter-llm-stream
to start a task. Install the CLI separately as
shown in the plugin README.
This package does not register MCP servers, hooks, or executable extensions in Pi.
Add the package to opencode.json:
{
"$schema": "https://opencode.ai/config.json",
"plugin": ["@xberg-io/opencode-liter-llm"]
}
Documentation
Full guides, the unified chat() API for every binding, multimodal I/O, the proxy/gateway, and the complete provider list live at docs.liter-llm.xberg.io.
Contributing
Contributions are welcome! See CONTRIBUTING.md for guidelines.
Join our Discord community for questions and discussion.
Part of Xberg.io
- Xberg — document intelligence: text, tables, metadata from 101 formats with optional OCR.
- Xberg Enterprise — managed extraction API with SDKs, dashboards, and observability.
- crawlberg — web crawling and scraping with HTML→Markdown and headless-Chrome fallback.
- html-to-markdown — fast, lossless HTML→Markdown engine.
- liter-llm — universal LLM API client with native bindings for 14 languages and 174 providers.
- tree-sitter-language-pack — tree-sitter grammars and code-intelligence primitives.
- alef — the polyglot binding generator that produces every per-language binding across the 5 polyglot repos.
License
MIT — see LICENSE for details.