liter-llm

xberg-ioMIT活跃维护

统一调用 170+ 家 LLM 提供商的单一 API 客户端,Rust 内核并提供各主流语言原生绑定,省去为每家模型单独对接的麻烦。

One API for every LLM: 170+ providers through one client, with a Rust core and native bindings for every major language.

v2.7.0发布于 2026-11-06 20:01 UTC265star +1 / 7 天27fork同步于 2026-11-11 00:01 UTC

怎么上手

由 README 提炼,执行前请确认
pip install liter-llm
brew install xberg-io/tap/liter-llm
liter-llm api --config liter-llm-proxy.toml
硬件要求按 README 说明

README 未说明显存或 GPU 要求;客户端为 35 MB 单二进制,无解释器依赖,本地模型可经 Ollama 原生 API 或 OpenAI 兼容接口(LM Studio、vLLM、llama.cpp 等)接入。

有 Docker 镜像
下载:源码 zip·GitHub Releases(安装包)本站不镜像安装包

项目说明

由 README 自动翻译提炼

一个客户端统一调用 174 家 LLM 提供商,Rust 内核加 14 种语言原生绑定,换模型只改名字

统一调用 170+ 家 LLM 提供商的单一 API 客户端,Rust 内核并提供各主流语言原生绑定,省去为每家模型单独对接的麻烦。

核心能力

  • 用统一 API 接入 174 家提供商,切换模型只改模型名
  • Rust 内核编译出 14 种语言绑定,跨语言行为一致
  • 内置 OpenAI 兼容代理与 MCP 工具服务器,单二进制 35 MB
  • Tower 中间件组合限流、缓存、成本追踪与自动回退
  • 支持 SSE 与 AWS EventStream 流式,含并行工具调用与结构化输出

容易踩的坑

各语言包名与仓库名不一致,Python 包名是 liter-llm、npm 包是 @xberg-io/liter-llm,装错会找不到;代理需自备 liter-llm-proxy.toml 配置并设置 LITER_LLM_MASTER_KEY;调用云端提供商仍需各自的 API Key,云厂商鉴权由库自动刷新。

内容由模型根据 README 生成,可能不完整或不准确,以官方文档为准。看原文 README

最近发版

5 个版本v2.7.0
  1. v2.7.0最新正式版2026-11-06 · 2026-11-06 20:01 UTC

    Fixed

    • Anthropic: reasoning_effort on Claude Opus 4.6+, Sonnet 4.6+, Haiku 5.5, Fable and Mythos now sends adaptive thinking with output_config.effort (minimal becomes low, and xhigh becomes max on Opus and Sonnet 4.6), instead of a thinking.budget_tokens that Claude Opus 5.5, Sonnet 5.5 and Haiku 5.5 reject. When the request sets no max_tokens, it is raised to at least 16,000 (64,000 at xhigh and max), since adaptive thinking spends it on thinking and answer together. Older models keep the token budget, with xhigh between high and max, and only they still send the thinking-2025-04-14 beta header. none on Claude Sonnet 5.5 sends thinking: {type: "between_tools"}, its documented way to turn thinking off (#473).
    • Anthropic: thinking blocks survive a tool loop. A reply keeps each thinking block, with its signature, and each redacted_thinking block in the new AssistantMessage.thinking_blocks (ThinkingBlock, with the block's position among the turn's text and tool calls), and sending the message back returns them to Anthropic unchanged at that position, as Anthropic requires alongside tool results. A block without its signature (or data) is rejected with BadRequest naming the field. Streams carry them as StreamDelta.thinking_blocks fragments keyed by block index, including the signature that was dropped before; in Rust, ThinkingBlockAccumulator merges a stream's fragments into the blocks a reply without streaming returns. Other providers never receive the field. Rust code that builds AssistantMessage, StreamDelta or ThinkingBlock with a struct literal and without ..Default::default() must add the new fields (#474).

    Swift Package Manager

    The Swift binding is distributed as a pre-built artifact bundle. No Rust toolchain required.

    Recommended — depend on the package by version:

    .package(url: "https://github.com/xberg-io/liter-llm", from: "2.7.0")
    

    The v2.7.0 tag's Package.swift pins the artifact bundle checksum. The release/swift/2.7.0 branch points at the same commit for manifests that use .package(url: "https://github.com/xberg-io/liter-llm", branch: "release/swift/2.7.0").

    Alternative — use .binaryTarget directly in your Package.swift:

    .binaryTarget(
        name: "LiterLlm",
        url: "https://github.com/xberg-io/liter-llm/releases/download/v2.7.0/LiterLlm-rs.artifactbundle.zip",
        checksum: "6ec5d59b7d415850863c8f7aac889a00c5e6c5017ae12baf5daefc863f54c260"
    )
    

    Artifact bundle: LiterLlm-rs.artifactbundle.zip Checksum: 6ec5d59b7d415850863c8f7aac889a00c5e6c5017ae12baf5daefc863f54c260

    Zig

    Add to your build.zig.zon:

    .dependencies = .{
        .@"liter-llm-zig" = .{
            .url = "https://github.com/xberg-io/liter-llm/releases/download/v2.7.0/liter-llm-zig-v2.7.0.tar.gz",
            .hash = "liter_llm-2.7.0-Jfgk_PeTvj1iGeD9bxTmAqSnlJWCrGQHGNsKQ4gb0vCj",
        },
    },
    
  2. v2.6.12026-11-06 · 2026-11-06 12:26 UTC
    查看更新说明

    Changed

    • macOS 15 is the minimum for every macOS artifact (2.6.0 required macOS 26): the FFI libraries, language extensions, CLI, Swift bundle, Python wheels (now tagged macosx_15_0) and the Swift package (.macOS(.v15)). The floor is the oldest macOS release Apple still supports (#470).

    Swift Package Manager

    The Swift binding is distributed as a pre-built artifact bundle. No Rust toolchain required.

    Recommended — depend on the package by version:

    .package(url: "https://github.com/xberg-io/liter-llm", from: "2.6.1")
    

    The v2.6.1 tag's Package.swift pins the artifact bundle checksum. The release/swift/2.6.1 branch points at the same commit for manifests that use .package(url: "https://github.com/xberg-io/liter-llm", branch: "release/swift/2.6.1").

    Alternative — use .binaryTarget directly in your Package.swift:

    .binaryTarget(
        name: "LiterLlm",
        url: "https://github.com/xberg-io/liter-llm/releases/download/v2.6.1/LiterLlm-rs.artifactbundle.zip",
        checksum: "58515dc274bb7d0d954462149bdba36a1c4b1c978623295d0573e88be32be4f1"
    )
    

    Artifact bundle: LiterLlm-rs.artifactbundle.zip Checksum: 58515dc274bb7d0d954462149bdba36a1c4b1c978623295d0573e88be32be4f1

    Zig

    Add to your build.zig.zon:

    .dependencies = .{
        .@"liter-llm-zig" = .{
            .url = "https://github.com/xberg-io/liter-llm/releases/download/v2.6.1/liter-llm-zig-v2.6.1.tar.gz",
            .hash = "liter_llm-2.6.1-Jfgk_JivgD1z0Fwcsfq0QQXDf9cY2GXCW1IhniM28bYb",
        },
    },
    
  3. v2.6.02026-11-05 · 2026-11-05 15:10 UTC
    查看更新说明

    Changed

    • Breaking for macOS 25 and earlier: every macOS binary the release ships (FFI libraries, language extensions, CLI, Swift bundle and Python wheels) and the Swift package now require macOS 26, instead of macOS 11 or 13 for the binaries and macOS 14 for the 2.5.0 Package.swift. The macOS Python wheels are tagged macosx_26_0 (#470).
    • Regenerate the bindings with Alef 0.124.2. The published Python package now declares requires-python = ">=3.11" and builds abi3-py311 wheels, matching the Python 3.11 minimum. Python option enums subclass StrEnum.

    Fixed

    • Swift: the core vX.Y.Z tag's Package.swift now pins the artifact bundle checksum, so from: and exact: consumers resolve a working manifest. A release is now two steps: dispatch Prepare Swift release on main, which calls the shared xberg-io/actions workflow to build the bundle once, push the checksum pin with the tag and attach the exact ZIP and receipt to a draft release; then dispatch Publish Release on the tag, which verifies that state, never rebuilds the bundle and creates release/swift/<version> at the tagged commit (procedure). Publish Release verifies the tag and the attached Swift bundle before any registry publishes, rechecks them before it publishes the draft, and rejects republish and force_republish outside dry runs. The swift-<version> tags are no longer created (#460).

    Swift Package Manager

    The Swift binding is distributed as a pre-built artifact bundle. No Rust toolchain required.

    Recommended — depend on the package by version:

    .package(url: "https://github.com/xberg-io/liter-llm", from: "2.6.0")
    

    The v2.6.0 tag's Package.swift pins the artifact bundle checksum. The release/swift/2.6.0 branch points at the same commit for manifests that use .package(url: "https://github.com/xberg-io/liter-llm", branch: "release/swift/2.6.0").

    Alternative — use .binaryTarget directly in your Package.swift:

    .binaryTarget(
        name: "LiterLlm",
        url: "https://github.com/xberg-io/liter-llm/releases/download/v2.6.0/LiterLlm-rs.artifactbundle.zip",
        checksum: "2eb3bccc68f9c6f375dec607e6c8b1ced95af033035e180154bde1bab9a20cc2"
    )
    

    Artifact bundle: LiterLlm-rs.artifactbundle.zip Checksum: 2eb3bccc68f9c6f375dec607e6c8b1ced95af033035e180154bde1bab9a20cc2

    Zig

    Add to your build.zig.zon:

    .dependencies = .{
        .@"liter-llm-zig" = .{
            .url = "https://github.com/xberg-io/liter-llm/releases/download/v2.6.0/liter-llm-zig-v2.6.0.tar.gz",
            .hash = "liter_llm-2.6.0-Jfgk_MAmgD0-KrS-Ht09CyVbv016btOHTbpVIqu-HDJF",
        },
    },
    
  4. v2.5.02026-11-04 · 2026-11-04 04:46 UTC
    查看更新说明

    Added

    • blocked_networks: a per-client list of CIDR networks the client never connects to. A host name is refused when any address it resolves to is in a listed network, and an IP-literal URL is refused before the request is sent. Set it on ClientConfigBuilder, ClientBuilder, ClientOptions (every binding) or the config file. A refused request fails with OutboundForbidden. wasm32 refuses a list that is not empty with a BadRequest error.

    Changed

    • Minimum supported versions raised: Node.js 22, PHP 8.3, Elixir 1.17, Go 1.27, Android API 24, Dart 3.13, Zig 0.17, macOS 14.
    • Python 3.11 is now the minimum supported version; Python 3.10 reached end of life.

    Fixed

    • Refresh served skill lock digests after regenerating coding-agent plugin artifacts.
    • Keep macOS extension linker flags out of iOS checks and XCFramework builds.
    • Keep CI Python imports sorted and use the builtin exception group in the Hermes bridge.
    • Regenerate Rust E2E suites with compatible Alef 0.115.15 and per-call client trait imports, preserving stream checks without unused bindings under strict warnings.
    • Refresh generated error-family interfaces and required collection JSON serializers while preserving the existing Dart bindings.
    • Align the generated Python package README with the Python 3.11 minimum requirement.
    • Mistral: preserve parallel tool-call control and presence/frequency penalties, including false and zero values (#458).
    • Outbound policy: refuse IPv6 forms that carry a blocked IPv4 address, such as IPv4-mapped, IPv4-compatible, SIIT, NAT64, 6to4 and Teredo addresses, and every ISATAP address (#462).

    Swift Package Manager

    The Swift binding is distributed as a pre-built artifact bundle. No Rust toolchain required.

    Recommended — use .binaryTarget directly in your Package.swift:

    .binaryTarget(
        name: "LiterLlm",
        url: "https://github.com/xberg-io/liter-llm/releases/download/v2.5.0/LiterLlm-rs.artifactbundle.zip",
        checksum: "9cdb64fab6994ac331f7734c74bccdbaa5be9f2e443e24fbcad9ea4c410fa7ea"
    )
    

    Alternative — pin via tag-specific Swift package release:

    .package(url: "https://github.com/xberg-io/liter-llm", from: "swift-2.5.0")
    

    The main repository's Package.swift uses placeholder values. Swift consumers pin to a tag-specific release in the swift- namespace (e.g. from: "swift-2.5.0"), which includes the real artifact bundle checksum. Other languages continue to pin to from: "2.5.0".

    Artifact bundle: LiterLlm-rs.artifactbundle.zip Checksum: 9cdb64fab6994ac331f7734c74bccdbaa5be9f2e443e24fbcad9ea4c410fa7ea

    Zig

    Add to your build.zig.zon:

    .dependencies = .{
        .@"liter-llm-zig" = .{
            .url = "https://github.com/xberg-io/liter-llm/releases/download/v2.5.0/liter-llm-zig-v2.5.0.tar.gz",
            .hash = "liter_llm-2.5.0-Jfgk_FgzhD0_SRedRwqWS1DN3W9_-AF9WVgd66tp6yMp",
        },
    },
    
  5. v2.4.42026-10-31 · 2026-10-31 22:08 UTC
    查看更新说明

    Fixed

    • Refresh stale coding-agent plugin provenance that blocked OpenCode and Pi package publication.

    Changed

    • Regenerate bindings with Alef 0.115.10, including private Swift CodingKeys and Java Maven Enforcer 3.7.0.

    Swift Package Manager

    The Swift binding is distributed as a pre-built artifact bundle. No Rust toolchain required.

    Recommended — use .binaryTarget directly in your Package.swift:

    .binaryTarget(
        name: "LiterLlm",
        url: "https://github.com/xberg-io/liter-llm/releases/download/v2.4.4/LiterLlm-rs.artifactbundle.zip",
        checksum: "7d51ab4b47cffac4a7294ad2953c84d4994f30ebdc19db5ca5e9a3197ee08291"
    )
    

    Alternative — pin via tag-specific Swift package release:

    .package(url: "https://github.com/xberg-io/liter-llm", from: "swift-2.4.4")
    

    The main repository's Package.swift uses placeholder values. Swift consumers pin to a tag-specific release in the swift- namespace (e.g. from: "swift-2.4.4"), which includes the real artifact bundle checksum. Other languages continue to pin to from: "2.4.4".

    Artifact bundle: LiterLlm-rs.artifactbundle.zip Checksum: 7d51ab4b47cffac4a7294ad2953c84d4994f30ebdc19db5ca5e9a3197ee08291

    Zig

    Add to your build.zig.zon:

    .dependencies = .{
        .@"liter-llm-zig" = .{
            .url = "https://github.com/xberg-io/liter-llm/releases/download/v2.4.4/liter-llm-zig-v2.4.4.tar.gz",
            .hash = "liter_llm-2.4.4-Jfgk_IT_eT1CwwVK0JQ-Kbi56eB4v16NXi01JwbWFYSr",
        },
    },
    

用户点评

登录用户的真实使用记录,可按版本和硬件筛选
共 0 条

    还没有人点评。第一个说出真实体验的人最有价值。

    README 原文

    未翻译

    liter-llm

    One API for every LLM — in your language, without the rewrites.

    What and Why?

    liter-llm gives you a single, consistent way to call any large language model. Reach 174 providers — OpenAI, Anthropic, Google, Bedrock, and more — through one client, and switch models by changing a name instead of your code. The same client ships to 14 languages, all built on one Rust core, so you get identical behavior everywhere. Need a gateway? Point your existing OpenAI SDK at the built-in, drop-in proxy — no rewrites, no separate service to run.

    • Work with any provider without rewrites — 174 providers behind one API; change the model name to switch, your code stays the same.
    • Use it from your language — Python, TypeScript, Go, Java, Kotlin, C#, Ruby, PHP, Elixir, Dart, Swift, Zig, and WebAssembly, all built on the same Rust core, plus a C/FFI surface for everything else.
    • Secure by default — API keys are wrapped, redacted from logs, and never serialized; cloud auth for Azure, AWS, and Vertex refreshes and rotates for you.
    • One small binary — the client, a drop-in OpenAI-compatible proxy, and an MCP server for AI agents ship in a single 35 MB binary with no interpreter and no dependency tree to compromise.
    • Stay up when a provider fails — route by cost, latency, or weight and fall back automatically, with built-in rate limiting, caching, and cost tracking.

    Deeper details — observability, middleware, and the full endpoint list — live in the documentation.

    We credit litellm for proving the category; our provider registry was bootstrapped from theirs. See ATTRIBUTIONS.md.

    Features

    FeatureDescription
    174 providersOpenAI, Anthropic, Google, AWS Bedrock, Groq, Mistral, Together, Fireworks, DeepSeek, Cohere, and 130+ more — compiled at build time
    14 native bindingsRust, Python, Node.js, Go, Java, Kotlin, C#, Ruby, PHP, Elixir, Dart, Swift, Zig, WebAssembly — plus a shared C/FFI surface
    First-class streamingSSE and AWS EventStream binary protocol with zero-copy buffers
    Proxy & MCP serverDrop-in OpenAI-compatible proxy (22 endpoints) and MCP tool server in a 35 MB Docker image
    Tower middlewareRate limiting, caching (40+ OpenDAL backends), cost tracking, budget enforcement, health checks, and fallback — all composable
    ObservabilityOpenTelemetry with GenAI semantic conventions, cost-tracking spans, and HTTP-level tracing
    Tool callingParallel tools, structured outputs, and JSON-schema validation
    Search & OCRWeb search across 12 providers, document OCR across 4
    TOML configurationliter-llm.toml auto-discovery, custom providers, cache backends, and middleware config
    Local LLM supportOllama through its native API; LM Studio, vLLM, llama.cpp, LocalAI, and llamafile through OpenAI-compatible APIs

    Quick Start

    Language Packages

    pip install liter-llm
    

    See Python README for full documentation.

    pnpm add @xberg-io/liter-llm
    

    See Node.js README for full documentation.

    cargo add liter-llm
    

    See Rust crate for full documentation.

    go get github.com/xberg-io/liter-llm/packages/go/v2
    

    See Go README for full documentation.

    Available on Maven Central as io.xberg.literllm:liter-llm. See Java README for the dependency snippet and current version.

    gem install liter_llm
    

    See Ruby README for full documentation.

    composer require xberg-io/liter-llm
    

    See PHP README for full documentation.

    dotnet add package LiterLlm
    

    See .NET README for full documentation.

    Add {:liter_llm, "~> 1.6"} to your mix.exs dependencies. See Elixir README for full documentation.

    dart pub add liter_llm
    

    See Dart README for full documentation.

    Add via Swift Package Manager. See Swift README for full documentation.

    Available on Maven Central as io.xberg.literllm:liter-llm-android. See Kotlin README for the dependency snippet and current version.

    See Zig README for installation and usage.

    pnpm add @xberg-io/liter-llm-wasm
    

    See WebAssembly README for full documentation.

    Build from source as part of this workspace. See FFI crate for full documentation.

    The liter-llm CLI ships both the OpenAI-compatible proxy and the MCP tool server. Install it any of these ways:

    brew install xberg-io/tap/liter-llm
    cargo install liter-llm-cli                 # from crates.io
    cargo binstall liter-llm-cli               # prebuilt binary from GitHub Releases
    pnpm dlx @xberg-io/liter-llm-cli --help         # npm (self-installs the binary)
    docker run -p 4000:4000 -e LITER_LLM_MASTER_KEY=sk-your-key ghcr.io/xberg-io/liter-llm
    

    Windows users can install the same binary through Scoop:

    scoop bucket add xberg https://github.com/xberg-io/scoop-bucket
    scoop install liter-llm
    

    Then run the proxy or the MCP server:

    liter-llm api --config liter-llm-proxy.toml   # OpenAI-compatible proxy (22 endpoints)
    liter-llm mcp --transport stdio               # MCP tool server (stdio)
    liter-llm mcp --transport http --port 3001    # MCP tool server (Streamable HTTP)
    

    See the MCP server guide and the proxy guide for transports, routing, virtual keys, and budgets. To use the MCP server inside a coding agent, install the liter-llm plugin (below) — it registers the server for your MCP client. The CLI release compatibility requirements below also apply to auto-installed binaries.

    AI Coding Assistants

    Install the liter-llm plugin from xberg-io/liter-llm. It ships the liter-llm agent skills (chat, streaming, tools, embeddings across 174 providers) — select your runtime below.

    The bundle targets Claude Code, Codex, Cursor, Gemini CLI, Kimi, OpenCode, Factory Droid, Hermes, Pi, Agent Plugins and GitHub Copilot CLI. Read the plugin guide for runtime resources, CLI prerequisites and configuration. MCP-capable adapters expose the product's tools; Hermes uses a local stdio bridge and explicitly selected skills. Pi loads skills and prompt templates and requires the product CLI for executable operations.

    Install only the public Agent Skills, without a plugin or MCP registration, using the skills CLI:

    npx skills add xberg-io/liter-llm --list
    npx skills add xberg-io/liter-llm
    

    Choose the skills and target agents interactively. Installing instructions does not install the product CLI or configure provider credentials.

    Published CLI 2.2.3 predates the local-stdio trust and deferred-provider-credential fixes. Until a CLI release includes them, use a source build for startup without an extra auth config:

    cargo install --git https://github.com/xberg-io/liter-llm \
      --rev f14fb8862f9cc73e2c4ab38a2876efe77fab8bf4 --locked liter-llm-cli
    

    Put the installed binary on the assistant's PATH. If a plugin has already cached an older binary, remove that plugin's bin/liter-llm (or bin/liter-llm.exe) before retrying so it selects the source build. The released 2.2.3 binary requires mcp.stdio_trust_local = true or a configured mcp.stdio_key_id; configured providers may also require credentials at startup. The source build defers missing provider credentials until a request uses that provider. HTTP authentication remains configured separately.

    The source-built stdio MCP server trusts its local host by default and starts without a provider key. Set provider credentials such as OPENAI_API_KEY or ANTHROPIC_API_KEY before sending requests to that provider; missing credentials fail the request. HTTP MCP access retains its configured authentication. Use liter-llm api for the proxy and liter-llm mcp for MCP; chat and streaming are SDK, proxy or MCP operations rather than CLI subcommands.

    /plugin marketplace add xberg-io/liter-llm
    /plugin install liter-llm@liter-llm
    
    /plugins add https://github.com/xberg-io/liter-llm
    

    Then search for liter-llm and select Install Plugin.

    Settings → Plugins → Add from URL → https://github.com/xberg-io/liter-llm, then select liter-llm.

    From a checkout of this repository:

    gemini extensions install ./plugin
    

    The manifest is inside plugin/. See the Gemini extension reference for local installation and updates.

    droid plugin marketplace add https://github.com/xberg-io/liter-llm
    droid plugin install liter-llm@liter-llm
    
    copilot plugin marketplace add https://github.com/xberg-io/liter-llm
    copilot plugin install liter-llm@liter-llm
    

    Install from a local checkout now:

    pi install ./plugin
    

    After the next plugin release, install the same package used by OpenCode:

    pi install npm:@xberg-io/opencode-liter-llm
    

    Pi loads the bundled skills and prompt templates. Use /liter-llm-chat and /liter-llm-stream to start a task. Install the CLI separately as shown in the plugin README. This package does not register MCP servers, hooks, or executable extensions in Pi.

    Add the package to opencode.json:

    {
      "$schema": "https://opencode.ai/config.json",
      "plugin": ["@xberg-io/opencode-liter-llm"]
    }
    

    Documentation

    Full guides, the unified chat() API for every binding, multimodal I/O, the proxy/gateway, and the complete provider list live at docs.liter-llm.xberg.io.

    Contributing

    Contributions are welcome! See CONTRIBUTING.md for guidelines.

    Join our Discord community for questions and discussion.

    Part of Xberg.io

    • Xberg — document intelligence: text, tables, metadata from 101 formats with optional OCR.
    • Xberg Enterprise — managed extraction API with SDKs, dashboards, and observability.
    • crawlberg — web crawling and scraping with HTML→Markdown and headless-Chrome fallback.
    • html-to-markdown — fast, lossless HTML→Markdown engine.
    • liter-llm — universal LLM API client with native bindings for 14 languages and 174 providers.
    • tree-sitter-language-pack — tree-sitter grammars and code-intelligence primitives.
    • alef — the polyglot binding generator that produces every per-language binding across the 5 polyglot repos.

    License

    MIT — see LICENSE for details.