aisix

true v1.8.0

🚀 Get started

New to AISIX? Get the gateway running and route your first LLM request in minutes:

• 📖 Documentation — https://docs.api7.ai/ai-gateway/
• ⚡ Gateway quickstart (one container, declarative resources.yaml) — https://docs.api7.ai/ai-gateway/getting-started/gateway-quickstart

📦 Download

Pull the container image from the GitHub Container Registry:

docker pull ghcr.io/api7/aisix:1.8.0

The rolling latest tag and the docker.io/api7/aisix mirror for private/offline deployments are listed on the package page.

AISIX 1.8.0 lets MCP servers act on the caller's behalf, accepts PROXY protocol headers from trusted load balancers, and moves budget decisions onto the gateway itself, so a control-plane outage no longer turns into a blanket refusal. An idle gateway now needs a fraction of the memory it used to.

• MCP servers can act on the caller's behalf (#1361, #1362): relay the caller's own token upstream, or exchange it for a downstream token through the Microsoft Entra ID On-Behalf-Of grant. A new x-aisix-api-key header carries the gateway key so Authorization stays free for the caller's own token.
• PROXY protocol v1 and v2 from trusted load balancers (#1374), on every proxy listener, detected automatically.
• Budgets are decided on the gateway (#1366) from budget state the control plane pushes with the configuration. A control-plane or database outage no longer refuses traffic.
• Every proxy route emits every per-request metric family that applies to it (#1367), and access logs carry token counts on every endpoint (#1365).
• Unknown startup-config keys are ignored with a warning instead of stopping the gateway (#1368).
• Idle memory drops from about 120 MiB to under 10 MiB (#1370).

Control-plane, console and AISIX Enterprise / AISIX Cloud changes in this release (two-factor sign-in, password policy, control-plane API and Helm aisix-cp changes) are described in the AISIX release notes.

⚠️ Breaking changes

See Upgrade notes for what to update.

• Debug response headers on /v1/chat/completions are off by default (#1364). x-aisix-call-id, x-aisix-served-by, x-aisix-route, x-aisix-cache, x-aisix-cache-layer and x-aisix-cache-similarity are no longer sent unless proxy.expose_debug_headers: true is set in the gateway config (Helm chart aisix: config.proxy.expose_debug_headers). The same facts are in usage events, access logs, metrics and traces. x-aisix-request-id, server, x-ratelimit-* and retry-after are still sent.
• Four access-log fields take their usage-event names (#1365): status → status_code, error_kind → error_class, model → requested_model, latency_ms → downstream_latency_ms. Only the new names are written.
• aisix_llm_spend_micro_usd_total is removed (#1367). Only Realtime ever put a value on it. Read spend from cost_usd on usage events, or the control plane's recorded cost. A config that still lists it under observability.metrics.labels boots and warns.
• One inbound credential rule on every entry (#1383). An empty x-aisix-api-key, Authorization or x-api-key header counts as an offered credential and is a 401; it is no longer skipped, so an MCP client using anonymous access that sends one of them empty is now refused instead of served anonymously. On gateway_key passthrough routes, an Authorization that is present but not a usable Bearer credential no longer falls back to x-api-key; it gets 401.
• A connection whose first bytes are PROXY is read as a PROXY v1 header (#1374) and closed unless the peer is in proxy.real_ip.trusted_proxies. It used to get an HTTP 400.
• Dual-stack ([::]) listeners report and match IPv4 clients as plain IPv4 (#1374) instead of ::ffff:a.b.c.d, in allowlists, usage events, OTLP `client.a

🚀 Get started

New to AISIX? Get the gateway running and route your first LLM request in minutes:

• 📖 Documentation — https://docs.api7.ai/ai-gateway/
• ⚡ Gateway quickstart (one container, declarative resources.yaml) — https://docs.api7.ai/ai-gateway/getting-started/gateway-quickstart

📦 Download

Pull the container image from the GitHub Container Registry:

docker pull ghcr.io/api7/aisix:1.7.1

The rolling latest tag and the docker.io/api7/aisix mirror for private/offline deployments are listed on the package page.

AISIX 1.7.1 is a patch release. The gateway now handles every request field by what the target platform documents: a parameter the target can carry is sent as written and the upstream's own error comes back, a parameter it cannot carry is dropped, and a value that is illegal in the caller's own protocol is a 400 naming the field. Answers come back in full, including every choice, logprobs, refusal and annotations.

• Request parameters follow the target platform's contract (#1352, #1353): Bedrock, Vertex Gemini and Claude receive the fields they document, in the slot they document, and the upstream's rejection reaches the caller.
• Answers come back in full (#1353, #1354, #1356): every choice, logprobs, refusal, annotations and legacy function_call, and answers whose content is an array of parts work end to end.
• Legacy function calling works on every provider (#1353), including Claude, Vertex Gemini and Amazon Bedrock.
• Streamed chat usage follows stream_options.include_usage on every upstream (#1351).
• Exporters accept plaintext http:// endpoints on any host (#1346).
• Vertex AI global, us and eu locations reach Google's endpoints, and transcription usage and latency are recorded more completely (#1357, #1359, #1360).

⚠️ Behavior changes

See Upgrade notes for what to update.

• Streamed /v1/chat/completions usage follows stream_options.include_usage on every upstream (#1351). With include_usage: true, usage is sent exactly once, on its own chunk (choices: []) after every other chunk. Without it, no chunk carries usage, as with OpenAI. Before, Claude (any platform), Vertex Gemini and OpenAI-compatible upstreams that report usage on the finish chunk showed it there even without include_usage. Usage events and TPM limits are unchanged. Clients that read usage from a stream must request include_usage.
• Parameters the gateway used to drop now reach the upstream (#1352, #1353). Fields the target defines go out as the caller wrote them, and the upstream's rejection reaches the caller with its status and message; the gateway no longer drops, rewrites or pre-validates them. On Amazon Bedrock this means stop to Grok or GPT 5.6 and later, temperature / top_p / logprobs to a reasoning GPT, the penalties and n, and a tool's strict for a model that does not take it can now get AWS's 400; on Vertex Gemini 2.5, so can a /v1/messages thinking.budget_tokens above the model's ceiling. Details are under Providers and endpoints.
• Values that are illegal in the caller's own protocol answer 400 naming the field (#1351, #1353). This covers a body that is valid JSON but does not fit the request (for example max_tokens: invalid value: integer -1, expected u32 or missing field messages, instead of invalid JSON request body), a message content that is a number, boolean or object, and, on the Claude, Converse and Vertex Gemini bridges, a wrong-typed stop, max_tokens, max_completion_tokens, temperature, top_p, tool_choice, reasoning_effort or service_tier, a malformed tool, or a replayed tool call without id or function.name. These used to be dropped silently, forwarded raw or read as empty. On Bedrock Converse, a stop that is not a non-empty string or string array, and a max_completion_tokens that is not

🚀 Get started

New to AISIX? Get the gateway running and route your first LLM request in minutes:

• 📖 Documentation — https://docs.api7.ai/ai-gateway/
• ⚡ Gateway quickstart (one container, declarative resources.yaml) — https://docs.api7.ai/ai-gateway/getting-started/gateway-quickstart

📦 Download

Pull the container image from the GitHub Container Registry:

docker pull ghcr.io/api7/aisix:1.7.0

The rolling latest tag and the docker.io/api7/aisix mirror for private/offline deployments are listed on the package page.

AISIX 1.7.0 serves Claude on Amazon Bedrock and Vertex AI natively, stops dropping media on native-protocol upstreams, and brings exported telemetry onto the released OpenTelemetry GenAI conventions. /v1/messages and /v1/messages/count_tokens go to each platform's own Anthropic Messages route, and prompt caching, including auto_prompt_caching, now works there. Images, audio and files in chat requests reach Vertex Gemini, Anthropic and Bedrock instead of being dropped. Bedrock gains API-key, default-credential-chain and assumed-role authentication. API keys, teams and members can carry cost-attribution tags that flow to every usage event and can drive rate limits. Two decision endpoints, /v1/systemone and /v1/decisions, are new.

• Claude on Bedrock and Vertex AI, natively (#1318, #1316, #1323, #1325): signed thinking, native stop reasons and cache counters on /v1/messages, a working count_tokens, and prompt caching on every entry point.
• Media reaches native-protocol upstreams (#1314, #1327): image, audio and file parts are translated for Vertex Gemini, Anthropic, Claude on Vertex AI and Bedrock, and URLs the upstream cannot read are downloaded by the gateway behind one SSRF setting.
• Bedrock, broadened (#1319, #1322, #1328): API-key, default-chain and AssumeRole authentication, GPT 5.6+ and Grok on Bedrock's Chat Completions endpoint, Converse requests that follow what the caller sent, and the proxy environment honoured.
• Telemetry on the GenAI semantic conventions (#1288, #1299, #1300): OTLP spans follow v1.41.1, and the OTLP, Datadog and SLS exporters carry structured messages with reasoning and tool calls. Each exporter chooses its own headers to record.
• Cost-attribution tags (#1301) on API keys, teams and members, stamped on every usage event and usable as tag:<key> rate-limit dimensions.
• Decision endpoints (#1293, #1294): /v1/systemone and /v1/decisions, with conversion between the two formats.

⚠️ Breaking changes

• OTLP spans follow the OpenTelemetry GenAI semantic conventions v1.41.1 (#1288, #1299). Saved queries and dashboards keyed on the old names must be updated:
  • gen_ai.system is removed; use gen_ai.provider.name (a hosting platform wins, so Claude on Bedrock is aws.bedrock).
  • gen_ai.request.model is now the model name sent upstream. The model or Model Group name the caller sent moves to aisix.requested_model.
  • gen_ai.usage.input_tokens now includes cached tokens for Anthropic-shape usage (Anthropic, Bedrock Converse, and passthrough routes reporting cache_*_input_tokens). OpenAI-shape usage is unchanged. The Datadog exporter's gen_ai.usage.input_tokens changes the same way; its aisix.* counters do not.
  • Span names change: the operation span is {operation} {upstream model} (for example chat gpt-4o) instead of chat.completions, the SERVER span is POST /v1/chat/completions and the like, and MCP spans are tools/call {tool}.
  • Successful spans leave status unset instead of OK. A 4xx no longer marks the SERVER span as an error, and a caller that hung up (499) is not an error on any span.
  • gen_ai.prompt / gen_ai.completion are removed. With content_mode: full, content is exported as gen_ai.input.messages, gen_ai.output.messages, gen_ai.system_instructions and `gen_ai.tool.definitio

🚀 Get started

New to AISIX? Get the gateway running and route your first LLM request in minutes:

• 📖 Documentation — https://docs.api7.ai/ai-gateway/
• ⚡ Gateway quickstart (one container, declarative resources.yaml) — https://docs.api7.ai/ai-gateway/getting-started/gateway-quickstart

📦 Download

Pull the container image from the GitHub Container Registry:

docker pull ghcr.io/api7/aisix:1.6.0

The rolling latest tag and the docker.io/api7/aisix mirror for private/offline deployments are listed on the package page.

AISIX 1.6.0 lets a semantic router decide with a classification model, puts readable names on metrics, and makes what the gateway records describe what actually happened. A semantic router can pick its route with TypeSafe's jev decision model instead of embedding similarity. Prometheus series can carry API key, team, provider key and policy names beside their ids, and usage events can record request headers you choose. Streamed responses are counted in the request metrics when they end, under the status they ended on, and every refusal the gateway makes itself is named on both the access log and the usage event. Function calling now works end to end on Vertex AI Gemini, reasoning crosses the Anthropic Messages and chat bridges, and passthrough routes enforce their boundaries, their concurrency and their guardrails for the whole stream.

• jev classifier for semantic routers (#1257) — a semantic model can carry a classifier block and route by each route's description instead of examples and thresholds; every semantic router's access-log line now names its decision.
• Readable name labels on metrics (#1268) — opt-in api_key_name, team_name, provider_key_name and policy_name labels beside the id labels.
• Request headers on usage events (#1269) — observability.usage_event.request_headers records the headers you list on every usage event.
• Streamed responses are recorded at stream end with their real status (#1273), and gateway refusals carry their reason on the usage event and the access log (#1275, #1279).
• Tools work on Vertex AI Gemini, and Claude tool calls reach streaming OpenAI-protocol clients (#1277, #1280, #1276); reasoning crosses the Messages/chat bridges (#1267), and Claude models that reject the effort parameter get a thinking budget for reasoning_effort (#1274).
• Passthrough routes hold their boundaries (#1262) — unsafe paths and conflicting queries are refused, redirects are relayed, timeout_ms bounds event streams, concurrency is held for the whole stream, and output guardrails fail closed on a response they cannot evaluate.

⚠️ Breaking changes

• Access-log error_kind and usage-event error_class share one vocabulary (#1273). Non-streamed access-log lines, and terminal usage events of failed requests that carry no attempt record, change these values: upstream_error → upstream_status, transport_error → transport, upstream_decode_error → upstream_decode, upstream_in_band_error → upstream_in_band, config_error → config, invalid_request_error → invalid_config (invalid upstream configuration) and authentication_error → invalid_credentials. Per-attempt events and streamed lines already used the new names. Log and event pipelines that match the old values must accept the new ones. The error.type a client receives is unchanged. A streamed passthrough line now carries an error_kind where it used to be empty.
• A semantic router with on_embedding_failure: fail answers a failed embedding with its own error code (#1274). It still answers 503, but with type: api_error, code: semantic_routing_failed and the message semantic router '<model>' could not pick a route (<reason>), where the reason is embedding_timeout, embedding_upstream or embedding_unavailable, instead of type: provider_unavailable ("no bri