gpt-load

GPT-Load

A self-hosted AI gateway for multi-channel, multi-credential setups

API keys, subscription accounts, traffic scheduling, failure handling, request logs, and usage accounting — behind a single entry point.

English · 中文 · 日本語 | Official Website

---

Sponsors

Become a sponsor

Thanks to Modelflare for sponsoring this project! Modelflare: Full strength, stable, nothing watered down. Global SOTA models at a lower cost. Operated by US-based Havenbyte LLC, Modelflare supports GPT, Claude, Gemini, Grok, and other leading models through common compatible protocols, plus 40+ image and video models.

Cache compensation is a highlight: selected OpenAI groups (price-first, stable, and premium) guarantee daily cache hit rates of 65%, 75%, or 85%, and if eligible requests fall short, Modelflare refunds the difference, visible on the usage dashboard after settlement.

Special offers: a single top-up of US$20 unlocks the GPT discount group at a 0.015x price multiplier, and a single top-up of US$50 unlocks the Claude discount group at a 0.03x multiplier. Register via this exclusive link to get started.

---

Why GPT-Load

Your application only needs one base URL and one AccessKey. Providers, accounts, credentials, models, and routing policy are all configured in the management UI.

• One gateway, native protocols — Manage official APIs, cloud platforms, model services, and compatible relays together while clients keep their OpenAI, Anthropic, or Gemini native interfaces.
• One mechanism for API keys and subscriptions — Codex, Claude, Antigravity, Grok, and API-key channels share credential management, scheduling, and health handling.
• Scheduling and failure isolation built in — Multi-credential scheduling, configurable weights, retries, cooldown, blacklisting, and session affinity reduce the impact of overloaded or failing credentials.
• Observable, self-hosted, and simple to deploy — Inspect health, routes, logs, usage, and cost estimates in an embedded UI backed by SQLite, MySQL, or PostgreSQL with local credential encryption.

Online demo

https://demo.gpt-load.com

Login key: gpt-load-demo

Quick start

[!WARNING] If you are using 1.x, read Moving from 1.x first. 2.0 cannot open, import, or migrate 1.x data in place.

1. Start the service

Requires Docker and Docker Compose.

git clone --depth 1 https://github.com/tbphp/gpt-load.git
cd gpt-load

cp .env.example .env
docker compose up -d

Confirm the service is up:

curl --fail http://127.0.0.1:3001/health

The first start generates a management key. Read it and store it safely:

docker compose exec gpt-load sh -c 'cat /app/data/auth.key'

Open http://127.0.0.1:3001 and sign in to the console with that key.

You can also set AUTH_KEY explicitly in .env before starting. By default the service listens on the loopback address only and is not exposed to the internet.

2. Initial configuration

Initial setup takes three steps:

1. Add a channel — Choose an upstream service and add one or more API keys. For subscription channels, complete the OAuth flow or import credentials as prompted.
2. Create a group — Pick a channel, then configure available models and runtime policy.
3. Create an AccessKey — Set the groups and client protocols it may use, then give the generated AccessKey to your application.

The Codex, Claude, and Antigravity OAuth clients use fixed callback ports. Compose publishes them on the address configured by HOST, which defaults to 127.0.0.1; setting HOST=0.0.0.0 also publishes these callback ports on all host interfaces. Because the ports are fixed by the upstream clients, only one default Compose instance can run on a host at a time.

When working over SSH or from a remote browser, the browser's localhost may not reach GPT-Load — paste the full callback URL into the authorization dialog to finish the flow.

Screenshots

Groups — View channels, models, credential counts, traffic, and health in one place

Usage statistics — Review request trends, cache hit rate, token categories, and cost estimates

Scope

Client protocols

Protocol	Main entry
OpenAI Chat Completions	POST /v1/chat/completions
OpenAI Responses	/v1/responses and its resource paths
OpenAI Images	POST /v1/images/...
OpenAI Embeddings	POST /v1/embeddings
Rerank	POST /v1/rerank
Audio Speech	POST /v1/audio/speech
Mistral native	/v1/ocr, /v1/audio/transcriptions
Anthropic Messages	POST /v1/messages
Gemini	/v1beta/models/...
Gemini Embeddings	POST /v1beta/models/{model}:embedContent / :batchEmbedContents

Built-in channels

• Official and cloud — OpenAI, Anthropic, Gemini, xAI, Azure OpenAI, AWS Bedrock, Google Vertex AI
• Model services — DeepSeek, Moonshot AI, SiliconFlow, Zhipu AI, Alibaba, Volcengine, OpenRouter, Cline, Groq, Cerebras, Mistral, Nebius, Parasail, Wafer, Hugging Face (Chat), Cohere (text reranking), OpenCode Go, OpenCode Zen
• Subscription — Codex, Claude, Antigravity, Grok
• Custom — OpenAI Compatible (any compatible relay)

Deployment and data

Docker Compose uses application-managed SQLite by default. Data lives in the gpt-load-data named volume and includes the database, auth.key, and encryption.key.

[!IMPORTANT] encryption.key decrypts channel credentials. When backing up or migrating, the database and the key must be kept together. Once the key is lost or replaced, existing encrypted credentials cannot be recovered, and this version does not support master key rotation.

Use the unified DATABASE_DSN to connect SQLite, MySQL, or PostgreSQL:

mysql://user:[email protected]:3306/gpt_load?charset=utf8mb4&collation=utf8mb4_bin
postgres://user:[email protected]:5432/gpt_load?sslmode=require

Common operations:

docker compose logs -f      # view logs
docker compose pull && docker compose up -d   # update to the latest 2.x image
docker compose stop         # stop the service

The official Compose file uses ghcr.io/tbphp/gpt-load:2. Before GA, 2 tracks verified 2.0 Beta and RC releases; after GA, it tracks stable 2.x releases only. Exact image tags omit the Git tag's v prefix (for example, 2.0.0-beta.25), while 2.0-beta remains the 2.0 Beta channel. latest remains on 1.x.

Download the build for your platform from GitHub Releases, and verify it against the bundled SHA256SUMS first:

chmod +x ./gpt-load-linux-amd64

HOST=127.0.0.1 DATA_DIR=./data ./gpt-load-linux-amd64

Then open http://127.0.0.1:3001. Portable builds are provided for five targets across Linux, macOS (amd64 / arm64), and Windows; gpt-load-windows-amd64.exe keeps running in the foreground as before.

Windows desktop users can instead download gpt-load-windows-setup.exe. After one administrator approval, Setup installs and starts a low-privilege Windows service, enables automatic startup, and creates desktop and Start Menu shortcuts to the GPT-Load management page. Setup displays the generated management key before it finishes; save it before closing the page. The protected copy remains at %ProgramData%\GPT-Load\data\auth.key. Service configuration and its .env live in %ProgramData%\GPT-Load, with persistent data in %ProgramData%\GPT-Load\data.

Installing a newer Setup stops the service gracefully before updating it. Windows uninstall removes the program and service but preserves data. Advanced users can still manage an installed service with gpt-load-windows-amd64.exe service start|stop|restart|status.

Environment configuration

At startup, the application reads .env in the current directory; existing process environment variables take precedence. Unless noted otherwise, changes require restarting the process or container; see .env.example for the common configuration template.

Variable	Default	Description
HOST	127.0.0.1	Native listening address, and the default host address for Compose's main port and OAuth callback ports; Compose always listens on 0.0.0.0 inside the container.
PORT	3001	HTTP service port, must be 1–65535; Compose also uses it for the container port, host publishing, and health check.
BIND_ADDRESS	Empty, inherits HOST	Compose only; overrides the host publishing address for the main service port without changing OAuth callback ports.
OAUTH_CALLBACK_BIND_ADDRESS	Empty, inherits HOST	Compose only; overrides the host publishing address for the fixed OAuth callback ports 1455, 54545, and 51121.
GRACEFUL_SHUTDOWN_TIMEOUT	10	Maximum time to wait for requests after a stop signal, positive integer in seconds.
CONTAINER_STOP_GRACE_PERIOD	15s	Docker duration to wait before Compose force-stops the container; should be longer than GRACEFUL_SHUTDOWN_TIMEOUT.
READ_TIMEOUT	60	HTTP request read timeout, positive integer in seconds.
IDLE_TIMEOUT	120	HTTP keep-alive idle connection timeout, positive integer in seconds.
DATA_DIR	./data	Directory for the managed database, auth.key, encryption.key, and runtime state; official Compose uses /app/data, while the Windows Setup service uses %ProgramData%\GPT-Load\data.
DATABASE_DSN	Empty, uses ${DATA_DIR}/gpt-load.db	Empty uses application-managed SQLite; non-empty values support SQLite paths or URLs, MySQL URLs, and PostgreSQL URLs, and are treated as operator-managed external databases. Container file paths must be inside a mounted directory.
DATABASE_MAX_OPEN_CONNECTIONS	10	Maximum open connections for MySQL and PostgreSQL, positive integer. SQLite always uses one connection.
DATABASE_MAX_IDLE_CONNECTIONS	5	Maximum idle connections for MySQL and PostgreSQL, positive integer and no greater than DATABASE_MAX_OPEN_CONNECTIONS. SQLite always uses one connection.
AUTH_KEY	Empty, reads or generates ${DATA_DIR}/auth.key	Bearer key for the management UI and /api management API, not a data-plane AccessKey.
ENCRYPTION_KEY	Empty, reads or generates ${DATA_DIR}/encryption.key	Encrypts channel credentials; changing or losing it makes existing credentials undecryptable, so back it up with the database.
CLIENT_IP_HEADER	Empty, uses the connection IP	Client IP header, such as X-Forwarded-For or CF-Connecting-IP; missing or invalid values fall back to the connection IP. Shared by logs, AccessKey IP restrictions, and other IP consumers; supports IPv4/IPv6. Restart after changes.
TRUSTED_PROXIES	Empty	Optional comma-separated proxy IPs or CIDRs; applies only with CLIENT_IP_HEADER. Empty means trusting the selected header directly, with trust enforced by your deployment. When set, only matching connection peers may supply the header; others use their connection IP. For X-Forwarded-For, scan right to left for the first untrusted IP (leftmost if all are trusted); without a list, use the leftmost IP. Other headers accept one IP only. Restart after changes.
HTTP_PROXY	Empty	Environment proxy for HTTP upstream requests.
HTTPS_PROXY	Empty	Environment proxy for HTTPS upstream requests.
NO_PROXY	Empty	Comma-separated hosts, domains, or IPs that bypass the environment proxy.
LOG_LEVEL	info	Supports panic, fatal, error, warn, warning, info, debug, and trace; invalid values warn and fall back to info.
LOG_FORMAT	text	Supports text and json; any other value fails startup.
MODELS_DEV_AUTO_SYNC_ENABLED	Unset, initial default true	When unset, uses the persisted management UI setting; when set, forces Models.dev auto-sync on or off and makes the same UI option read-only.

Environment proxies apply only when no proxy is specified on the credential, group, or global settings.

Moving from 1.x

[!WARNING] GPT-Load 2.0 is a complete rewrite. It cannot open, import, or migrate 1.x data in place.

Deploy 2.0 with its own database, DATA_DIR, port, and Docker volume. Cut traffic over only after verification, and keep the original 1.x deployment until the rollback window closes. Documentation for the 1.4.x maintenance line is at the official docs.

Open-source dependencies

Some of GPT-Load's capabilities build on these projects, with thanks:

Project	Role	License
Bifrost Core	Provider authentication, request/response conversion, streaming, usage normalization	Apache-2.0
CLIProxyAPI	OAuth and execution adapter for subscription channels	MIT
Lobe Icons	Channel brand icons in the management UI	MIT

GPT-Load owns credential storage, account selection, scheduling, retry, health, affinity, logging, and usage policy. Third-party notices are in THIRD_PARTY_NOTICES.md, full license texts in LICENSES/, and each release ships a CycloneDX SBOM covering the Go dependency graph.

Channel icons identify their respective upstream providers. All trademarks belong to their owners; this project is not affiliated with or endorsed by them.

Project support

---

MIT License · Third-party notices · Security policy