Atomic-Chat

AtomicBot-ai活跃维护

本地运行的 AI 聊天应用与推理引擎，基于 llama.cpp 加载 GGUF 开源权重模型，完全离线、保护隐私，支持 Apple Silicon 桌面端。

Local AI app and inference engine for agents. Run open-weight LLMs locally — private, 100% offline on your computer. Join our Discord: https://discord.com/invite/8wGSsvmg4V

v2.3.5发布于 2026-11-04 22:41 UTC1,973star +17 / 7 天231fork同步于 2026-11-10 11:45 UTC

本地推理离线聊天 GGUF llama.cpp 桌面应用

对比 GitHub 官网 ai-chat ai-tools apple-silicon chatgpt deepseek

怎么上手

由 README 提炼，执行前请确认

原文

从 Releases 下载对应平台安装包（macOS universal dmg / Windows x64 setup.exe / Linux AppImage），启动后在应用内加载模型；随后即可调用：curl http://localhost:1337/v1/chat/completions -H "Content-Type: application/json" -d '{"model":"<model-id-loaded-in-atomic-chat>","messages":[{"role":"user","content":"Say hello in one word"}]}'

硬件要求按 README 说明

README 未给出具体显存数字；llama.cpp 系引擎支持 CPU、Metal、CUDA、Vulkan，MLX 需 Apple Silicon，TensorRT-LLM/vLLM/SGLang 需 NVIDIA GPU（Windows 走 WSL 2），GenieX 需骁龙 X 系列。

支持 Apple Silicon需要 CUDA支持 ROCm / AMD

下载：源码 zip·GitHub Releases（安装包）本站不镜像安装包

项目说明

由 README 自动翻译提炼

没人工点评？写下你的经验

本地运行开源大模型的桌面应用与推理引擎，隐私离线，还能当 OpenAI 兼容 API 用

本地运行的 AI 聊天应用与推理引擎，基于 llama.cpp 加载 GGUF 开源权重模型，完全离线、保护隐私，支持 Apple Silicon 桌面端。

核心能力

内置 12 种推理引擎，统一走 OpenAI 兼容接口
本地起 http://localhost:1337/v1 服务，可直接替换 OpenAI SDK
支持 MTP、DFlash、EAGLE-3 等投机解码加速
TurboQuant KV 缓存把显存占用压到约 1/4
一键拉起 Claude Code、Cline、Goose 等 Agent

容易踩的坑

本地 API 默认只绑定 127.0.0.1，局域网访问需改成 host: 0.0.0.0；模型需先在应用内加载，请求里的 model 要填已加载的 model-id；部分引擎（如 TensorRT-LLM、vLLM、SGLang）需额外下载并在 Settings → Providers 里一键安装。

内容由模型根据 README 生成，可能不完整或不准确，以官方文档为准。看原文 README

用户点评

登录用户的真实使用记录，可按版本和硬件筛选

共 0 条

还没有人点评。第一个说出真实体验的人最有价值。

README 原文

未翻译

在 GitHub 打开

Atomic Chat

Local AI app and inference engine for agents. Run open-weight LLMs locally — private, on your machine.

Getting Started · Hugging Face · Discord · X / Twitter · Bug Reports

📦 Download

🔌 Use It as an API

Atomic Chat runs an OpenAI-compatible server at http://localhost:1337/v1 — a drop-in replacement for the OpenAI SDK. Load a model in the app, then point any client at it:

curl http://localhost:1337/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<model-id-loaded-in-atomic-chat>",
    "messages": [{ "role": "user", "content": "Say hello in one word" }]
  }'

from openai import OpenAI

# Atomic Chat is OpenAI API-compatible — only the base_url changes.
client = OpenAI(base_url="http://localhost:1337/v1", api_key="not-needed")

resp = client.chat.completions.create(
    model="<model-id-loaded-in-atomic-chat>",
    messages=[{"role": "user", "content": "Say hello in one word"}],
)
print(resp.choices[0].message.content)

Bound to 127.0.0.1 by default; set host: 0.0.0.0 to expose it on your LAN. Works with any agent, CLI, or IDE plugin that speaks the OpenAI API — see Launch With below.

✨ Features

Local models

Run open-weight LLMs locally from HuggingFace — Llama, Gemma, Qwen, Mistral, Phi, and others
Multi-Token Prediction (MTP) speculative decoding — 30–70% throughput boost on supported models, up to 3× on Gemma 4
DFlash block-diffusion decoding — up to 6× faster on Qwen 3.6, Gemma 4, Kimi K2.5
Flash Attention toggle (on / off / auto)
Automatic reasoning-context tracking for chain-of-thought models
Auto context-window expansion with overflow notifications
EAGLE-3 speculative decoding for Gemma 4 on Apple Silicon (MLX)
MTP on MLX for Qwen 3.5 / 3.6 and DeepSeek V4
TurboQuant KV cache (turbo3 / turbo4) on llama.cpp — now on Windows & Linux too, not just macOS: up to ~4.3× smaller KV cache footprint, CPU and GPU (CUDA / Vulkan)
TurboQuant KV cache on MLX-VLM — smaller memory footprint via RHT-correct fast paths

Cloud models

Built-in providers: OpenAI, Anthropic, Mistral, Groq, MiniMax, Qwen, Moonshot
Bring your own key, switch model per chat, mix local and cloud freely

Tools & integrations

One-click agent launch — launch agents like Atomic Agent, Claude Code, Codex CLI, Cline, OpenCode, Droid, Goose, OpenHands, Copilot CLI, Kilo Code, Zed and ZCode in one click from the Integrations tab
Artifacts — live preview panel for HTML/CSS/JS code with copy, download and print
Connect multiple MCP servers — bring your own tools, file access, web search
Custom assistants with per-assistant system prompts
Projects with conversation tree view in the sidebar

Local API

OpenAI-compatible server at http://localhost:1337/v1 — drop-in replacement for the OpenAI SDK
Works with any agent, CLI, or IDE plugin that speaks the OpenAI API
Bound to 127.0.0.1 by default; set host: 0.0.0.0 to expose on LAN

Privacy

Everything runs locally when you want it to — local server is loopback-only by default
Your conversations and keys stay on your machine

⚙️ Inference Engines

Twelve engines under the hood, all exposed through one OpenAI-compatible API at http://localhost:1337/v1. Each shows up as a provider only where it can run; the ones that need a download install from Settings → Providers in one click.

Text and vision models

Engine	What it runs	Where
Upstream llama.cpp	GGUF models; the official `ggml-org` build, with MTP. The default engine on Windows and Linux for the widest hardware coverage.	macOS, Windows, Linux — CPU, Metal, CUDA, Vulkan
atomic-llama-cpp-turboquant	GGUF models on our `llama.cpp` fork with TurboQuant KV cache (`turbo3` / `turbo4`) for faster, lower-memory quantized inference ("Atomic Llama.cpp Turboquant").	macOS, Windows, Linux — CPU, Metal, CUDA, Vulkan
PrismML llama.cpp	PrismML's Bonsai models, whose 1- and 2-bit tensor types the other llama.cpp builds don't load.	macOS, Windows x64, Linux x64 — CPU, Metal, CUDA, Vulkan, ROCm
MLX-VLM	MLX models, vision-language included, with DFlash, EAGLE-3 and MTP. Faster than llama.cpp on M-series chips for supported models.	macOS on Apple Silicon
Apple Foundation Models	Apple's on-device model, nothing to download.	macOS 26+ with Apple Intelligence enabled
Foundry Local	Microsoft's ONNX models on the CPU, GPU or NPU.	Windows x64 and arm64, macOS 15+ on Apple Silicon, Linux
LiteRT-LM	Google's on-device `.litertlm` models (Gemma and others), with the Intel NPU through OpenVINO on Windows.	macOS 12+ on Apple Silicon, Windows x64, Linux
GenieX	Qualcomm's runtime for Snapdragon: GGUF models on the Hexagon NPU, Adreno GPU or CPU, and Qualcomm AI Hub bundles on the NPU.	Windows on Snapdragon X (X Elite, X Plus, X, X2 Elite, X2 Plus)
TensorRT-LLM	NVIDIA's high-throughput server for Hugging Face checkpoints.	Linux, and Windows x64 through WSL 2 — NVIDIA GPU
vLLM	Hugging Face checkpoints on vLLM's server.	Linux, and Windows x64 through WSL 2 — NVIDIA GPU
SGLang	Hugging Face checkpoints on SGLang's server.	Linux, and Windows x64 through WSL 2 — NVIDIA GPU

TensorRT-LLM, vLLM and SGLang run in a container that Atomic Chat sets up and manages itself; on Windows it lives in Atomic Chat's own WSL distribution.

Images and video

Engine	What it runs	Where
stable-diffusion.cpp (our build)	Images (FLUX, Qwen-Image, Z-Image, Krea) and video (Wan 2.2, LTX-2), in the Images and Videos tabs.	macOS on Apple Silicon, Windows, Linux — CPU, Metal, CUDA, Vulkan, ROCm

Speculative-decoding features available across backends:

MTP (Multi-Token Prediction) — a draft model predicts ahead, the full model verifies in one pass. Available on macOS and Windows.
DFlash — block-diffusion speculative decoding for Qwen 3.6, Gemma 4, Kimi K2.5 and others. Apple Silicon only; can't be enabled together with MTP.
Flash Attention — Settings → on / off / auto.

Tools talking to http://localhost:1337/v1 don't need to know which backend is running underneath — switch engines without reconfiguring clients.

The engines are driven by atomic-chat-core — the core of Atomic Chat, in its own repository. It doesn't do inference itself; it manages the engines: downloads and launches backends, manages models, routes requests between local engines and cloud providers, and serves the OpenAI-compatible API on :1337. The desktop app ships a pinned core release; the core also runs on its own as a CLI (atomic-chat-core serve …) or as a library.

🚀 Launch With

Atomic Chat runs an OpenAI-compatible server at http://localhost:1337/v1, so any agent, CLI, IDE plugin, or app that speaks the OpenAI API can run on top of your local models — no extra glue needed. Just point its base URL at Atomic Chat and you're done.

A few projects already ship first-class support with their own setup docs:

Tool	What it is	Setup
OpenCode	Open-source TUI coding agent. Add Atomic Chat as a local provider in `opencode.json`.	Setup guide →
Goose	Open-source extensible AI agent (CLI, desktop, API).	Setup guide →
nanobot	Ultra-lightweight personal AI agent with chat channels, MCP, and WebUI.	Repo →
nanoclaw	Containerized agent runtime that calls Atomic Chat as an MCP tool.	Skill guide →
OpenClaude	Open-source coding-agent CLI for cloud and local models. Lists Atomic Chat as a supported provider.	Providers list →
Kilo Code	Open-source AI coding agent for VS Code, JetBrains, and CLI. Ships with first-class Atomic Chat provider support and auto-discovery.	Setup guide →
Hermes Desktop	Native desktop companion for Hermes Agent. Includes an Atomic Chat local preset at `http://localhost:1337/v1`.	Repo →
Hermes Workspace	Local-first agent workspace built on Nous Research's Hermes. Uses Atomic Chat as its inference backend.	Repo →

Built something that runs on Atomic Chat? Open a PR and we'll add it here.

🛠️ Build from Source

Prerequisites

Node.js ≥ 20.0.0
Yarn ≥ 4.5.3
Make ≥ 3.81
Rust (for Tauri)
(Apple Silicon) MetalToolchain xcodebuild -downloadComponent MetalToolchain

Run with Make

git clone https://github.com/AtomicBot-ai/Atomic-Chat
cd Atomic-Chat
make dev

This handles everything: installs dependencies, builds core components, and launches the app.

Available make targets:

make dev — full development setup and launch
make build — production build
make test — run tests and linting
make clean — delete everything and start fresh

Manual Commands

yarn install
yarn build:tauri:plugin:api
yarn build:core
yarn build:extensions
yarn dev

💻 System Requirements

macOS: 13.6+ (8GB RAM for 3B models, 16GB for 7B, 32GB for 13B)
Windows: 10/11 x64 (same RAM recommendations as macOS)
Windows on Arm: 11 arm64 on a Snapdragon X — the upstream llama.cpp engine (CPU, or the Adreno GPU through OpenCL), Foundry Local, and GenieX on the Hexagon NPU
Linux: x86_64 with glibc ≥ 2.35 (Ubuntu 22.04+, Debian 12+, Fedora 40+, Arch, Mint, Pop!_OS), or arm64 with glibc ≥ 2.38 (Ubuntu 24.04+, Debian 13+, Fedora 40+, Arch) — same RAM recommendations as macOS. Optional: a Vulkan loader (vulkan-1 package, or mesa-vulkan-drivers / proprietary NVIDIA driver) for GPU acceleration.
iOS: download from App Store
Android: download from Google Play

🐧 Running on Linux

Install it from the Atomic Chat package repository, and your package manager updates it with the rest of the system. The packages put the app in /opt/atomic-chat with an atomic-chat command, a menu entry, and the system's WebKitGTK; the built-in updater stays off. On arm64 they need glibc 2.38 or newer, which the bundled llama.cpp engine is built against there. The repository and the rpm packages are signed with the Atomic Chat packages key in packaging/atomic-chat.asc, fingerprint 1047 F119 58C6 442D D472 E497 E4C3 D361 FB7F F123.

Debian, Ubuntu — Debian 12+ / Ubuntu 22.04+ on x86-64, Debian 13+ / Ubuntu 24.04+ on arm64:

curl -fsSL https://atomic-chat-packages.fsn1.your-objectstorage.com/atomic-chat.gpg | sudo tee /usr/share/keyrings/atomic-chat-app.gpg >/dev/null
echo "deb [signed-by=/usr/share/keyrings/atomic-chat-app.gpg] https://atomic-chat-packages.fsn1.your-objectstorage.com/deb stable main" \
  | sudo tee /etc/apt/sources.list.d/atomic-chat-app.list
sudo apt update && sudo apt install atomic-chat

Fedora 40+ (x86-64, aarch64):

sudo curl -fsSL -o /etc/yum.repos.d/atomic-chat-app.repo https://atomic-chat-packages.fsn1.your-objectstorage.com/rpm/atomic-chat.repo
sudo dnf install atomic-chat

Arch Linux (x86-64, aarch64): yay -S atomic-chat-bin, or any other AUR helper.

Then start Atomic Chat from the app menu, or run atomic-chat.

Updates come with sudo apt upgrade, sudo dnf upgrade or yay -Syu. sudo apt remove atomic-chat, sudo dnf remove atomic-chat or sudo pacman -R atomic-chat-bin removes the app and leaves your chats and models. The repository keeps the last three releases; every release also carries its .deb and .rpm files on the releases page.

Or run the self-contained .AppImage — no installer, no root, and it updates itself:

chmod +x Atomic.Chat_*_amd64.AppImage
./Atomic.Chat_*_amd64.AppImage

If prompted about FUSE on first launch: sudo apt install fuse libfuse2 (Debian/Ubuntu) or sudo dnf install fuse fuse-libs (Fedora). GPU acceleration (Vulkan) is auto-detected on first launch; only GGUF models run on Linux.

When your bar has a tray (Omarchy, Waybar's tray module, KDE), closing the window keeps Atomic Chat and its local API on :1337 running in the tray; use Quit in the tray menu to stop it. Launching the app again brings the window back. Without a tray (GNOME with no indicator extension), closing the window quits as usual.

Window rules (Hyprland, Sway). Match the Wayland app id (Hyprland class) chat.atomic.app, e.g. ^chat\.atomic\.app$. Under X11/XWayland the window class is still Atomic-Chat.

Packaging. A distro package (AUR, a pacman repo, Flatpak) that owns updates should place an empty atomic-chat-package-managed file next to the .AppImage (or, for an extracted AppImage, next to usr/bin/Atomic-Chat), or start the app with ATOMIC_CHAT_PACKAGE_MANAGED=1. Prefer the file: the "launch at startup" entry runs the AppImage or binary directly, so it does not see a variable set in a launcher script. The built-in updater then never checks for, offers or installs updates, and Settings points users to their package manager.

🧯 Troubleshooting

If something isn't working:

Copy your error logs and system specs
Open an issue on GitHub
Or ask for help in our Discord

👥 Contributors

Atomic Chat is built by a small core team and 140+ contributors — including everyone who shaped the project from its earliest days. Pull requests are welcome in both repositories:

Atomic-Chat (this repo) — the desktop and mobile app: UI, chats, assistants, projects, MCP, integrations, settings. See CONTRIBUTING.md to get started.
atomic-chat-core — the core that manages the engines (llama.cpp, MLX, Foundation Models, Foundry Local, LiteRT-LM, GenieX, TensorRT-LLM, vLLM, SGLang, stable-diffusion.cpp): backend and model management, cloud providers, the router and the OpenAI-compatible server. See its AGENTS.md to get started.

Not sure where a change belongs? Open an issue in either repo and we'll point you the right way.

⭐ Star History

📄 License

Apache 2.0 — see LICENSE for details.

🙏 Acknowledgements

Built on the shoulders of giants:

🌱 Heritage

Atomic Chat began as a fork of Jan by Menlo Research — an excellent open-source local-AI app. We're grateful to the Jan team and its contributors for the foundation they built. Atomic Chat has since grown its own direction, engines, and roadmap, but we tip our hat to where it started. 🙏

怎么上手

项目说明

核心能力

容易踩的坑

最近发版

🚀 New Features

🔧 Improvements & Fixes

🚀 New Features

🔧 Improvements & Fixes

⚠️ Breaking Changes

🙏 Contributors

🚀 New Features

🔧 Improvements & Fixes

🚀 New Features

🔧 Improvements & Fixes

🙏 Contributors

🚀 New Features

🔧 Improvements & Fixes

⚠️ Breaking Changes

🙏 Contributors

用户点评

README 原文

Atomic Chat

📦 Download

🔌 Use It as an API

✨ Features

⚙️ Inference Engines

🚀 Launch With

🛠️ Build from Source

Prerequisites

Run with Make

Manual Commands

💻 System Requirements

🐧 Running on Linux

🧯 Troubleshooting

👥 Contributors

⭐ Star History

📄 License

🙏 Acknowledgements

🌱 Heritage

相关项目