LocalAI

🎉 LocalAI 4.12.0 Release! 🚀

LocalAI 4.12.0 is out!

This release redesigns the web UI and adds streaming voice responses, native decision routing, and more control over model memory and prompt state. It also adds image upscaling, live motion capture, Qdrant stores, and authenticated chat history. It is also a preparation for the upcoming release which will include addon support. Our plan is to move few components (e.g. Studio, Talk) to separate add-ons that can be installed on-demand from LocalAI.

Highlights:

• 🖥️ Redesigned web UI: shared controls, light and dark themes, clearer Models and Studio workflows, and node-scoped backend maintenance.
• 🎙️ Streaming voice and bundled audio models: feed LLM fragments into Qwen TTS, use explicit clause streaming with other engines, and combine audio analysis roles through Parakeet bundles.
• 🧭 Native decision routing: classify requests with decision models and use the new /v1/decisions adapter for predicates, choices, and scores.
• 🖼️ Image upscaling in Studio: create an upscaled child image without leaving the project, with a dedicated /v1/images/upscale API.
• 🛡️ Privacy and tool configuration: Responses PII filtering, opt-in rejection of uninspectable media, alias profiles, and OAuth2 client credentials for remote MCP servers.
• 🕺 Live GEM-X motion capture: stream RGB frames into a bounded, session-based pose API on CPU or Vulkan.
• 🗄️ Persistent Qdrant stores: use the existing stores API with TLS and environment-backed credentials.
• 💾 Inference controls: configure MoE expert caching, lazy tensor reads, and explicit slot prompt-cache save and restore.

📊 This release in numbers

Counts compare v4.11.0 with the release-preparation snapshot. Direct commits include user-facing features as well as gallery and maintenance work.

📌 TL;DR

🎉 LocalAI 4.11.0 Release! 🚀

LocalAI 4.11.0 is out!

This release makes LocalAI more useful for audio understanding, resilient model serving, structured decisions, and day-to-day operation. Audio scenes can now combine transcription, diarization, sound detection, and remembered speaker names. Ordered failover chains keep a public model available across local and remote targets, while new Studio and operations pages expose these capabilities without requiring distributed mode.

The release also adds first-class decision models through /v1/systemone, signed OCI model galleries, and Kimodo text-to-animation. It includes 243 merged pull requests, 353 commits, 79 new gallery entries, focused fixes across APIs and backends, and broad backend-source updates.

Highlights:

• 🎙️ Audio scenes and remembered speakers - combine speech-to-text, speaker diarization, sound-event detection, and speaker identification. Studio can diarize a recording, preview clean intervals, and register selected speakers by name.
• 🔀 Model failover chains - place local and remote targets behind one model name, retry before response commitment, observe health and target switches, and pin a target from the API, MCP tools, or the React UI.
• 🧭 Decision models - advertise the decisions use case and answer structured choice, score, and noul questions through /v1/systemone, with validation and dedicated gallery models.
• 📦 Signed OCI galleries - distribute complete model galleries as OCI artifacts, verify them with Sigstore policies, and use digest-bound last-known-good caches.
• 🕺 Kimodo text-to-animation - generate skeletal motion from text through POST /3d/animate, preview it in Studio, and export binary glTF animation.
• 🖥️ Operate this machine - inspect resources and locally loaded models, view logs, and stop models on a single LocalAI host without enabling distributed mode.
• 🛡️ Safer gallery and API behavior - path-confinement fixes, stricter credential checks, correct pre-stream errors, preserved streamed Responses items, and more reliable model installation.

Plus PDF attachment extraction in chat, deeper Hugging Face repository discovery, improved hardware detection, new Italian Piper voices, NeMo diarization and ASR models, and large model-gallery batches.

📸 [ screenshot: diarize a recording and remember speakers by name ] Studio turns anonymous speaker segments into reusable named speaker profiles.

📊 This release in numbers

Where the work landed:

📌 TL;DR

🎉 LocalAI 4.10.0 Release! 🚀

LocalAI 4.10.0 is out!

Twenty-eight days and 280 pull requests. The work landed on three fronts: operating a fleet, feeding it models from private sources, and fixing the backends you depend on. A fleet operations dashboard replaces the flat node list with cluster-wide health, capacity, running models, and bulk lifecycle actions. A credentials file lets one credentials.yaml authenticate OCI registries, galleries and direct downloads without scattering tokens across environment variables. And local-ai benchmark measures end-to-end latency and throughput from the CLI.

The ds4 backend had four fixes that together make it reliable on CUDA: the kernels now build for the target architecture, abandoned inference is cancelled, generation boundaries are enforced, and prefilled reasoning is separated from content. Distributed mode gained seven stabilisation fixes across staging, liveness, virtual models and the control-plane database. Four CVEs were patched, enable_thinking=false is now honoured across every text backend, and Apple M5 no longer SIGSEGVs at startup.

Highlights:

• 📊 Fleet operations dashboard - a real-time cluster overview replacing the flat node list. Aggregate health bands, capacity gauges for VRAM/RAM/CPU/disk, a running-models tab showing replica placement across workers, bulk drain/resume/remove with bounded concurrency, and a slide-out node inspector. Workers now report CPU telemetry at registration and heartbeat.
• 🔑 Credentials file for private sources - one credentials.yaml authenticates OCI registries, galleries, HTTP downloads and github: URIs. Supports basic auth, bearer tokens and custom headers, with secrets read at use time so rotated K8s secret mounts take effect without restart, and full masking in logs and error messages.
• 📏 local-ai benchmark CLI command - measures end-to-end latency and throughput against configured text models. Reports min/mean/max latency and completion tokens/sec across warmup and measured runs, with table and JSON output, redirect blocking, and API-key redaction.
• 🔧 vllm-cpp LoRA fusion at load time - lora_adapters and lora_scales are now consumed by the vllm-cpp video engine, fusing LoRA deltas into DiT weights at engine load. Works for both LTX2.5 and MiniMax-H3 through one generic path.
• 🌍 Environment variables per model - an env: map in model config injects variables into the backend process environment, so CUDA_VISIBLE_DEVICES or VLLM_CACHE_DIR can be pinned per model without wrapping the backend in a script.
• 💬 template.system_messages_after_first - agent frameworks that append system instructions mid-conversation no longer crash Qwen3.x models. Late system turns can be merged into the leading system message or forwarded as user turns.
• 🧠 context_size in /v1/models/capabilities - clients can now read each model's effective context window to budget tokens and truncate prompts without probing the model config out-of-band.
• 🛡️ Four CVEs patched - ip-address 10.3.1 (CVE-2026-69192), containerd 1.7.32 and 1.7.33 (CVE-2026-46680, CVE-2026-53488), and react-router (GHSA-qwww-vcr4-c8h2).
• 🍎 Apple M5 SIGSEGV fixed - go-m1cpu bumped to v0.2.2, fixing a segmentation fault on Apple M5 hardware at startup.

Plus a focus mode toggle in chat settings, face enrollment replay for the 1:N registry, a ROCm build of stablediffusion-ggml, in-place editing of distributed scheduling rules, OCI layer download resumption, and 140 new gallery entries.

📊 This release in numbers

🎉 LocalAI 4.9.0 Release! 🚀

LocalAI 4.9.0 is out!

Thirteen days and 146 pull requests, spent on the parts of LocalAI you touch every day rather than on new engines. Authentication is now deny-by-default, chat gained end-to-end context compression, models and backends each have one canonical page instead of three, and vllm-cpp grew a video modality serving MiniMax-H3 with a real audio track.

Highlights:

• 🔐 Authentication is deny-by-default - every HTTP route requires credentials unless it appears in an explicit public registry. This closes a class of bypass in which unprefixed aliases such as /moderations, /models, /backends and /mcp/chat/completions fell outside the old protected-prefix list. Reported by Naor Yaacov.
• 🗜️ Chat context compression - opt-in per model, older complete turns are compressed through a LocalAI model before inference, preserving system prompts, the newest messages and whole tool-call units. Ratio and duration come back as response metadata and metrics.
• 🖥️ One page per resource - /app/models now owns Explore and Installed, /app/backends owns Catalog and Installed, and the nested Host view is gone. Old /app/manage bookmarks still work.
• 🎬 MiniMax-H3 video generation - vllm-cpp opens a second engine handle for the H3 checkpoint set and renders video and audio jointly, so the MP4 arrives with a real AAC track. Ask for speech in the prompt and the model lip-syncs it.
• 🗣️ Qwen3-TTS on llama.cpp - text-to-speech on the full accelerator matrix already shipped for text generation (CUDA, ROCm, SYCL, Vulkan, Metal, L4T), using upstream's own GGUF conversion.
• 🧭 KNN as a first-class router - similarity-weighted voting over a curated, persisted corpus of labelled prompts. No classifier model, and a prompt unlike anything labelled is treated as undecidable rather than guessed.
• 📊 Global admission control and live backend traces - process-wide HTTP admission bounds, in-flight backend operations are represented while they run, and the UI links straight to their logs.
• 🕵️ Reversible PII pseudonyms - masked values become request-scoped deterministic pseudonyms (EMAIL_001) and are restored if the backend echoes them, across JSON and SSE tokens split over writes.
• 📦 Parallel Hugging Face downloads - snapshot materialization runs up to N whole-file transfers at once, so a repository split into many shards stops spending its wall clock in per-file latency.
• 🖧 Cold model loads are durable jobs - the per-model advisory lock no longer spans a multi-GB transfer, which had made a 35.7 GB load look permanently broken from the operator's seat while staging progressed normally underneath.
• 🎮 vllm-cpp covers the cards you own - CUDA builds went from one or two architectures to eight on amd64 and five on arm64, picking up A100, L4, 4090, H100/H200, B200, Jetson Orin and Jetson Thor.

Plus a single shared WebRTC UDP port for Realtime, Metal actually enabled in the macOS Stable Diffusion and Parakeet builds, backend crash diagnostics at the default log level, and Portuguese (Brazil) and Indonesian UI translations.

📊 This release in numbers