splash

Splash 1.5.0

Splash now has a desktop app: download and run models, chat with them, and connect coding agents from one window. This release also adds local model loading, custom tools in the Responses API, faster decode on M5 and M6, and a fix for image requests that could crash the engine.

Splash for Mac

The desktop app includes its own runtime, shares models with the CLI, and supports:

• Model downloads, switching, and launch settings.
• Streaming chat, saved conversations, images, and reasoning controls.
• Claude Code, Codex, OpenCode, Hermes, and Pi in your chosen project folder.
• Menu bar controls and automatic update checks.

Requires Apple M3 or newer and macOS 26.4 or later. Download Splash, open the DMG, and drag Splash into Applications. Or install with Homebrew:

brew install --cask incoai/tap/splash-app

The app uses the default Hugging Face cache and does not inherit HF_HOME, HF_HUB_CACHE, or HF_TOKEN from your shell. Quitting it stops the model. GGUF files added locally run text only; use the CLI for a separate vision projector or local draft. MLX folders retain their vision support. (#610)

Speed and memory

• Faster decode on M5 and M6. On a 32-core M5 Max, Qwen3.8-27B MLX 4-bit takes 11.7% less time per decode cycle with one request and 7.5–8.3% less with two to four. The 27B UD-Q4_K_M takes 2.2–4.4% less. On a 12-core M6, the 35B UD-Q2_K_XL takes 0.4–1.8% less. These are decode-cycle measurements; per-token gains for the batched 27B MLX are 1–2 percentage points smaller at three and four requests. Other GPU core counts were not measured end to end. (#235)
• Lower MoE router memory. The 35B MLX 4-bit keeps its router quantized, saving 62 MB and reducing router reads from 84 MB to 22 MB per decode cycle. Quantized GGUF MoE routers also load; F16 routers remain unsupported. (#599)
• Faster quantized kernels on M3, M4, and M5, including MLX 4-bit and several GGUF formats. These are kernel-level improvements; whole-model gains vary. (#600, #602)

Models and APIs

• Local model files: serve a GGUF file or MLX directory with --model-dir, without contacting the Hub. Use --draft-model for a local DFlash2 draft and --mmproj for a GGUF vision projector. Otherwise, the matching draft must already be cached. (#338)

  splash serve --model local/qwen \
    --model-dir /models/Qwen3.8-27B-GGUF/Qwen3.8-27B-UD-Q4_K_M.gguf \
    --draft-model /models/Qwen3.8-27B-DFlash2
  

• Custom tools: Responses accepts freeform tools such as Codex's apply_patch and returns custom_tool_call items. Tool grammars are described to the model, not enforced; streamed tool input arrives when the call closes. (#426)

• Request timings: Responses and Messages report prompt/decode counts, latency, rates, cached tokens, and draft counts. Messages also reports thinking-token usage. (#427)

• Reasoning discovery: /v1/models reports supported reasoning efforts and defaults. (#610)

• Pi prefix caching: splash pi enables later system messages when the model's template supports them, preserving reusable prefixes. (#609)

• Shell completion: expands to 11 suggested models and completes local model, draft, and projector paths. (#338, #610)

Fixes

• Image requests no longer trigger GPU timeouts on models whose experts stream from SSD. Image encoding runs before prefill in a separate GPU command, and image requests use Neural Engine prefill again. (#601, fixes #596)
• Ctrl+C stops model downloads promptly. The next download from that repository cleans up interrupted partial files. (#610)
• Browser chat asks for an API key only after a 401 response. (#610)
• Update filelock to 4.2.4. (#603)

Upgrade

brew upgrade splash

Keep the CLI and desktop app on the same release.

• Use full CLI option names: abbreviations such as --max-mem are no lon

Splash 1.4.1

Splash 1.4.1 adds Ornith-1.5-122B and Flint, improves MoE performance on base M3/M4 Macs, and fixes startup and request stalls under memory pressure.

New models

Ornith-1.5-122B loads from GGUF or MLX with its matching DFlash2 draft selected automatically (#539, #560).

splash serve --model unsloth/Ornith-1.5-122B-GGUF:UD-Q4_K_XL

GGUF supports vision. ornith-ai/Ornith-1.5-122B-MLX-4bit also includes its vision tower; --language-only is optional.

Flint, RecursiveSI's 40B-A2.5B MoE, serves from GGUF with automatic incoai/Flint-DFlash2 selection, tool calling, structured output, and up to 262,144 tokens of context (#573–#576).

splash serve --model unsloth/flint-GGUF:UD-Q4_K_XL

Flint is text-only and decides when to reason itself; reasoning settings cannot force it on or off. MLX checkpoints are not supported.

Pruned-expert GGUFs now load, including ISTA's Flash-Next GSQ-RCO-Coder variants (#561).

Speed and memory

• Flint measured 190 tok/s across ten mixed prompts on an M5 Max with 36 GB, versus 89 tok/s for llama.cpp on the same Mac and UD-Q4_K_XL checkpoint. These are the pooled results from four runs (#576).
• Flint releases sliding-window K/V once it is no longer needed. With INT8 KV, cached pages beyond the window use 71% less memory; full-history attention is retained (#574).
• Faster MoE kernels on base M3/M4 GPUs with ten cores or fewer. On the ten-core M4, most tested formats took 12–45% less GPU time per MoE layer in small-batch benchmarks. This is a kernel measurement, not an end-to-end speedup (#578).

Fixes

• Low-memory startup: release idle warmup buffers before checking the host memory reserve, avoiding false startup refusals on 24 GB Macs. Startup failures also report their actual reason instead of a generic EOF error (#577, #563).
• Large images: fix GPU timeouts when the 27B's image encoding overlaps ANE prefill. Image-encoding commands run on the GPU; subsequent text chunks still use ANE (#593).
• Concurrent requests: fix queued requests stalling under memory pressure while another request grows its KV cache, while preserving capacity for long single requests (#579).
• Accept split GGUFs whose first file contains only metadata (#560).
• Report missing MLX vision shards clearly and suggest --language-only for affected checkpoints (#568).
• Update Pillow to 13.0.0 and certifi to 2026.10.19 (#558).
• Improve model verification checks, including penalty and warmup-EOS checks, memory handling, and stale weight records (#562, #564–#567).

Upgrade

Existing GGUF installations refresh derived metadata locally on first start, reusing downloaded weights. Their persistent conversation cache starts fresh because the model identity changes. Existing MLX cache layouts remain compatible when model and draft identities are unchanged.

Thanks to @zhijian-liu, @ezoushen, @wmTJc9IK0Q, @linson007, @cresslank, @Litee and @realzop for contributions, reports and testing.

Full changelog: https://github.com/incoai/splash/compare/1.4.0...1.4.1

Splash 1.4.0 adds Qwen3.8-Flash-Next, SSD streaming for experts and PLE, active KV spill, and up to 1M context. Adaptive draft verification and mixed structured-output batching improve decode throughput.

Qwen3.8-Flash-Next

Flash-Next GGUFs now load with QSA sparse attention and an automatically selected DFlash2 draft (#467–#469):

splash serve --model ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF:Q2_0

Unsloth's Flash-Next GGUFs also load. Flash-Next MLX checkpoints are not supported yet.

On an M5 Pro (20-core GPU, 64 GB), Q2_0 measured 95 tok/s decode, 1,312 tok/s prefill at 32K, 171 ms cached TTFT, and 171 tok/s aggregate decode with four concurrent requests. These use selected SPEED-Bench coding prompts over HTTP with 1,024-token outputs; prefill rate uses time to the first token, and all 16 replayed prompts hit the cache (#521).

SSD streaming

The planner shares one memory budget between weights, context and streaming caches (#463, #468, #474–#481, #495):

• MoE experts: GGUF and MLX targets keep experts resident when they fit, otherwise stream them from the SSD. --expert-memory SIZE sets their RAM store; off loads every expert.
• Flash-Next PLE: the n-gram table streams when it cannot fit, with a default 512 MiB row cache. --ple-memory SIZE changes that cache; off loads the whole table.
• Active KV: Flash-Next can spill a running request's KV to the SSD, keeping its sparse index and hot blocks in RAM. --sparse-kv-memory SIZE controls the hot cache. The 27B and 35B keep active KV resident.

Streaming is automatic and trades SSD reads for RAM. KV spill requires a writable SSD cache with available quota. --max-memory leaves room for other applications and shrinks the automatic stores. Streamed prefill runs in passes so other requests can decode and cancellations take effect between passes.

Up to 1M context

All three families support static YaRN beyond their native 262,144-token window. Add --max-context 1M to enable it; 1M means 1,000,000 tokens, with an upper limit of 1,010,000 (#431, #469, #518).

The requested context must fit available RAM and SSD resources. YaRN stays off within the native window and changes outputs even on short prompts when enabled, so request the longer window when needed.

Faster decode

MoE targets adapt how many draft tokens they verify, avoiding work on unlikely proposals (#487). Against verifying every row, Flash-Next Q2_0 with a 12G expert store gained 13–38% decode throughput at one to four concurrent requests on an M5 Max 36 GB. The 35B UD-Q2_K_XL with a 4G store gained 6.5–42% on an M6 24 GB.

JSON Schema and ordinary requests now share decode batches (#520). With two of each on an M5 Max 36 GB, aggregate throughput rose 349 → 457 tok/s for the 35B MLX 4-bit and 138 → 210 tok/s for the 27B MLX 4-bit. Single-request throughput was unchanged within noise.

These comparisons measure individual changes. Metal optimizations also improve MoE projections, gathering, low-bit decode and sparse attention.

Other changes

• Preserve conversation replay states during routine memory reclamation, and durable restart points when the SSD cache fills. Fix spill-resume accounting, pinned-cache lifetime errors and engine relaunch with spilled KV (#484, #512, #513, #524).
• A single unexpected exit no longer clears persistent cache; repeated failures or engine-reported execution faults during startup probation still can (#498).
• Honor explicit --max-cache-disk SIZE quotas subject to the volume's free-space floor. Fix a host-step race that could cause GPU timeouts, and avoid watchdog restarts while streamed commands make progress (#479, #494, #515).
• Load split GGUFs, Q2_0 and additional GGUF norm/router/gate tensor types. Add network-free install.local.prepare_local_model() for local checkpoints (#451, #452, #500).
• Normalize tool-schema type aliases and preserve untyped arguments as text. Bound schema/prompt cache

Splash 1.3.2 adds NVFP4 and FP8 checkpoints, more GGUF layouts, optional BF16 recurrent states and thinking budgets. The SSD prefix cache is now on by default, with faster prefill and lower TTFT.

Models

These changes apply to Qwen3.8-27B and Qwen3.6-35B-A3B checkpoints and compatible fine-tunes:

• Load MLX nvfp4, NVIDIA Model Optimizer and compressed-tensors NVFP4/FP8 checkpoints, and llama.cpp NVFP4 GGUFs. For example: nvidia/Qwen3.8-27B-NVFP4 and mlx-community/Qwen3.6-35B-A3B-nvfp4 (#356).
• Load more GGUF variants, including UD-Q8_K_XL, Q5_0/Q5_1 weights, IQ1_M and IQ2_S token tables, other previously refused embedding types, fused MoE gate/up experts and mixed-type GDN alpha/beta tensors. Quantized checkpoints can now contain BF16 and F16 weights; whole BF16, F16 and F32 targets remain unsupported (#379, #403; fixes #375 and #394).

Memory and caching

• --state-format bf16 halves the GDN recurrent-state storage. A cached state shrinks from about 153 to 81 MiB on the 27B and from 64 to 34 MiB on the 35B. Computation stays FP32, with rounding when BF16 states are stored; FP32 remains the default (#385).
• The SSD prefix cache defaults to auto, with a quota up to 32 GiB that follows available space on its local volume. --max-cache-disk 0 turns it off. The cache adjusts as the volume fills and can keep serving after a full-disk write; unused persistent caches are cleaned up after 14 days (#408).
• --persistent-cache keeps the SSD cache across restarts and remains off by default. It now works without an explicit disk quota. --max-cache-disk-writes SIZE reduces optional writes to a daily budget; conversation resume states, their KV pages and the clean-stop flush can exceed it. Writes have no budget unless one is set (#408).
• The RAM prefix cache grows only into spare memory and gives memory back when macOS warns. Idle reclaim avoids repeatedly discarding an extra recovery margin when no request needs it (#383, #418; fixes #325).
• POST /v1/memory/release frees idle model weights on demand while keeping the prefix cache. It works with --idle-release off; the next request restores the weights. A busy engine or a restore in progress returns 409 engine_busy, and 200 confirms the weights are released (#374, #416).
• cache_salt isolates KV and cached states between different salts, including on disk. Chat/text completion metrics and the Done log now report KV reused beyond a request's initial cached prefix as deduplicated_tokens, including reuse after suspension (#382, #415; fixes #368).

Thinking and agent compatibility

• Limit reasoning with thinking_token_budget in Chat Completions and Responses, or set a server default with --thinking-token-budget N. The default is uncapped; -1 disables the cap for a request. Messages honors enabled thinking's required budget_tokens (at least 1024) and accepts budgets at or above max_tokens (#397, #411; fixes #373).
• --thinking-end-text TEXT can turn the model toward its answer when the budget runs out. The text must end with </think> and fit within 64 tokens. With structured output or constrained tools, the cutoff waits for a complete character before closing thinking, avoiding grammar failures on split Unicode characters; reasoning can therefore run a few tokens past the budget (#405, #421).
• Messages applies Claude Code's mid-session tool_addition and tool_removal blocks in conversation order, including additions by reference. Later definitions replace tools of the same name (#391; fixes #389).
• Tool names can contain dots, spaces, punctuation and Unicode, up to 256 characters within the call format's rules. Non-strict tools with unusable JSON Schema descriptions no longer reject the whole request; strict tools still require valid schemas (#400; fixes #392).
• Output limits larger than the remaining context now generate up to the window's end instead of returning 400. Optional null fields, seed: -1, stop strings alongside tools o