mlx-dspark

mlx-dspark runs two EAGLE-family speculative-decoding drafters natively on Apple Silicon: DeepSeek's DSpark (semi-autoregressive, from the DeepSpec codebase, used to accelerate DeepSeek-V4) and z-lab's DFlash (block diffusion). Both are lossless — the target verifies every token, so output is identical to normal decoding — and run under one verify loop, so you can serve them, script them, or benchmark them head-to-head.

Documentation: arahim.dev/mlx-dspark — setup for every agent and app, the HTTP API, a model picker, and how speculation behaves on a Mac.

What this is not: DeepSeek-V4 inference. The targets are consumer-size models (Gemma-4, Qwen3, Meta's Muse-Glimmer, NVIDIA's Nemotron, PrismML's ternary Bonsai-27B, …) with published DSpark drafters — so this runs the real drafter method on a Mac, but the model producing tokens is one of those, not V4. V4 Flash/Pro (MoE, batched serving) is DSpark's own headline use case.

Supported models

Every row auto-resolves its drafter from --model (any quant of the target matches). Measured warm on an M4 Pro, medians of 3 — most rows with mlx-dspark benchmark --trials 3 (three prompts: chat/code/math), the Muse row per-content best (footnoted); full tables, baselines, and method in Results at a glance. Sorted by best measured speedup. These are short-context numbers; at agent-sized prompts (16k–32k tokens) see Long context:

target	best measured speedup	speed (chat → best)
Nanbeige4.2-3B (bf16, looped)[^nanbeige]	4.31× math · 2.50× code · 1.98× chat	~34–74 tok/s
Gemma-4 12B (8-bit)	4.24× math · 2.90× code · 2.62× chat	~48–78 tok/s
Qwen3.8-27B (8-bit, DFlash 2)[^q38]	4.22× math · 3.80× code · 2.99× chat	~25–35 tok/s
MiniCPM5-2B (bf16, dense)[^minicpm5]	4.21× math · 3.15× code · 2.12× chat	~107–213 tok/s
LFM2.5-1.2B (bf16, conv-hybrid)[^lfm2]	3.83× code · 3.82× math · 2.40× chat	~240–385 tok/s
LFM2.5-2.6B (bf16, conv-hybrid)[^lfm2]	3.49× math · 2.65× code · 2.24× chat	~97–152 tok/s
Muse-Glimmer-30B (8-bit, dense)[^muse]	3.27× math · 2.50× code · 2.22× chat	~18–26 tok/s
Qwen3.8-27B (4-bit, DFlash 2)[^q38]	3.17× code · 3.13× math · 2.49× chat	~37–47 tok/s
Qwen3-8B (8-bit)	2.83× math · 2.03× code · 1.71× chat	~50–83 tok/s
Qwen3.6-27B (8-bit)	2.67× math · 2.26× chat · 1.96× code	~16–22 tok/s
Ornith-1.0-9B (8-bit)	2.53× code · 2.48× math · 2.21× chat	~59–68 tok/s
Qwen3-4B (8-bit)	2.46× math · 1.78× chat · 1.72× code	~89–126 tok/s
Qwen3-14B (8-bit)	2.36× math · 2.11× code · 1.62× chat	~25–36 tok/s
Qwen3.6-35B-A3B (4-bit, MoE)[^moe]	1.67× math · 1.24× code · 1.05× chat	~91–145 tok/s
Nemotron-3.5-Lightning-30B-A3B (4-bit, MoE+Mamba)[^nemotron]	1.34× math · 1.27× code · 1.07× chat	~87–112 tok/s
Ternary-Bonsai-27B (2-bit)	1.13× code	~26–29 tok/s

Qwen3.8-27B's rows are its DFlash 2 drafter (incoai/Qwen3.8-27B-DFlash2), which --mode auto (the default, and what the Mac app uses) resolves for both quants; the DSpark heads for this target remain available with --mode dspark — the paired numbers are in DFlash 2 on Qwen3.8-27B.

The speed column is the measured range across the three benchmark contents at the row's best configuration — chat at the low end, code/math at the high end (decoding speed depends on what is being generated: copy- and structure-heavy content accepts longer drafts). Baselines and per-content splits are in Results at a glance.

This table is the set of pairs we have measured and vouch for, which is also exactly the auto-resolve registry — that is the only thing the registry is for. It is not the set of models that work: any DeepSpec-native drafter runs against any compatible target via --drafter, and any target at all gets drafter-free speculation via --mode auto. See Bring your own drafter. Per-model caveats and methodology live in the numbered footnotes at the end of this page — click a marker to jump.

Target precision: the quants shown are each model's measured best — ratios are non-monotone in bits and peak at 8-bit on current MLX (full Ornith sweep: 4-bit 1.38× · 8-bit 2.17× · bf16 1.54× on code; bf16 loses in both ratio and absolute speed because MLX's unquantized matmul pays a ~2× cost cliff at verify width 2). Details in Results at a glance.

Nothing in this table is hand-tuned per model, and none of it is pinned to this M4 Pro: with no --max-draft, mlx-dspark measures your machine's verify/drafter cost curves once (~5 s, cached per model + quant + mlx version) and derives the draft cap from them — an M1 or an M5 gets its own optimum, not the one these rows were measured at. --max-draft auto additionally adapts the cap per round while generating. See Tuning.

Copy-heavy code editing goes further: when the model re-emits or refactors code already in its context (the daily agent/assistant workload), match-scaled lookup drafts reach 4.5× on Gemma-12B (75 tok/s) and 3.6× on Ornith-9B (93 tok/s). Any model not listed still gets drafter-free lookup speculation via --mode auto.

The Mac app

Everything below is also available as a native Mac app — chat with saved sessions, a model manager that answers "will this fit my Mac?" before you download, live speculative-decoding telemetry (per-round acceptance, this machine's measured cost curves), a "This Mac" roofline view (your measured memory bandwidth, the plain-decode ceiling for the loaded model and how far above it speculation runs, macOS memory pressure and swap), a decoder Race with a checked lossless verdict, one-click coding-agent setup, and a menu-bar gauge with live tok/s and model memory.

brew tap ARahim3/mlx-dspark https://github.com/ARahim3/mlx-dspark
brew trust arahim3/mlx-dspark          # Homebrew 6+: third-party taps need explicit trust
brew install --cask mlx-dspark
xattr -dr com.apple.quarantine /Applications/mlx-dspark.app   # not notarized yet: clear
                                       # quarantine once, or use System Settings › Privacy &
                                       # Security › "Open Anyway" after the first launch

(On Homebrew ≤ 5, brew install --cask --no-quarantine mlx-dspark still works and replaces the trust/xattr steps — Homebrew 6 removed that flag.)

Or download the DMG from Releases (app-v* tags) and drag it to Applications. First launch sets up its own private engine runtime (no Homebrew Python, no venv of yours touched, ~2–4 min once) and keeps the engine on the latest release automatically; the app itself tells you when a newer app version exists (brew upgrade --cask mlx-dspark). pip install mlx-dspark stays engine-only — the app is not in the wheel, and the app never touches a pip-installed engine.

Install

pip install mlx-dspark          # or:  uv pip install mlx-dspark

Apple Silicon + Python ≥ 3.10; installs mlx ≥ 0.32.0 automatically (0.32's quantized-matmul kernels are what current speedup numbers are measured on). Model weights download from the Hugging Face cache on first use (none bundled). No server framework is pulled in — the API server is built on the standard library.

Known upstream incompatibilities (both handled): mlx-vlm 0.6.5 moved an internal rope-utils module, which crashed import mlx_dspark on fresh installs of mlx-dspark ≤ 0.4.2 — fixed in 0.4.3 (both module layouts supported), so upgrade mlx-dspark rather than pinning mlx-vlm. Separately, mlx-vlm 0.6.4 × transformers ≥ 5.12 breaks loading the gemma4 target with a misleading OSError: Can't load video processor … (#4, upstream Blaizzy/mlx-vlm#1578 — fixed in mlx-vlm 0.6.5). mlx-dspark ≥ 0.3.2 shims that one at load time, so any mlx-vlm ≥ 0.6.3 works; the shim self-retires on fixed releases, and mlx-dspark doctor reports when it is active.

Quickstart

mlx-dspark generate --model mlx-community/Qwen3-8B-8bit --prompt "Explain rainbows."
mlx-dspark serve    --model mlx-community/Qwen3-8B-8bit   # OpenAI + Anthropic API on :8080

That's the whole setup — swap in any target from the table above. Three things worth knowing, then you can stop reading:

• Pick a model, not a configuration. --model takes any HF repo or local path (exactly like mlx-lm); the matching drafter and that pair's measured-best settings resolve automatically. A model that isn't in the table still gets drafter-free speculation via --mode auto, or pass --drafter <repo> yourself.
• Don't set the draft cap. The speedups above were measured on one M4 Pro — your machine's optimum is different, so mlx-dspark measures your Mac on a pair's first run (~5 s, cached) and derives its own cap from those curves. An M1 and an M5 each get their own answer. --max-draft auto additionally adapts per round while generating (the safest choice if you only remember one flag); --max-draft N pins a value only if you've measured a better one.
• It's lossless by construction. The target verifies every drafted token, so the output is identical to running the target alone — every mode, every cap, only the speed changes. Want proof and your own numbers? mlx-dspark benchmark --model <repo> --trials 3 is the same reproducible sweep this README's tables come from (the Mac app's Race shows it live, with a token-by-token identical-output verdict).

Prefer clicking to typing? The Mac app wraps all of this — including the calibration and the model picker with "will it fit my Mac?" answered up front.

Serve an API (OpenAI and Anthropic on one port)

mlx-dspark serve --model mlx-community/Qwen3-8B-8bit        # → http://127.0.0.1:8080/v1
#   --max-batch 4   continuous batching: up to 4 concurrent requests share each forward
#                   (~2.5× aggregate; a finished request returns immediately, its slot
#                   admits the next one mid-flight)
#   --kv-bits 8     quantized KV cache (long-context bandwidth saver)
#   --mode auto|dspark|dflash|lookup|baseline   ·   --no-thinking   ·   --api-key KEY
#   --cpu-split 0   turn off CPU co-prefill (default: a calibrated share of each wide prefill
#                   matmul runs on the CPU's matrix units alongside the GPU — ~1.3–1.4× prefill)
#   --trust-remote-code   allow a checkpoint's own Python to be imported (refused by default)
#   --reasoning-effort low|medium|xhigh   default reasoning depth on models that support it
#                   (Qwen3.8-class; /health reports support, requests can override)
#   --no-model      start instantly with nothing loaded; POST /admin/load loads later,
#                   POST /admin/unload frees the model again (port survives both).
#                   A first-time load reports download progress in /health and is
#                   cancellable (POST /admin/load/cancel — partials resume by default);
#                   /admin/load also takes per-swap mode / max_draft / lookup_drafts /
#                   confidence_threshold / context_window / kv_bits / memory_guard /
#                   enable_thinking / reasoning_effort overrides (the last two are the thinking
#                   default API clients without a reasoning toggle inherit; sticky across swaps)
#   --no-memory-guard   keep caches even when macOS reports memory pressure (default: on WARN
#                   the engine returns its retained buffers + prefix-cache rungs; conversations
#                   stay cached). --no-warmup skips the on-load warmup generation.

--mode auto picks the best available speculation for any target (a known DSpark drafter → else DFlash → else drafter-free n-gram lookup), so any repo serves with some speedup and no extra flags.

Then point any OpenAI client at it — the speculative speedup is transparent:

from openai import OpenAI
client = OpenAI(base_url="http://127.0.0.1:8080/v1", api_key="not-needed")
print(client.chat.completions.create(
    model="Qwen3-8B-8bit",
    messages=[{"role": "user", "content": "Explain rainbows briefly."}],
).choices[0].message.content)

Works with any OpenAI-compatible client — the same frontends people point at llama.cpp's llama-server or Ollama work here by switching one setting: set the OpenAI base URL to http://127.0.0.1:8080/v1 (API key: anything). That covers Open WebUI, SillyTavern, Continue/Cline, LibreChat, Raycast AI, and the rest of that ecosystem. (There is no GGUF interop — mlx-dspark runs MLX weights natively — but the HTTP surface is shared, which is the part those tools actually talk to.)

The server speaks the OpenAI API: POST /v1/chat/completions (streaming and non-streaming, multi-turn), POST /v1/completions, GET /v1/models, GET /health (incl. a warnings list — memory pressure, context-window RAM notes), GET /metrics, and GET /machine (this Mac's measured bandwidth, the loaded model's bytes per token, the plain-decode ceiling and a verdict). It supports temperature, top_p, top_k, max_tokens, stop, seed, presence_penalty / frequency_penalty, logprobs / top_logprobs, tool calling (tools / tool_calls), a per-request thinking toggle (enable_thinking), and per-request reasoning_effort on models whose template supports it (Qwen3.8-class; GET /health reports supports_reasoning_effort). Each response carries an x_mlx_dspark block — accept length, decode and end-to-end tok/s, time to first token, prompt tokens served from the prefix cache, and the decode rate as a multiple of this Mac's single-stream roofline — so the spec-decode gain is visible and "why was this turn fast/slow" has an answer. Continuous batching (--max-batch N) serves concurrent requests in one batched forward for ~2.5× aggregate throughput (see Concurrent throughput); prefix caching (on by default) reuses the conversation prefix so multi-turn chat and agents don't re-prefill each turn — measured on an ~8k-token context: first token in ~62 s cold vs 0.2–1 s on cached turns, hybrid targets included (see Prefix caching).

Security

mlx-dspark serve is a local, single-user server. It binds 127.0.0.1 by default; if you expose it further, use --api-key — then every route except GET /health requires the key (Authorization: Bearer … or x-api-key). A checkpoint that ships its own Python (config.json: model_file, auto_map custom classes) is refused by default, because mlx-lm/transformers would otherwise import and run it; --trust-remote-code (or MLX_DSPARK_TRUST_REMOTE_CODE=1) opts the whole process in, and there is no per-request override on /admin/load. See SECURITY.md for the threat model and how to report.

Use it from Claude Code

The same server also speaks Anthropic's Messages API, which is the dialect Claude Code talks — so Claude Code can run entirely on a model on your Mac. Start the server, then in another terminal:

mlx-dspark serve --model mlx-community/Qwen3-8B-8bit --no-thinking   # terminal 1
mlx-dspark claude                                                    # terminal 2

That's it — mlx-dspark claude finds the running server, points Claude Code at it, and hands over the terminal. Anything after -- goes to claude (mlx-dspark claude -- --continue).

It changes nothing outside that one process. The configuration is passed as the launched process's environment and nowhere else: no shell profile edited, no settings.json written, no login replaced. Your other Claude Code sessions — open now or started later — keep their normal account, model, and endpoint, and this one reverts the moment it exits. Your claude.ai login stays saved and untouched; Claude Code notes at startup that a credential variable takes precedence over it, which is just that notice. To wire it up yourself instead, mlx-dspark claude --print-env prints the shell exports and --print-settings prints a project-scoped .claude/settings.local.json block.

Endpoints: POST /v1/messages (streaming and non-streaming), POST /v1/messages/count_tokens. Tool calling, multi-turn tool_use/tool_result history, stop_sequences, and system prompts are translated to whatever the loaded model's own chat template expects — including each family's tool syntax (Hermes JSON, Gemma-4, and the XML <function=> form) — and a reasoning model's <think> or <|channel>thought output is lifted into proper Anthropic thinking blocks rather than leaking as prose.

Use it from Codex

The server also speaks OpenAI's Responses API, the dialect Codex requires once its provider is configured with wire_api = "responses" (Codex dropped Chat Completions support). Point a model_providers entry in ~/.codex/config.toml at the running server's /v1 base URL and Codex talks to whatever's loaded, same as any other OpenAI-compatible client.

Endpoints: POST /v1/responses (streaming and non-streaming). input accepts either a bare string or the structured item list (message, function_call, function_call_output), tools are accepted in the Responses API's flat shape, and multi-turn tool use round-trips through the same tool-syntax translation the Chat Completions and Anthropic dialects already use. This is a stateless Responses implementation, like Ollama's — no previous_response_id / server-side conversation store, since a client resubmits its own history each request (Codex does this already, so nothing is lost).

Measured — each of these ran a real Claude Code session that read a buggy file and fixed it with the Edit tool (M4 Pro, --no-thinking, identical task):

target	accept length	prefix cache	wall clock
mlx-community/Qwen3-8B-8bit	3.01	on — 2 of 3 requests, ~26k tokens reused	~2:20
mlx-community/Ornith-1.0-9B-8bit	5.07	off — hybrid target, recurrent state can't be reused	~4:10
mlx-community/gemma-4-12B-it-8bit	3.68	on, but its sliding window wraps at this prompt size	~4:10

The ranking is the point: Claude Code sends ~18–26k tokens of system prompt and tool schemas on every request, so prefill dominates wall clock and the target that reuses it wins — Qwen3-8B is nearly 2× faster here despite the lowest accept length of the three. Ornith's 5.07 is the highest acceptance this project has measured anywhere (tool-call JSON is very predictable), but it spends the win on re-prefilling. Choose for prefix-cache compatibility first, drafter quality second.

That table was measured on 0.6.0. 0.7.0 changes its premise for the hybrid row: checkpoint prefix caching (see Prefix caching) gives Ornith reuse under --no-thinking, which is the setting this table used — and 0.10.1 drops that condition: stable-boundary snapshots make checkpoint reuse fire with thinking on and on Qwen3.6/3.8-class templates too, so every hybrid row's wall clock should improve. Not yet re-measured, so the numbers above stand as recorded rather than being quietly restated.

Other agent clients

The Anthropic endpoint isn't Claude Code–specific. pi works out of the box against either API — add a custom provider to ~/.pi/agent/models.json:

{ "providers": { "mlx-dspark": {
    "baseUrl": "http://127.0.0.1:8080", "api": "anthropic-messages", "apiKey": "mlx-dspark",
    "models": [{ "id": "Qwen3-8B-8bit", "contextWindow": 40960, "maxTokens": 8192 }] } } }

then pi --provider mlx-dspark --model Qwen3-8B-8bit. (Swap "api" for "openai-completions" and "baseUrl" for http://127.0.0.1:8080/v1 to use the OpenAI endpoint instead — both work.)

pi is markedly better suited to a local model than Claude Code, for one reason: its system prompt is ~1.5k tokens against Claude Code's ~18–26k. Since prefill dominates, that lands directly on the clock — the same one-bug fix takes ~6 s instead of ~2:20, and a four-tool task (read, two edits, read, write) finishes in 8.5 s at 24 tok/s on Qwen3-8B. Ornith-1.0-9B runs the same task in 18.8 s. Gemma-4-12B doesn't converge on pi's tool protocol (on either endpoint, so it's the model, not the server) — use it with Claude Code instead.

Practical notes for a local model:

Use a tool-calling model	These are tool-use agents first. Qwen3-8B and up handle it; smaller models flail.
Agent choice moves the clock more than model choice	The client's prompt size is the dominant cost on a local model — a lean agent like pi is an order of magnitude faster on the same hardware and the same task.
--no-thinking is a speed knob, not a requirement	Leaving it off works fine — reasoning is streamed as proper thinking blocks either way. It just costs: on Qwen3-8B the same Claude Code task ran 3:17 and 2762 output tokens with thinking vs ~2:20 and 169 without, since the model thinks before every tool call. A client sending thinking: {"type": "disabled"} gets the same effect per-request. Note it's a no-op on Gemma-4, whose template doesn't think by default.
Leave prefix caching on	It is doing most of the work (see the table). The first request of a cold server is the slow one; since 0.10.1 a fresh session over a system prompt the server has already seen partially reuses it (rungs — see Prefix caching).
Context	An over-long request is refused with the wording Claude Code recognises as a context limit, so it compacts and retries instead of dying. --context-window N lowers the bar deliberately (e.g. to keep the KV cache inside your RAM budget).
Streams stay alive, and disconnects actually stop generation (v0.12.3)	Keep-alive frames flow every 15 s through stretches with nothing on the wire (long prefills; tool-call responses are buffered until complete), so agent clients don't idle-timeout mid-request. And if a client does vanish, generation stops at the next round instead of running to max_tokens while later requests queue behind it — the "server stalls but /health is fine" failure mode.

One-shot generation (CLI)

# downloads the drafter + instruct target on first run
mlx-dspark generate --model mlx-community/Qwen3-4B-8bit --prompt "Explain how rainbows form."

# baseline (plain target) vs dspark — same output, faster (record each, stack for a demo)
mlx-dspark generate --model mlx-community/Qwen3-4B-8bit --mode baseline --prompt "..." --max-new-tokens 400
mlx-dspark generate --model mlx-community/Qwen3-4B-8bit --mode dspark   --prompt "..." --max-new-tokens 400

# z-lab DFlash drafter (--max-draft 0 = full 16-block, its native operating point)
mlx-dspark generate --model mlx-community/gemma-4-12B-it-8bit --mode dflash --max-draft 0 --prompt "Write a binary search."

# sampled (not greedy) — lossless w.r.t. the target at temperature T (dspark and dflash)
mlx-dspark generate --model mlx-community/Qwen3-4B-8bit --prompt "Write a short poem." --temperature 1.0 --top-p 0.95 --seed 0

python -m mlx_dspark … works too, and the old flat --prompt … form still maps to generate.

Python

from mlx_dspark import load_pair, speculative_generate

target, tok, drafter, cfg = load_pair("mlx-community/Qwen3-8B-8bit")   # drafter auto-resolved
res = speculative_generate(target, tok, drafter, "Explain how rainbows form.")
print(res.text, res.mean_accept_len, res.tokens_per_sec)

from mlx_dspark import load_dflash_pair, dflash_generate   # z-lab DFlash instead

target, tok, drafter, cfg = load_dflash_pair("mlx-community/gemma-4-12B-it-8bit")
res = dflash_generate(target, tok, drafter, "Write a binary search in Python.")  # max_draft_tokens=None = full block
print(res.text, res.mean_accept_len, res.tokens_per_sec)

Models

Pass any target repo/path to --model; the matched drafter auto-resolves for the targets below (quantization-agnostic — a -4bit / -8bit / -bf16 of the same model resolves the same drafter). For anything else, add --drafter <repo>. Run mlx-dspark models to print this table.

target (--model)	DSpark drafter (--mode dspark)	DFlash drafter (--mode dflash)	peak RAM	+ cache at 128k ctx
mlx-community/Qwen3-4B-8bit	deepseek-ai/dspark_qwen3_4b_block7	z-lab/Qwen3-4B-DFlash-b16	~8 GB	not measured yet
mlx-community/Qwen3-8B-8bit	deepseek-ai/dspark_qwen3_8b_block7	z-lab/Qwen3-8B-DFlash-b16	~11 GB	not measured yet
mlx-community/gemma-4-12B-it-8bit	deepseek-ai/dspark_gemma4_12b_block7	z-lab/gemma4-12B-it-DFlash	~15 GB	not measured (partly window-bounded)
prism-ml/Ternary-Bonsai-27B-mlx-2bit	Rahim/Ternary-Bonsai-27B-dspark	—	~12 GB	not measured yet
mlx-community/Qwen3.6-27B-8bit	satgeze/Qwen3.6-27B-DSpark (community)	—	~32 GB	~11 GB (est., same arch as Qwen3.8)
mlx-community/Qwen3.8-27B-4bit[^q38]	RedHatAI/Qwen3.8-27B-speculator.dspark (Red Hat, speculators)	incoai/Qwen3.8-27B-DFlash2 (default)	~18 GB	~11 GB measured (~23 GB at full 256k)
mlx-community/Qwen3.8-27B-8bit[^q38]	RedHatAI/Qwen3.8-27B-speculator.dspark (Red Hat, speculators)	incoai/Qwen3.8-27B-DFlash2 (default)	~29 GB	~11 GB measured (~23 GB at full 256k)
mlx-community/Ornith-1.0-9B-8bit	stanleyphoong/Ornith-1.0-9B-DSpark (community)	—	~13 GB	not measured yet
mlx-community/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-4bit	mlx-community/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-DSpark-bf16 (NVIDIA head, MLX)	—	~20 GB	not measured yet
mlx-community/Muse-Glimmer-30B-4bit	DaoCloud/Muse-Glimmer-30B-DSpark (community, DFlash-lineage)	—	~26 GB (4-bit) / ~40 GB (8-bit[^muse])	not measured yet
MercuriusDream/Nanbeige4.2-3B-mlx-bf16[^nanbeige]	Nanbeige/Nanbeige4.2-3B-DSpark (official, SpecForge)	—	~10 GB	not measured yet
mlx-community/MiniCPM5-2B-bf16[^minicpm5]	openbmb/MiniCPM5-2B-DSpark (official, DeepSpec)	—	~6 GB	not measured yet

Peak RAM is measured on an M4 Pro (8-bit target + 4-bit drafter + KV cache) at chat-length context; add headroom for macOS. Cache at 128k ctx is what a long-context session adds on top: the target's attention KV plus the drafter's context cache grow linearly with every token of context. The Qwen3.8-27B figure is measured on-device (0.086 GB per 1k tokens — 64 KB/token target KV + 20 KB/token drafter ctx, matching the architecture math exactly: 16 full-attention layers × 4 KV heads × 256 head-dim in bf16; the 48 linear-attention layers hold fixed-size state, which is why a 27B "256k-context" model is usable at long context on a Mac at all) and is the same for both quants — the cache is bf16 regardless of weight bits. So Qwen3.8-27B-4bit at 128k needs ~26 GB total, the 8-bit ~40 GB; --kv-bits 8 roughly halves the attention-KV share. Cap it with --context-window (or the context_window override on /admin/load) when RAM is the constraint — requests past the cap get the "prompt is too long" error agent clients auto-compact on. Rows marked (community) use drafters published by the community, not by DeepSeek — quality varies more than with the official checkpoints, and it shows up directly as acceptance length (= your speedup). The Ornith drafter is the strong case: rigorously qualified by its author (17/17 gates, 95% of the DSpark paper's reference acceptance) and it produces the best chat speedups in this table. The Qwen3.6-27B drafter is the other strong one: block-15, trained against the bf16 target and warm-started from z-lab's DFlash head for it. Its row is measured on the 8-bit target; a -4bit target resolves the same drafter and, by the pattern every other model here follows, should trade ratio for absolute tok/s — that combination is not measured. A 4-bit target (--model …-it-4bit) roughly halves the target's share (fits smaller Macs). Use the matched instruct target the drafter was trained against — a base model drops acceptance sharply. The legacy --family qwen3|gemma4 flags still work but are deprecated in favor of --model.

Already use LM Studio? Its MLX downloads are reused automatically: pass the same publisher/model id LM Studio shows (e.g. --model lmstudio-community/Qwen3-8B-MLX-8bit) and the engine loads it straight from LM Studio's folder (~/.lmstudio/models) instead of re-downloading — they also appear in mlx-dspark models and the app's "On this Mac" list. Only MLX downloads work this way; LM Studio's GGUF files are a different format our loaders don't read (grab the mlx-community version of the same model instead). The reverse direction — using mlx-dspark as an engine inside LM Studio's chat window — isn't possible (LM Studio only runs its own bundled engines), but the server speaks the standard OpenAI API, so any client that takes a custom OpenAI endpoint can point at it.

Serving other devices? mlx-dspark serve --host 0.0.0.0 --api-key <key> listens on every interface; with a key set, every route needs Authorization: Bearer <key> (or x-api-key). The Mac app has the same two switches under Settings → Local server.

Keep models somewhere else? (an external drive, ~/models, a NAS…) Point MLX_DSPARK_MODEL_DIRS at those folders (:-separated) and they're searched before anything is downloaded — as publisher/model trees, publisher_model dirs, or bare model dirs (MLX checkpoints only: config.json + .safetensors). They show up in mlx-dspark models and the app's list too. Your Hugging Face cache is honoured wherever HF_HOME puts it, and a plain path (--model ~/models/Qwen3.8-27B-4bit) always works. mlx-dspark doctor --json lists every folder the engine looks in, in order.

--drafter lets you run any other matched z-lab / DeepSpec checkpoint with no code change and no registry entry — the registry only saves you from having to name the drafter:

# a pair we have measured -> the drafter auto-resolves
mlx-dspark generate --model mlx-community/Qwen3-14B-8bit --prompt "Explain how rainbows form."

# anything else -> name the drafter yourself; identical machinery from here on
mlx-dspark generate --model mlx-community/Qwen3-32B-8bit \
  --drafter deepseek-ai/dspark_qwen3_32b_block7 --prompt "Explain how rainbows form."

Bring your own drafter — what runs and what doesn't

New DSpark/DFlash drafters keep landing on HF in several different packagings; here is the honest compatibility contract (loaders refuse incompatible checkpoints with an error naming the reason, never a silent mis-load):

checkpoint style	example	status
DeepSpec-native standalone drafter (qwen3/gemma4 backbone, any size/quant)	deepseek-ai/dspark_qwen3_32b_block7	✅ runs via --drafter — no registry entry needed (4B/8B/14B/gemma-12B are measured and registered, so they need no flag; larger sizes should run — reports welcome)
z-lab DFlash adapter for a qwen3/gemma4-family target	z-lab/Qwen3-8B-DFlash-b16	✅ runs via --drafter (the loop is read from the checkpoint; --mode dflash also works)
DFlash 2 (Inco AI: candidate selector + dynamic convs)	incoai/Qwen3.8-27B-DFlash2	✅ runs via --drafter; the Qwen3.8-27B heads are measured and registered (auto-resolve, and --mode auto picks them as that target's best)
DSpine (Tencent: a block-causal DFlash backbone with adjacent causal injection + predecessor-conditioned decoding, paper, Apache-2.0)	tencent/Qwen3.8-27B-DSpine-b16	✅ mlx-dspark generate --model mlx-community/Qwen3.8-27B-4bit --drafter tencent/Qwen3.8-27B-DSpine-b16 (same for serve); trees and --max-draft auto included. On mlx-community/Qwen3.8-27B-4bit (M4 Pro) 2.70× at the measured cap 7 (3.04× code / 2.86× math / 2.22× chat, accept 5.08, 38.4 tok/s) — its full 15-draft block accepts 7.15 but a 9+-row verify costs 1.65× more on this GPU (2.23× at cap 15), so DFlash 2 (2.93×) stays the registered default
PrismML dspark GGUF (Bonsai-27B)	prism-ml/Ternary-Bonsai-27B-gguf → *-dspark-bf16.gguf	✅ pre-converted repacks auto-resolve (Rahim/*-dspark); any future GGUF-only drop runs via --drafter gguf:<repo>/<file>.gguf (converted locally, once)
vLLM "speculators" format (dspark algorithm)	makora-ai/gemma4-26b-a4b-dspark, mgoin/Qwen3-8B-speculator.dspark	✅ runs via --drafter — the config schema is translated on load (the tensor names are already DeepSpec's). Includes EAGLE-3-style reduced draft vocabularies (draft_vocab_size + a d2t table). Other speculators algorithms (eagle/eagle3) are refused by name. makora-ai/gemma4-26b-a4b-dspark (Google's 26B/4B-active MoE) measures 1.27× on mlx-community/gemma-4-26b-a4b-it-8bit (--max-draft 2 --no-lookup-drafts: 1.38× code / 1.37× math / 1.06× chat, 46.9→59.5 tok/s) — not registered for auto-resolution while the ratio is under review
LiquidAI LFM2.5-DSpark (Lfm2DSparkDraftModel, for the lfm2 / lfm2_moe conv+attention hybrid targets)	LiquidAI/LFM2.5-2.6B-DSpark	✅ measured and registered (auto-resolves, any quant). A plain qwen3 backbone in a fifth packaging (taps nested in dflash_config, no projector_type), reusing the target's tied embed+head with interleaved rope. LFM2 is the first conv-recurrence target (a new shortconv capture/rollback). The 2.6B/1.2B are strong wins; the 8B-A1B MoE is a modest win at bf16 (~1.3×, net loss at 8bit — the MoE verify ceiling)
Full model with embedded drafter	deepseek-ai/DeepSeek-V4-Pro-DSpark (893 GB, MLA+MoE)	❌ different architecture & packaging — out of scope for consumer Macs
DFlash+Markov community hybrids	Hikari07jp/DSpark-Gemma-4-31B-draft	❌ hybrid head — not yet

Targets: any dense mlx-lm text model routes automatically; a one-time load probe verifies the hidden-state tap reproduces the model's own forward and fails loudly if the family needs bespoke support (drafter-free --mode lookup / --mode auto still work with any target). If you run a pair we haven't measured, mlx-dspark benchmark --json produces a device-stamped result we can fold into the table — please share it.

Very small targets are not worth a drafter. Speculation buys time in proportion to what one target step costs, so below ~4B there is little to win. Measured, so you don't spend the download: satgeze/Qwen3.5-0.8B-DSpark on Qwen3.5-0.8B-8bit runs correctly and losslessly but comes out at 0.96× — the target is already 215 tok/s, and a draft round costs nearly as much as the step it skips. Run these models plain.

PrismML Bonsai 27B (ternary / 1-bit Qwen3.6-27B)

Bonsai 27B is PrismML's 1.7-bit ternary (and 1-bit) rebuild of Qwen3.6-27B — a full 27B-class reasoning model in ~8 GB. It ships with a DSpark drafter that PrismML publishes GGUF-only and, per their own docs, accelerates their CUDA path but "not Macs yet". mlx-dspark runs it on a Mac:

mlx-dspark generate --model prism-ml/Ternary-Bonsai-27B-mlx-2bit \
  --max-draft auto --prompt "Implement binary search in Python."
# first run: downloads the target (8.5 GB) + the matched drafter (6.8 GB bf16 safetensors —
# our 1:1 repack of PrismML's GGUF-only drafter, quantized to 4-bit at load)

Measured on an M4 Pro 48 GB (greedy, warm): baseline ~25.5 tok/s; ~1.15× on code/structured content (acceptance ~2.9/round at cap 2), ~break-even on open chat. Output is lossless — byte-identical to plain greedy decoding. Bonsai's backbone is hybrid linear attention (48 of 64 layers carry recurrent state, which can't be rolled back like a KV cache), so mlx-dspark records each verify round's recurrence inputs and, on a partial accept, rebuilds the state at the exact accept point (bit-exact, a few ms) — rejected drafts cost about as little as a dense KV trim. As far as we know this is the first working speculative decoding for this model family on Apple Silicon.

Two honest caveats. The ceiling is the 2-bit quantization itself: extra verify rows on a 2-bit model are compute-bound (they cost the same as on a 4-bit model, measured), while its plain step is very fast — so chat-level acceptance hovers at break-even instead of the 1.6–2.1× the 8-bit presets reach. --max-draft auto stays the recommended setting: it picks the cap from this machine's measured curves + live acceptance and can still park speculation (plain pipelined steps + probe rounds) on content where it would lose. And it requires mlx ≥ 0.32.0 (older mlx lacks the multi-row 2-bit matmul path that makes verification affordable at all). Prefix caching works here too via checkpoint mode — the recurrent state can't trim, so it's snapshotted at boundaries instead (see Prefix caching) — and baseline batching handles hybrids since 0.7.0; only batched speculative decoding stays dense-only. Baseline/--mode lookup also work for any other qwen3_5 (Qwen3.5/3.6-family) checkpoint.

The 1-bit Bonsai-27B-mlx-1bit pack runs on stock MLX as of mlx-vlm 0.6.5 (which ships a Python-hosted 1-bit kernel; stock mx.quantize still has no 1-bit mode) — but speculative decoding measures a net loss on it: that kernel re-reads the full weight stream once per verified token, so verify cost is linear in draft length, and dspark lands at 0.71–0.77× baseline at every cap (M4 Pro, healthy acceptance, losslessness intact). mlx-dspark therefore keeps the pack unintegrated — plain generation via mlx-vlm ≥ 0.6.5 is the right tool for it (~35 tok/s on an M4 Pro vs ~25 for the ternary), and load_target refuses it with a pointer saying so. The ternary 2-bit variant remains the speculative-decoding operating point.

How it works

• DSpark — a parallel backbone (5 layers) consumes the target's hidden states (EAGLE3-style) and proposes a 7-token block at once; a rank-256 Markov head adds a cheap previous-token correction that kills "suffix decay"; a confidence head scores each position (optional adaptive block length).
• DFlash (--mode dflash) — a block-diffusion drafter that denoises a whole 16-token block in one parallel pass and reuses the target's own embed/lm-head. Different trade-offs (see below).
• The target verifies every token, so output is greedy-correct by construction (identical to plain decoding up to floating-point tie-breaking). --temperature > 0 switches to lossless speculative sampling — an exact sample from the target at temperature T (with --top-p / --top-k).

The drafter loads 1:1 from the HF checkpoint and is 4-bit quantized by default (cheap to run each round; quantization doesn't change acceptance — that's set by the drafter↔target match).

Which target should I use?

Mode first, because it's the short one: --mode auto picks each pair's measured best. That is DSpark everywhere except Qwen3.8-27B, where the DFlash 2 drafter wins on both quants (same verify width, higher acceptance — receipts in DSpark vs DFlash). The original-DFlash verdict is version-dependent and worth knowing about: on mlx 0.31, verify cost rose steeply with the number of tokens verified, which made DFlash's full 16-block the winner on Gemma-12B code/math; 0.32's kernels made narrow multi-row verify disproportionately cheaper and flipped it to DSpark — and v0.12.0's small-M verify kernel flattened the curve enough that DFlash 2's acceptance-at-fixed-width wins it back on Qwen3.8. If your mlx/hardware differs, --max-draft auto re-measures the curves on your machine, and mlx-dspark benchmark settles it empirically.

Target, by the Mac you have (all numbers from the table above; speedups and caps are this M4 Pro's — yours are derived fresh on first run):

• ~48 GB — Qwen3.8-27B-8bit (best ratio in the project: 3.66× with its DFlash 2 drafter, 4.22× on math — --mode auto picks it; Red Hat's DSpark head ties it, 3.79× paired, with ~3× on chat) or Muse-Glimmer-30B-8bit (a dense 30B, 2.5× at 8-bit). Both ~29–40 GB resident.
• ~24–36 GB — Qwen3.8-27B-4bit (27B quality in ~18 GB at ~31–45 tok/s, the fastest decode among the 27B-class targets — DFlash 2 via --mode auto, no cap flag needed), gemma-4-12B-it-8bit (big ratio and real speed: 3.25×, ~48–78 tok/s), or Qwen3.6-27B-8bit.
• ~8 GB — MiniCPM5-2B-bf16 (3.16× mean, 4.21× on math, ~107–213 tok/s in ~6 GB resident — the best small-Mac pick, and a tool-calling model).
• ~16 GB — Ornith-1.0-9B-8bit (2.4× at ~59–68 tok/s, the mid-size sweet spot), Qwen3-8B-8bit, or Qwen3-4B-8bit (~89–126 tok/s, fits ~8 GB).
• Raw tokens per second above all — the MoEs: Qwen3.6-35B-A3B-4bit (~91–145 tok/s). Their speedup ratio is modest for a structural reason (sparse targets are already fast — see the MoE note under Results), but nothing here decodes faster.
• Agents (Claude Code, pi, …) — prefill dominates agent turns, so prefix-cache reuse matters more than acceptance: Qwen3-8B-8bit wins wall-clock in the Claude Code comparison, and since v0.10.1 the hybrid targets (Qwen3.8/Qwen3.6/Ornith) cache too. Use --no-thinking and an 8B+ tool-calling model.

For target precision: 8-bit is the sweet spot (best acceptance + quality); 4-bit gives the highest absolute throughput and fits smaller Macs but a smaller speedup ratio; bf16 is slower on M-series (verify dominates). The drafter stays 4-bit either way. Full numbers and the reasoning are in Benchmarks & deep dive.

On copying caps and flags from these tables. Every cap and flag shown is this M4 Pro's derived optimum for that exact quant — a starting point, not a setting to paste in. Two things that trip people up:

• Don't copy the cap. With no --max-draft, mlx-dspark derives it from your machine's own curves (per model + quant + mlx version). A faster, higher-bandwidth machine can land lower, not higher — an M5 Max may pick cap 2 for the 8-bit 27B where this M4 Pro picks 7, because cheaper verify shifts the optimum down, and forcing 7 there is a small net loss. Prefer no flag, or --max-draft auto.
• --confidence-threshold is drafter-specific, not just quant-specific. It only pays when the drafter leaves acceptance headroom for early truncation to recover. On Qwen3.8-27B-4bit the RadixArk head did, and gained from --max-draft 7 --confidence-threshold 0.3, while the DimInfer head accepted too well (3.3–5.3/round) for it to help — plain cap 7, no confidence flag was its best, and 0.5 measured worse. (Both have since been replaced as the DSpark head for this target; the current one, Red Hat's, runs at its derived cap 7 with no flag.) Each row's per-content badge is its own; don't paste a flag across quants or drafters.

Results at a glance

DSpark vs plain greedy decoding of the same model, each at its own derived cap (M4 Pro 48 GB, warm, 4-bit drafter, 200 tokens, end-to-end tok/s). Re-measured 2026-09-27 on mlx 0.32.2 with the current kernels (mlx-dspark benchmark --trials 3, zero-flag derived caps) for every row whose models were on this machine: Gemma-4 12B, Qwen3-4B/8B, Qwen3.8-27B (both quants), LFM2.5, MiniCPM5 and Nanbeige. The other rows keep their earlier stamps (mlx 0.32.0–0.32.1, 2026-07-22 to 2026-08-12) and predate the 2026-09 verify kernel — the 8-bit ones among them would likely move the way Qwen3-4B/8B and Gemma-4 did. The Qwen3.8-27B rows are its DSpark head since 2026-09-28, Red Hat's (from the paired head-to-head in the next section — same prompts and semantics, High Power mode); that target's default is DFlash 2. Every number is a median of 3 runs over the harness's three prompts, and the tok/s columns are the mean across them. Reproduce any row with that command.

The cap column is the headline change, twice over. mlx 0.32's quantized-matmul kernels widened the cheap verify region to width 5 for 8-bit weights — so the old hard-coded cap=2 was leaving 10–35% on the table for every 8-bit target. Then v0.12.0's small-M verify kernel removed the cliff that sat after that region (stock quantized_matmul re-pays the whole weight read per row at 2–8 rows — see the ceiling section), which is how the Qwen3.8-27B rows reach cap 7; its 2026-09 successor (mlx-dspark's own skinny_qmm, verify widths 5–16 at 4-bit and 6–16 at 8-bit) flattened the curve for the other 8-bit targets too, and moved Gemma-4 12B and Qwen3-4B/8B from cap 4 to 7 on its own — Gemma-4 12B went 2.82× → 3.25× (math 3.12× → 4.24×) in the same run. The cap is never hard-coded: it is derived from each machine+model+quant's measured cost curves (see Tuning), which is why Bonsai sits at 2, the older 8-bit stamps at 4, and every re-measured row at 7 — same code, different measured curves.

target	cap	accept len	baseline	mlx-dspark	speedup	chat / code / math
Qwen3.8-27B (8-bit, hybrid)[^q38]	7	5.55	8.2 tok/s	31.1 tok/s	3.79×	3.03× / 4.01× / 4.32×
LFM2.5-1.2B (bf16, conv-hybrid)[^lfm2]	7	5.28	99.6 tok/s	333.5 tok/s	3.35×	2.40× / 3.83× / 3.82×
Gemma-4 12B	7	4.98	18.4 tok/s	59.8 tok/s	3.25×	2.62× / 2.90× / 4.24×
MiniCPM5-2B (bf16, dense)[^minicpm5]	7	4.87	50.6 tok/s	159.7 tok/s	3.16×	2.12× / 3.15× / 4.21×
Nanbeige4.2-3B (bf16, looped)[^nanbeige]	7	3.95	17.3 tok/s	50.6 tok/s	2.92×	1.98× / 2.50× / 4.31×
LFM2.5-2.6B (bf16, conv-hybrid)[^lfm2]	7	4.00	43.6 tok/s	121.7 tok/s	2.79×	2.24× / 2.65× / 3.49×
Qwen3.8-27B (4-bit, hybrid)[^q38]	7	4.83	14.7 tok/s	38.6 tok/s	2.63×	2.25× / 2.81× / 2.83×
Muse-Glimmer-30B (8-bit, dense)[^muse]	4	3.31	8.2 tok/s	20.2 tok/s	2.47×	1.97× / 2.45× / 2.99×
Ornith-1.0-9B (hybrid)[^community]	4	3.64	26.7 tok/s	64.2 tok/s	2.40×	2.21× / 2.53× / 2.48×
Qwen3.6-27B (8-bit, hybrid)[^community][^q27]	4	3.15	8.4 tok/s	19.2 tok/s	2.29×	2.26× / 1.96× / 2.67×
Qwen3-8B	7	3.37	29.5 tok/s	64.5 tok/s	2.19×	1.71× / 2.03× / 2.83×
Qwen3-14B[^qwen14b]	4	2.87	15.3 tok/s	31.0 tok/s	2.03×	1.62× / 2.11× / 2.36×
Qwen3-4B	7	3.23	51.7 tok/s	102.5 tok/s	1.98×	1.78× / 1.72× / 2.46×
Qwen3.6-35B-A3B (4-bit, MoE, hybrid)[^community][^moe]	conf	4.72	86.9 tok/s	114.5 tok/s	1.32×	1.05× / 1.24× / 1.67×
Nemotron-3.5-Lightning-30B-A3B (4-bit, MoE+Mamba, hybrid)[^nemotron]	3	3.28	91.4 tok/s	100.9 tok/s	1.10×	0.95× / 1.23× / 1.13×
Ternary-Bonsai-27B (2-bit, hybrid)	2	2.60	25.4 tok/s	27.2 tok/s	1.07×	1.01× / 1.13× / 1.07×

DFlash 2 on Qwen3.8-27B — the project best (2026-08-19)

The table above is DSpark-mode; on Qwen3.8-27B the DFlash 2 drafter (incoai/Qwen3.8-27B-DFlash2) is the default — --mode auto, and the Mac app, resolve it for both quants. Both drafters run at the identical verify width (cap 7); since 2026-09-28 the DSpark head is Red Hat's (RedHatAI/Qwen3.8-27B-speculator.dspark), and it closes the gap the earlier DSpark heads left: DFlash 2 still leads at 4-bit, the two tie at 8-bit. Paired in one process (arms interleaved per prompt, the benchmark's prompts and semantics: 200 tok, end-to-end tok/s, 3-trial medians), High Power mode, current kernels:

8-bit (baseline 8.2 tok/s):

method (cap 7)	mean	chat	code	math	accept	tok/s
DSpark (RedHatAI)	3.79×	3.03×	4.01×	4.32×	5.55	31.1
DFlash 2	3.74×	3.07×	3.93×	4.23×	5.42	30.8

4-bit (baseline 14.7 tok/s, ~18 GB):

method (cap 7)	mean	chat	code	math	accept	tok/s
DSpark (RedHatAI, tree verify)	2.80×	2.48×	2.97×	2.94×	5.08	41.1
DFlash 2 (tree verify)	2.93×	2.49×	3.17×	3.13×	5.37	43.1

Both 4-bit rows were re-measured on 2026-11-08 with v0.20.4's tree verify, drafter overlap and deferred commit (same harness, High Power mode): DFlash 2 up from 2.72× / 40.0 tok/s, the Red Hat DSpark head up from 2.63× / 38.6 tok/s on 2026-09-28. The 8-bit table is still the 2026-09-28 measurement and predates those gains.

DFlash 2 adds a candidate path selector (top-16 target-head candidates per slot, a bilinear lattice walked from the verified anchor) and per-sublayer dynamic convolutions to the DFlash backbone — acceptance rises ~+1.1–1.4 tokens without widening the verify, which is exactly the kind of gain that converts on Apple Silicon (the small-M kernel made verify width 8 nearly flat on these targets). Greedy-lossless like everything here (the only divergences from single-row greedy are fp ties at margins 0.0–0.125), sampled decoding stays lossless through the selector's own proposal distribution, and prefix caching covers this mode (identical-repeat TTFT 159×, multi-turn 29×, outputs byte-identical to the cold runs). Against the DSpark heads of the time (DimInfer at 4-bit, RadixArk at 8-bit) DFlash 2 won by 1.14× and 1.24×; Red Hat's head, trained on 8k-token Qwen3.8-generated data with a 2048-token sliding window, accepts about as much as DFlash 2 (more on chat at 4-bit), so the choice between them is now within a few percent — --mode auto keeps DFlash 2 for both quants, and --mode dspark is a first-class alternative rather than a fallback.

The MoE row is the interesting one, and its lesson is about the baseline, not the drafter. Qwen3.6-35B-A3B activates ~3.8B of its 35B parameters per token, so plain greedy decoding already runs at 86.9 tok/s — faster than every other target here, including the 4B. The drafter is good: acceptance reaches 7.0 tokens/round on math, the highest this project has measured. It still only converts to 1.32×, because speculation's value scales with what a target step costs, and here a step is only ~11.5 ms while the drafter — 1.53B dense parameters, against a target with 3.8B active — costs ~5.7 ms of every round. Two other things follow from the same fact and are specific to this row:

• Hybrid lookup drafts are a net loss here (1.27× → 1.21×), the first target where the shipped-on default had to be turned off. A free n-gram draft still has to be verified, and on an MoE every extra verify row pulls in a fresh set of experts to read — so a low-acceptance free draft is not free at all. Every pair measured that way ships with lookup off as its registry default (all the MoEs, the 4-bit 27B hybrids, Muse — re-measured and confirmed for Qwen3.8 even after the small-M kernel flattened its curve); --lookup-drafts forces it back on.
• The confidence head finally pays (1.27× → 1.32×, and 1.50× → 1.67× on math), reversing this project's standing result that it reaches higher acceptance at lower throughput. That result was measured on dense targets with a flat verify region, where a fixed cap wastes nothing. Here the verify curve rises from the very first extra row and acceptance swings from 2.8 (chat) to 7.0 (math), so deciding per round how far to draft is worth real time. --confidence-threshold 0.3 (0.5 is within noise of it; 0.7 over-throttles, back to 1.21×). The rule this generalized to — refined once more by the Qwen3.8-27B rows: confidence pays iff the verify curve still rises inside the cap's window and the drafter leaves acceptance headroom for truncation to recover. The RadixArk 4-bit head (rising curve to width 5, modest acceptance) gained from cap 7 + 0.3; its 8-bit sibling (flat 1–8 under the kernel) measures worse with it. But the next 4-bit head, DimInfer, had the same rising curve yet accepted 3.3–5.3/round — no headroom left — so confidence bought nothing there either. Same curve, opposite verdict: the verify shape is necessary, not sufficient.

Drafter quantization was swept for this pair and 4-bit remains right — 3-bit is no cheaper (the drafter's cost is dominated by a 248K-vocab head, not by weight bytes) and 2-bit and 8-bit are both slower. Prefill's wide-GEMM lever gives only 1.03× here (905 → 931 tok/s, bit-identical) rather than the usual 1.07–1.15×, because the MoE expert weights are SwitchLinear rather than QuantizedLinear and the optimization never sees them. Turn-2 prefix reuse used to miss here: the Qwen3.6 chat template prefills a <think> opener, so the next turn's prompt landed 2 tokens short of the checkpoint boundary (4 with --no-thinking). 0.10.1's stable-boundary snapshots fix exactly this — the server measures each template's re-render-unstable tail and snapshots below it, so turn-2 reuse now fires (see Prefix caching).

The older 8-bit stamps (Qwen3-14B, Qwen3.6-27B, Ornith, Muse) peak at cap 4 and fell off sharply at cap 5 when measured — the cliff sat exactly where the verify curve left its cheap region (width 5 → 6). The small-M kernel removes that cliff: v0.12.0's first version did it for Qwen3.8-27B, and the 2026-09 kernel does it for every 8-bit target re-measured here — the curve comes back flat through width 8 and the derived cap moves to 7 on its own. Paired in the same run, cap 4 → the derived 7: Qwen3-4B 1.91× → 1.98×, Qwen3-8B 2.13× → 2.19×, Gemma-4 12B 2.82× → 3.25× (math gains most — its drafts run long enough to use the wider block; chat is flat to slightly down). Bonsai is the counter-example at the other end: its 2-bit verify cost climbs from width 2, so it peaks at cap 2 (cap 1 = 1.00×, cap 3 = 1.06×) and there is no wide-draft regime to reach.

Baselines are this harness's pipelined greedy loop, which measures at parity with mlx_lm.generate (the Qwen3-4B baseline is the same 51–52 tok/s either way). All paths produce identical output to plain decoding — they're just faster. Chat content accepts less than code everywhere; on the 2-bit Bonsai target chat lands ~break-even (its verify rows are compute-bound), and --max-draft auto adapts or parks where speculation would lose (see the Bonsai section). Why a Mac can't go much higher and the cost model are below.

The table is fresh-generation content. On copy-heavy editing — the model re-emitting or refactoring code already in its context — match-scaled lookup drafts (0.5.0, on by default) go well past it: Gemma-12B file re-emission 3.03× → 4.51× (75 tok/s), rename-refactor 4.33×; Ornith-9B rename-refactor 2.79× → 3.57× (93 tok/s), re-emission 2.45× — outputs still bit-identical, chat and fresh code unchanged. See the hybrid-drafting bullet in Flags that matter for how it works. The deep-dive's multi-prompt DSpark-vs-DFlash tables are mlx-0.31.2-era and are kept as the last full sweep — 0.32 shifted that balance toward DSpark (spot-checked; see that section's note).

Long context (16–32k tokens, 2026-09-27)

The tables above are short prompts. Agent clients (Claude Code, Codex, pi) send 10–40k-token prompts, and that is where speculative decoding used to fade on a Mac — at 32k, DSpark on Qwen3.8-27B could fall below plain decoding. Three engine changes hold the speedup at depth:

• a multi-row attention kernel — mlx's decode attention re-reads the whole KV cache once per verified row; this one reads each KV tile once for all rows (and for the DSpark drafter's own context attention);
• a preallocated DSpark drafter context — the drafter no longer copies its whole context twice per layer per round;
• a 4096-row DSpark drafter context window — DeepSpec-style heads lose acceptance when they attend a context far longer than they were trained on; attending the recent 4096 rows restores their short-context acceptance at any depth (--drafter-window, see Tuning). Heads trained with their own sliding window — DFlash 2, and Red Hat's Qwen3.8 DSpark head (2048 tokens, trained on 8k-token sequences) — hold acceptance by construction and keep it.

Measured on this M4 Pro (16" MacBook Pro, High Power energy mode for the Qwen3.8 rows) with agent-style content (mlx-lm's model sources + a coding task, thinking off): one cold prefill per depth, then every arm restores the cached prompt and decodes 256 tokens. Decode tok/s (prefill excluded), best of 2, speedup over plain greedy decoding at the same depth, acceptance in parentheses:

target + drafter	context	baseline	cap 3	cap 7
Nanbeige4.2-3B (bf16) + DSpark	16k	13.9 tok/s	1.97× (2.36)	1.89× (2.78)
	32k	10.6 tok/s	2.21× (2.74)	1.83× (3.20)
**