登录

vllm-metal

vllm-projectApache-2.0活跃维护

让 vLLM 在 Apple Silicon 的 Mac 上跑起来，通过 Metal 和 MLX 后端把大模型推理搬到本地苹果芯片，省去 GPU 服务器。

Community maintained hardware plugin for vLLM on Apple Silicon

v0.33.0发布于 2026-11-04 09:20 UTC1,982star +10 / 7 天347fork同步于 2026-11-10 23:46 UTC
大模型推理Apple SiliconMetal 后端MLX本地推理
对比 GitHub 官网apple-siliconllmmacosmetalmlx

怎么上手

由 README 提炼，执行前请确认
原文 
brew tap vllm-project/vllm-metal https://github.com/vllm-project/vllm-metal
brew install vllm-project/vllm-metal/vllm-metal
硬件要求按 README 说明

需要 Apple Silicon Mac，系统为 macOS 15 (Sequoia) 或更高；README 未给出具体显存或统一内存数值要求。

支持 Apple Silicon
下载：源码 zip·GitHub Releases（安装包）本站不镜像安装包

项目说明

由 README 自动翻译提炼
没人工点评？写下你的经验

让 vLLM 跑在 Apple Silicon Mac 上的社区硬件插件，用 MLX 作计算后端

让 vLLM 在 Apple Silicon 的 Mac 上跑起来，通过 Metal 和 MLX 后端把大模型推理搬到本地苹果芯片，省去 GPU 服务器。

核心能力

  • 以 MLX 为主要计算后端，统一 MLX 与 PyTorch 的 lowering 路径
  • 自带 paged varlen Metal 注意力内核，作为默认注意力后端
  • 利用 M5 NAX 张量单元加速 MHA、GQA、MQA 的 prefill
  • 支持投机解码与并发服务
  • 提供 sleep 模式，空闲时把内存归还给其他应用

容易踩的坑

Homebrew 安装后直接运行 vllm，不要激活虚拟环境；curl 安装脚本装完需先 source ~/.venv-vllm-metal/bin/activate；仅支持 README 支持矩阵中列出的模型，且需 macOS 15 以上。

内容由模型根据 README 生成，可能不完整或不准确，以官方文档为准。看原文 README

最近发版

5 个版本v0.33.0
全部 release
  1. v0.33.0.dev20261111011701pre-release2026-11-11 · 2026-11-11 01:17 UTC
    查看更新说明

    What's Changed

    Full Changelog: https://github.com/vllm-project/vllm-metal/compare/v0.33.0.dev20261111010347...v0.33.0.dev20261111011701

  2. v0.33.0.dev20261109121027pre-release2026-11-09 · 2026-11-09 12:10 UTC
    查看更新说明

    What's Changed

    Full Changelog: https://github.com/vllm-project/vllm-metal/compare/v0.33.0.dev20261109064304...v0.33.0.dev20261109121027

  3. v0.33.0.dev20261108123514pre-release2026-11-08 · 2026-11-08 12:35 UTC
    查看更新说明

    What's Changed

    Full Changelog: https://github.com/vllm-project/vllm-metal/compare/v0.33.0.dev20261108040348...v0.33.0.dev20261108123514

  4. v0.33.0.dev20261107074203pre-release2026-11-07 · 2026-11-07 07:42 UTC
    查看更新说明

    What's Changed

    New Contributors

    Full Changelog: https://github.com/vllm-project/vllm-metal/compare/v0.33.0.dev20261106033554...v0.33.0.dev20261107074203

  5. v0.33.0vLLM Metal v0.33.0最新正式版2026-11-04 · 2026-11-04 09:20 UTC

    vLLM Metal v0.33.0

    v0.33.0 tracks the vLLM v0.33.0 core. The release contains 70 merged pull requests from 23 contributors since vLLM Metal v0.32.0.

    Highlights

    • vLLM 0.33.0 core: the plugin moves to vLLM 0.33.0. vLLM published its 0.33.0 release on the v0.33.0rc3 tag (the same commit), so the installer and the Homebrew updater download the vLLM wheel from there (#1249).
    • Model Runner V2 by default on MLX: the MLX backend now runs under vLLM's Model Runner V2 configuration and scheduler contract by default, still through MetalModelRunner; VLLM_USE_V2_MODEL_RUNNER=0 opts back out (#1199, #1238).
    • Native MLX sampling by default: random sampling runs in MLX instead of torch, including prefill, mixed batches, and seeded requests, which draw from per-request keys; logit_bias and min_tokens are now accepted on the MLX backend instead of refused, and VLLM_METAL_NATIVE_SAMPLING=0 goes back to the torch sampler (#1196, #1201, #1233, #1239, #1096).
    • More multimodal input: Gemma 4 E2B/E4B take audio and video, MLX-quantized Qwen3.5/3.6 MoE checkpoints take images and video, and LFM2-VL/LFM2.5-VL and LFM3-VL take images (#1175, #1180, #1193, #1182, #1232).
    • New text models: LFM3, Recursive Flint, Muse Glimmer, EmbeddingGemma 2 embeddings, and MiniCPM4 MLX checkpoints serve, and Ministral 3 no longer fails at warm-up (#1212, #1200, #1120, #1214, #1086, #1156).
    • Qwen3.5 and IQ quants in GGUF: text-only Qwen3.5 GGUF files serve, including files that embed an MTP block, and IQ4_NL, IQ4_XS, and IQ3_S weights load, so llama.cpp IQ3_M and unsloth UD-Q3/Q4 exports work locally and as remote repo:TAG references (#1181, #1202, #1228, #1230, #1240).
    • Experimental PyTorch MPS backend grows: VLLM_METAL_BACKEND=mps adds Qwen3.5 Gated DeltaNet text models, LFM2-VL image input, greedy EAGLE3 speculative decoding, a fused residual-add RMSNorm kernel, and Qwen3-TTS through vLLM-Omni, and installs pin a PyTorch nightly that submits MPS work earlier (#1241, #1243, #1179, #1194, #1109, #1117).
    • MoE expert streaming: --additional-config '{"moe_expert_offload": {"cache_gb": <GiB>}}' keeps MoE experts on disk and streams the routed ones through a bounded cache, so MoE models larger than memory can serve, slower than resident; docs/moe_offload.md h

用户点评

登录用户的真实使用记录，可按版本和硬件筛选
共 0 条

    还没有人点评。第一个说出真实体验的人最有价值。

    README 原文

    未翻译
    在 GitHub 打开

    vLLM Metal is a plugin that enables vLLM to run on Apple Silicon Macs using MLX as the primary compute backend. It unifies MLX and PyTorch under a single lowering path.

    Latest News 🔥

    • [2026/10] Experimental Recursive Flint INT4 support is now available on the MLX backend, including concurrent serving.
    • [2026/10] DiffusionGemma now supports Jev-like System 1 decisions on Metal, with median decision latency under 0.2 s on an M5 Pro. See the structured decisions guide.
    • [2026/10] Desktop inference competes with your other applications for memory. Sleep mode lets an idle model server return memory to those apps without shutting down. Read Abhigith Neil Abraham's walkthrough.
    • [2026/10] Run local models with vLLM Metal through llmman. See the setup recipe.
    • [2026/09] SiliconBench, which evaluates vLLM Metal and other serving engines on speed, memory use, and output fidelity, was accepted at NeurIPS 2026's Evaluations and Datasets Track.
    • [2026/09] Read our official vLLM announcement, covering concurrent serving on Apple Silicon and v0.28.0.
    • [2026/08] vLLM Metal now uses M5 NAX tensor units to accelerate MHA, GQA, and MQA prefill.
    • [2026/08] Qwen3.8 now runs on Metal! mlx-community/Qwen3.8-27B-8bit serves a 27B hybrid SDPA + GDN linear model on a single Apple Silicon Mac.
    • [2026/04] We released the new version v0.2.0! Unified paged varlen Metal kernel is now the default attention backend. 83x TTFT, 3.6x throughput compared to v0.1.0.

    Architecture

    Upstream vLLM supplies the API server, scheduler, and paged block manager; mlx_lm supplies the token-wise model layers; vllm-metal owns the request-aware attention path — the paged varlen kernel, M5 NAX prefill, and speculative decoding.

    vllm-metal in the vLLM stack

    Requirements

    • macOS 15 (Sequoia) or later, on Apple Silicon

    Supported Models

    vllm-metal supports a growing set of models on Apple Silicon. See the full matrix in docs/supported_models.md.

    Installation

    • Stable release: Install with Homebrew, then run vllm without activating an environment.

      brew tap vllm-project/vllm-metal https://github.com/vllm-project/vllm-metal
brew install vllm-project/vllm-metal/vllm-metal

    • Latest development build: Use the curl installer, then activate its environment. No compiler needed.

      curl -fsSL https://raw.githubusercontent.com/vllm-project/vllm-metal/main/install.sh | bash
source ~/.venv-vllm-metal/bin/activate

    • Contributing Python or kernel code: Follow the source setup for an editable checkout and native build tools.

    相关项目

    同分类里 star 最高的项目
    对比全部 →
    mlx
    ml-explore·C++

    面向 Apple silicon 的数组计算框架，让机器学习模型能直接在 Mac 芯片上高效训练和推理，省去折腾 CUDA 的麻烦。

    Apple silicon+4硬件加速适配进阶
    29126+392.4kv0.32.42026-11-06 23:36 UTC
    burn
    tracel-ai·Rust

    Rust 写的张量库和深度学习框架，兼顾灵活性与效率，支持自动微分、跨平台后端和内核融合，可训练和部署神经网络。

    深度学习框架+4硬件加速适配硬核
    16266+161.1kv0.22.12026-11-10 23:02 UTC
    rio
    raphamorim·Rust

    用 Rust 写的 GPU 加速终端模拟器，基于 Metal/Vulkan 渲染，桌面和浏览器里都能跑，适合追求流畅终端体验的开发者。

    终端模拟器+4硬件加速适配
    7725+15381v0.5.292026-11-08 15:57 UTC
    mlx-lm
    ml-explore·Python

    在 Apple Silicon 上通过 MLX 框架运行大语言模型，支持加载、推理和微调，让 Mac 本地跑 LLM 更省心。

    MLX+4硬件加速适配
    7618+311.2kv0.32.22026-11-06 11:23 UTC