登录

sglang-omni

sgl-projectApache-2.0活跃维护

面向交互式超级智能系统的高性能推理服务运行时，支持语音识别、音频与音乐生成等多模态模型，并提供 OpenAI 兼容 API。

High Performance Serving Runtime For Interactive Super Intelligence System

voxt-v2.0.3发布于 2026-11-10 02:12 UTC1,748star +35 / 7 天720fork同步于 2026-11-11 05:44 UTC
推理服务多模态语音识别音频生成分布式推理
对比 GitHub 官网asraudio-generationcudadistributed-inferenceinference

怎么上手

由 README 提炼，执行前请确认
原文 
uv pip install --prerelease=allow "sglang-omni==0.2.0"
硬件要求按 README 说明

README 未说明具体显存与 GPU 型号要求，仅提到可在 Apple Silicon 上做本地语音识别（Voxt 工具）

支持 Apple Silicon需要 CUDA
下载：源码 zip·GitHub Releases（安装包）本站不镜像安装包

项目说明

由 README 自动翻译提炼
没人工点评？写下你的经验

面向语音、TTS 与全模态模型的多阶段推理服务运行时，把生成拆成异构阶段统一编排

面向交互式超级智能系统的高性能推理服务运行时，支持语音识别、音频与音乐生成等多模态模型，并提供 OpenAI 兼容 API。

核心能力

  • 把生成拆成预处理、编码器、自回归引擎、解码器、声码器等可组合阶段
  • 每个阶段配专属调度器，自回归阶段复用 SGLang 调度与执行
  • 控制面协调请求，数据面经共享内存、NCCL、NIXL、Mooncake 传输张量
  • 提供 OpenAI 兼容接口，覆盖多模态对话、语音生成、流式语音与转写
  • 支持 /v1/realtime 全双工实时听说，含 MiniCPM-o 摄像头输入

容易踩的坑

安装 0.2.0 需加 --prerelease=allow，否则预发布版本可能装不上；模型需按 cookbook 从 Hugging Face 单独拉取，不在安装包内；README 未给出默认端口，需查安装与 TTS 文档确认

内容由模型根据 README 生成，可能不完整或不准确，以官方文档为准。看原文 README

最近发版

5 个版本voxt-v2.0.3
全部 release
  1. voxt-v2.0.3Voxt v2.0.3最新正式版2026-11-10 · 2026-11-10 02:12 UTC

    Requirements

    • A Mac with Apple silicon and macOS 15 or later.

    Install

    • Open Voxt-2.0.3-arm64.dmg and drag Voxt to Applications. It replaces Voxt 1.x there.
    • This version does not update itself: install a newer one the same way.

    Changed

    • Plainer text across the app: about 300 English strings stop chaining clauses with colons and semicolons and leave out details nobody needs, about 360 Chinese strings read as written in Chinese and name each thing one way, and about 50 help texts that only said what the page already shows are gone. In Chinese, Rewrite is now 改写 (转写 also meant a transcript), a transcript is 转录稿, speech models are 语音识别模型 and an app is always App.
    • Sound: Built-in, macOS or None (Settings → Audio): one choice says where the recording's start and stop sounds come from. Built-in lists six pairs of a start and a stop sound, each to try with ▶ and pick with a click; macOS picks a system sound for each; None plays nothing. The default pair is now Pebble and Nib, softer than Tally and Wrap. The panel of sounds by group, tag and search is gone, and so is Use Voxt 1.x's sounds: macOS starts on Pop and Tink.
    • Settings sits in the sidebar's System section, under Logs, instead of a row of its own at the foot of the sidebar; the version number is on About. In the icon rail its icon is in that section too, with the dot while there are notifications.

    中文

    变更
    • 中英文文案过了一遍：约 300 条英文不再用冒号、分号把几句硬连在一起，也删掉了用不着的细节；约 360 条中文去掉了翻译腔，同一个东西只用一种叫法；约 50 条只是重复界面内容的说明删掉了。Rewrite 改叫“改写”（原来的“转写”也指转录稿），会议和文件的 transcript 统一叫“转录稿”，语音模型统一叫“语音识别模型”，app 统一写作“App”。
    • 音效：自带、系统或无（设置 → 音频）：一个选项决定录音开始和结束的提示音从哪里来。“自带”列出 6 组成对的开始音和结束音，点 ▶ 试听，点一下选中；“系统”给开始和结束各选一个 macOS 声音；“无”就不响。默认换成更轻的 Pebble 和 Nib（原来是 Tally 和 Wrap）。按分组、标签和搜索找声音的面板去掉了，“使用 Voxt 1.x 的提示音”也去掉了：选“系统”默认就是 Pop 和 Tink。
    • “设置”放进了侧边栏的“系统”分组，在“日志”下面，不再单独占侧边栏底部一行；版本号在“诊断与关于”里。侧边栏收成图标时，“设置”图标也在这一组里，有通知时照样显示红点。

    Build

    • Built from 4e03928295bba17c8196644f50d69e58facff6ea by the Voxt Release workflow. Signed with Developer ID and notarized by Apple. The SHA-256 checksum is attached.

    What's Changed

  2. voxt-v2.0.1Voxt v2.0.12026-11-07 · 2026-11-07 17:16 UTC
    查看更新说明

    Requirements

    • A Mac with Apple silicon and macOS 15 or later.

    Install

    • Open Voxt-2.0.1-arm64.dmg and drag Voxt to Applications. It replaces Voxt 1.x there.
    • This version does not update itself: install a newer one the same way.

    Added

    • Remote language models (the new Language models page): Transcribe's clean-up, Translate and Rewrite use a language model of a service you set up, over its OpenAI-compatible API: OpenAI, Anthropic, Google Gemini, DeepSeek, Qwen (Alibaba Cloud), Kimi, GLM, Doubao (Volcengine), MiniMax, StepFun, Xiaomi MiMo, xAI, OpenRouter, any other compatible server (Custom), or one on this Mac (Ollama, LM Studio, oMLX), so that the text stays here. Each service takes an API key, an address and a model, with Test connection; the key is kept in the macOS Keychain only, never in Voxt's settings or logs. Every feature's language model menu lists the services set up; Automatic uses the first one, and each card says which features use it. The page groups the servers that run on this Mac (free) apart from the online services. Only the recognised or selected text is sent, never audio.
    • Import from the Swift Voxt 1.x. When Voxt 1.x's data is on the Mac, the setup guide starts with a card that brings over its dictionary (replacement terms become "heard as" aliases, disabled terms stay disabled), its dictation, translation and rewrite history, and the speech models it downloaded that Voxt uses, so they are not downloaded again (its language models are not: Voxt no longer runs language models on this Mac). Voxt 1.x's data is only read. If you set up 2.0.0, which had no import, a card in the sidebar offers it once; it is also in the dictionary's ••• menu (docs/voxt-migration.md).
    • Import terms takes the dictionary file Voxt 1.x exports (Dictionary → More → Export): its terms, with their replacement terms as aliases.
    • System dictation is back as a speech model (Speech models → On this Mac): macOS's own speech recognition, on this Mac, with nothing to download, and the text shown as you speak. It hears your main language (Settings → General; without one, the Mac's dictation language), and your dictionary's terms help it. Choose it on a feature's page; Automatic still uses a speech model you downloaded. The first time, macOS asks for Speech Recognition (Settings → Permissions has it too); a language this Mac cannot recognise on device fails with a notice that says so.
    • Remote speech recognition (the Speech models page's Remote services): OpenAI Transcribe, Doubao (Volcengine), Zhipu GLM-ASR, Aliyun Bailian ASR, StepFun, Xiaomi MiMo and Google Gemini, as in Voxt 1.x. Doubao, Aliyun Bailian and Gemini show the text while you speak (tagged Realtime); the others get the recording once you stop. Set one up with its key (Doubao also takes its App ID), address and model, and test the connection before saving; keys stay in the Mac's Keychain, never in Voxt's settings or logs. OpenAI Transcribe, StepFun and Zhipu pointed at a server on this Mac (a local Whisper server) need no key. A service that is set up is offered after the speech models of this Mac in every feature's speech model menu, and used when you choose it there, or when this Mac has no speech model (system dictation counts as one). Doubao, Gemini and Aliyun Bailian's Fun-ASR and Qwen-Audio models get your dictionary's terms as hot words; OpenAI Transcribe, StepFun and Zhipu get them in their prompt; Xiaomi MiMo takes none.
    • Choose the model folder (Settings → Storage → Model folder → Change…): keep the models in another folder or on another disk. Voxt brings the models there first (hard links on the same disk, so no space is used; copies on another disk), including the other ActNow apps' models, wherever each app kept them, so that none is downloaded again; then those apps use the new folder too, and Voxt restarts. The old folders stay as they were: delete them in Finder once no app use
  3. voxt-v2.0.0Voxt v2.0.02026-11-06 · 2026-11-06 10:05 UTC
    查看更新说明

    Requirements

    • A Mac with Apple silicon and macOS 15 or later.

    Install

    • Open Voxt-2.0.0-arm64.dmg and drag Voxt to Applications. It replaces Voxt 1.x there.
    • This version does not update itself: install a newer one the same way.

    Added

    • Voxt 2.0.0, rebuilt from scratch: voice input for macOS on local models only. Press a shortcut in any app, speak, and the text appears where your cursor is. Audio and text never leave your Mac.
    • Local speech recognition: two models, accurate (recommended) and fast, with the text shown live as you speak, and a voice detector that sends only speech to them.
    • Local language models: a balanced one (recommended), a fast one and a translation model, for clean-up, translation and rewriting.
    • Transcribe, with an optional clean-up of fillers, false starts and punctuation; Translate, the selected text or what you say; Rewrite, the selected text or new text from a spoken request, with follow-ups in the answer card.
    • App enhancement: groups of apps and websites with their own scene prompt and a key pressed after inserting.
    • Dictionary: its terms help the speech model hear your names and jargon, and the language model write them as you spell them.
    • The Local models page: download, pause and resume, every file verified, delete; the memory a loaded model uses. Downloads use the system's proxy, or a mirror when Hugging Face cannot be reached.
    • A setup guide that installs the two recommended models and tries each feature; history, the overlay, the menu bar, shortcuts and settings.
    • Shortcuts: tap the right Option key to transcribe, right Option with Shift to translate, Control with right Option to rewrite; Control-Option-V pastes the last result. They leave fn to Auralink, so the two apps run side by side.
    • The app is signed and notarised. This version does not update itself: Settings → About says so, and a new version is downloaded by hand.
    • History entries have a context menu: right-click (or Control-click) one to copy its result or its recognised text, show its details, show its recording in Finder, or delete it. Home's recent results have the same menu. In History a selected entry also takes ⌘C (copy) and Delete.
    • Keep models loaded (Local models → Advanced): how long a model stays in memory unused. Automatic, the default, is as before (speech 10 minutes, language models 5); you can choose 5, 15 or 30 minutes, 1 hour, or until Voxt quits, so that every dictation starts at once. Other apps that use this model library can still ask Voxt to free the memory of models it is not using. A change applies at once.
    • Sounds to choose (Settings → Audio → Play sounds): the start and the stop sound are each chosen from a sound library (from React Bits Sound Lab, in the official builds; Tally and Wrap by default), or macOS's system sounds, or none. A compact picker searches them (names and tags), filters by type, plays each with ▶ (again to stop) and sets their volume in steps; a Try button plays the choice. A Voxt pair chosen in an earlier build is kept as those system sounds. A build from source without the library plays Pop and Tink.
    • Mute system audio while recording (Settings → Audio, on by default): the current output device is muted from just after the start sound until the recording stops, so that videos, music or calls are not recorded; a device you muted stays muted, one you unmute during the recording stays on, and a device that cannot be muted leaves the recording as it is (the overlay says so). No permission is needed. If Voxt or its helper stops while muting, the sound comes back at once, or at the next start.
    • A notice when your Mac is short of memory: when the language model was slow, or did not answer in time, while the Mac's swap was over 90% full, the overlay says "Low memory slows the model: quit unused apps." after the text went in. It shows at most once each time Voxt runs.
    • History settings (the button beside History's search field): keep the re
  4. v0.2.02026-10-27 · 2026-10-27 17:11 UTC
    查看更新说明

    What's Changed

  5. voxt-v1.16.3Voxt v1.16.32026-10-23 · 2026-10-23 00:09 UTC
    查看更新说明

    Voxt v1.16.3 is the same app as 1.16.2. It is the first Voxt release cut from a tag on main alone, with generated release notes.

    Requirements

    • macOS 15 or later.
    • Local models need Apple Silicon. On an Intel Mac, Voxt installs and works with a remote speech recognition provider.

    Install

    • Open Voxt-1.16.3.dmg and drag Voxt to Applications, or run Voxt-1.16.3.pkg.
    • Upgrading from an earlier Voxt: install over it, or let the app's update check install it; settings, history, permissions and downloaded models carry over.
    • New Homebrew installs (brew install --cask voxt) get this version.

    Changed

    • No changes to the app, the native runtime or the models since 1.16.2.
    • Voxt releases are now cut by pushing a voxt-vX.Y.Z tag on main; the version comes from the tag.

    What's Changed

    Build

    • Built from 7be131776f0c0587e5b93408cf7a4e2692b33a08 by the Voxt Release workflow, with the runtime from sglang_omni_mlx/native built against MLX 0.32.3 (macOS 15 wheels).
    • Signed with Developer ID and notarized by Apple. SHA-256 checksums are attached.

用户点评

登录用户的真实使用记录，可按版本和硬件筛选
共 0 条

    还没有人点评。第一个说出真实体验的人最有价值。

    README 原文

    未翻译
    在 GitHub 打开

    News

    • [2026/10] 🗺️ A unified runtime for interactive intelligence: native full-duplex models, turn-based multimodal models, and cascaded systems, coordinated through composable graphs and persistent sessions. [Blog]
    • [2026/10] 🚀 SGLang-Omni v0.2.0 is on PyPI. Install with uv pip install --prerelease=allow "sglang-omni==0.2.0". [Installation]
    • [2026/10] 🐟 Day-0 support for Fish Audio Drama 3: multi-speaker text-to-speech with voice cloning and natural-language control, streamed on /v1/audio/speech. [Blog] [Cookbook] [Technical Report]
    • [2026/10] 🎙️ Introducing Voxt, an open-source voice input tool for Mac, now integrating SGLang-Omni for faster local speech recognition on Apple Silicon. Combine ASR with LLM-based text refinement and personal system prompts for accurate technical terminology and personalized voice input, with fully local or remote inference options. [Download][Demo]
    • [2026/09] 🐧 Day-0 support for AuK and AuK-Flash: text + voice instructions → 24 kHz speech on /v1/audio/speech, audio + editing instructions → edited speech on /generate. [Cookbook]
    • [2026/08] 🎵 Day-0 support for MiniMax Music 3: lyrics + caption → 32 kHz stereo song on /v1/audio/speech. [Cookbook]
    • [2026/08] 🚀 TTS architecture refactor: shared pipeline state, engine construction, reference encoding, capability metadata, and vocoder scheduling. [Roadmap] [Blog]
    • [2026/06] 🔥 MOSS-TTS Local Transformer v1.5 on SGLang-Omni with native-streaming 48 kHz speech. [Blog] [Cookbook]
    • [2026/06] 🔥 Higgs Audio v3 TTS for real-time, controllable speech. [Blog] [Cookbook]

    About

    SGLang-Omni is a multi-stage serving runtime for omni, speech, and TTS models. Its design target is multi-stage decoding: generation split across heterogeneous stages with different compute patterns, dependency structures, and resource needs. SGLang-Omni owns the pipeline topology, stage lifecycle, inter-stage transport, model-family integration layer, and OpenAI-compatible serving surface, while composing with SGLang for high-performance autoregressive scheduling and model execution where applicable.

    • Multi-stage runtime: SGLang-Omni models generation as coordinated stages: preprocessing, encoders, autoregressive engines, talkers, decoders, vocoders, and aggregators.
    • Stage-specialized scheduling: Each stage runs behind a scheduler matched to its workload, from SGLang-backed autoregressive scheduling to lightweight preprocessing and streaming vocoder loops.
    • Transport-aware execution: A control plane coordinates requests while the relay data plane moves tensor payloads across shared-memory, NCCL, NIXL, and Mooncake backends.
    • API surface: OpenAI-compatible endpoints expose multimodal chat, speech generation, batch speech, streaming speech, uploaded voices, and transcription.

    What SGLang-Omni Serves

    Hardware Support

    BackendStatusNotes
    NVIDIA CUDASupportedDefault backend with full model coverage.
    Apple SiliconExperimentalQwen3-ASR runs through Torch MPS on macOS arm64. Install with install.sh and follow the Qwen3-ASR guide.
    Intel GPU (XPU)SupportedIntel Arc GPUs via PyTorch XPU. Qwen3-ASR, Whisper ASR, Fun-ASR-Nano, Nemotron 3.5 ASR, MOSS-Transcribe-Diarize, Qwen3-TTS, Fun-CosyVoice3, ZONOS2, Audar-TTS, Qwen3-Omni, MiniMax Music 3, MiniCPM-o, Ming-Omni, Ming-Omni-TTS, Nemotron VoiceChat, PersonaPlex, AuK/AuK-Flash and dots.tts serve end-to-end. The Intel XPU guide has the install steps, each model's launch command and how many cards it needs; the backend is auto-detected.

    Additional model guides, including experimental and research-oriented paths, are available in the Cookbook.

    Quick Start

    Community & Support

    SGLang-Omni welcomes contributors working on inference systems, kernels, scheduling, inter-stage communication, model runners and cache efficiency, model integration, benchmarking, production deployment. Join the SGLang Slack or read the developer reference.

    Organizations interested in supporting SGLang-Omni, TTS, or omni model serving can contact Chenyang Zhao at [email protected].

    Citation

    If you use SGLang-Omni for your research, please cite our technical blogs on the unified interactive runtime and the Stage--Process--GPU topology:

    @misc{sglangomniinteractive2026,
  title={Unified Runtime for Full-Duplex Interactive Intelligence with {SGLang Omni}},
  author={{SGLang-Omni Team}},
  howpublished={\url{https://www.sglang.io/blog/omni-interactive-runtime}},
  year={2026},
  month={oct}
}

@misc{sglangomni2026,
  title={Fish Audio Drama 3 Meets {SGLang-Omni}: Day-0 Support and the Stage--Process--GPU Topology},
  author={{SGLang-Omni Team}},
  howpublished={\url{https://www.sglang.io/blog/spg-topology}},
  year={2026},
  month={oct}
}

    Acknowledgments

    SGLang-Omni builds on the SGLang ecosystem and on open model work from the TTS, speech, and omni-model communities. We thank the model teams, systems contributors, and partner organizations helping make open multimodal serving faster, more reliable, and easier to extend.

    相关项目

    同分类里 star 最高的项目
    对比全部 →
    VoiceStudio
    debpalash·Python

    本地运行的 ElevenLabs 开源替代品，支持声音克隆、语音设计、视频配音、听写、转写和有声书制作，覆盖 646 种语言。

    语音克隆+4语音推理进阶
    107442+431812kv0.5.8-beta.42026-11-10 23:53 UTC
    voicebox
    jamiepine·Python

    开源 AI 语音工作室，支持声音克隆、语音听写和语音生成，基于 Qwen3-TTS 与 Whisper，可在 CUDA 或 MLX 上本地运行。

    语音克隆+4语音推理硬核
    60405+6527.6kv0.5.02026-10-07 01:17 UTC
    whisper.cpp
    ggml-org·C++

    OpenAI Whisper 语音识别模型的 C/C++ 移植版，无需 Python 和 GPU 依赖，在普通 CPU 上就能跑语音转文字。

    语音识别+4语音推理进阶
    55081+946.3kv1.9.52026-11-10 12:09 UTC
    faster-whisper
    SYSTRAN·Python

    基于 CTranslate2 重实现的 Whisper 语音转文字推理引擎，通过量化与高效推理大幅提速并降低显存占用，适合本地部署转录。

    语音识别+4语音推理进阶
    26657+852.2kv1.2.12026-11-10 10:13 UTC