VoxCPM

OpenBMBApache-2.0活跃维护

无需分词器的多语言语音合成模型,支持创意音色设计和逼真声音克隆,可直接从文本生成自然语音。

VoxCPM2: Tokenizer-Free TTS for Multilingual Speech Generation, Creative Voice Design, and True-to-Life Cloning

2.0.3发布于 2026-05-11 11:59 UTC42,140star +996 / 7 天4,801fork同步于 2026-11-10 05:19 UTC

怎么上手

由 README 提炼,执行前请确认
pip install voxcpm
硬件要求按 README 说明

README 未给出显存要求;实时流式在 NVIDIA RTX 4090 上 RTF 约 0.3,经 Nano-vLLM 或 vLLM-Omni 加速约 0.13;另有 llama.cpp-omni 支持 CPU、Metal、CUDA、Vulkan 端侧推理。

支持 Apple Silicon需要 CUDA支持 ROCm / AMD多卡 / 张量并行
下载:源码 zip·GitHub Releases(安装包)本站不镜像安装包

项目说明

由 README 自动翻译提炼

无分词器的多语言 TTS,直接生成连续语音表示,支持音色设计与克隆

无需分词器的多语言语音合成模型,支持创意音色设计和逼真声音克隆,可直接从文本生成自然语音。

核心能力

  • 用扩散自回归架构直接生成连续语音表示,绕开离散分词
  • 支持 30 种语言输入,无需语言标签直接合成
  • 仅凭自然语言描述即可设计全新音色,无需参考音频
  • 短参考音频克隆音色,可加风格引导控制情绪与语速
  • 参考音频加转写文本可复刻音色、节奏、情绪等细节

容易踩的坑

需 Python ≥3.10 且 <3.13、PyTorch ≥2.5.0、CUDA ≥12.0;首次运行需从 HuggingFace 或 ModelScope 下载 openbmb/VoxCPM2 权重;vLLM-Omni 服务需单独安装并指定 --omni 与端口;端侧推理需另下 GGUF 的 BaseLM 与 Acoustic 两个文件。

内容由模型根据 README 生成,可能不完整或不准确,以官方文档为准。看原文 README

最近发版

5 个版本2.0.3
  1. 2.0.3v2.0.3: fine-tuning validation, runtime stability, and streaming improvements最新正式版2026-05-11 · 2026-05-11 11:59 UTC

    VoxCPM v2.0.3

    This release focuses on fine-tuning usability, runtime stability, safer LoRA loading, and faster streaming inference.

    Highlights

    • Added voxcpm validate for pre-flight JSONL training manifest validation.
    • Added optional ref_audio support in the fine-tuning data pipeline.
    • Improved runtime device handling with explicit --device support and safer MPS dtype behavior.
    • Improved VoxCPM2 streaming VAE decoding by avoiding redundant overlap decoding.
    • Hardened legacy LoRA checkpoint loading with weights_only=True.
    • Fixed LoRA rank mismatch handling in lora_ft_webui.py.

    New Features

    • Add voxcpm validate --manifest train.jsonl to catch training data issues before fine-tuning.
      • Validates JSONL format, required text/audio fields, audio existence/readability, sample rate, duration stats, text length stats, and optional ref_audio.
    • Add optional ref_audio support for fine-tuning manifests.
      • Training packing now supports [103, ref_audio, 104, text, 101, target_audio, 102].
      • Loss is applied only to the target audio segment.
    • Add --device CLI argument for model inference commands.
      • Supports auto, cpu, mps, cuda, and indexed CUDA devices such as cuda:0.

    Performance

    • Improve VoxCPM2 streaming VAE decode with a stateful StreamingVAEDecoder.
      • Streaming decode now processes only the newest latent patch and carries causal convolution state internally.
      • This removes redundant overlap decoding and reduces streaming VAE decode overhead.

    Fixes

    • Fix CUDA Graph dynamic-shape accumulation by using the uncompiled feature encoder for prefill.
    • Fix CPU SDPA attention mask broadcasting by using an explicit broadcastable mask shape.
    • Fix non-string text validation order to raise the intended ValueError instead of AttributeError.
    • Fix file descriptor leaks when loading config.json in local model loaders.
    • Fix MPS audio quality issues by promoting low-precision dtypes to float32 on Apple Silicon by default.
    • Fix VOXCPM_MPS_DTYPE override validation to match supported dtype aliases.
    • Fix LoRA rank mismatch in lora_ft_webui.py by reloading the model when checkpoint rank differs.
    • Fix Web Demo control text handling by stripping parentheses before constructing the model prompt.

    Security

    • Legacy LoRA .ckpt / .pth loading now uses torch.load(..., weights_only=True).
    • This reduces the risk of arbitrary pickle payload execution while preserving tensor-only checkpoint compatibility.

    Documentation

    • Document vLLM-Omni as a production serving option for VoxCPM2.
    • Update Web Demo usage to python app.py --port 8808.
    • Update ModelScope local download example.
    • Clarify Python requirement as >=3.10,<3.13.
    • Add ComfyUI_RH_VoxCPM to the ecosystem list.

    Tests

    • Added coverage for training manifest validation, including sample-rate mismatch, missing audio, relative paths, ref_audio, and CLI exit codes.
    • Added runtime device selection tests.
    • Added LoRA checkpoint safety tests for tensor-only checkpoints and malicious pickle payloads.
    • Added CLI tests for --device defaults and argument forwarding.

    Contributors

    Thanks to the contributors included in this release:

    • @KevinAHM
    • @kuishou68
    • @sharziki
    • @SuperMarioYL
    • @Oumnya
    • @linyueqian
    • @shaun0927
    • @gluttony-10

    Full Changelog: https://github.com/OpenBMB/VoxCPM/compare/2.0.2...2.0.3

  2. 2.0.2v2.0.2: fixed some bugs2026-04-08 · 2026-04-08 10:19 UTC
    查看更新说明
  3. 2.0.1v2.0.1: removed auto-trim feature for reference audio2026-04-08 · 2026-04-08 06:06 UTC
    查看更新说明
  4. 2.0.0VoxCPM 2 Released2026-04-06 · 2026-04-06 14:51 UTC
    查看更新说明

    What's Changed

    New Contributors

    Full Changelog: https://github.com/OpenBMB/VoxCPM/compare/1.5.0...2.0.0

  5. 1.5.0VoxCPM 1.5 Released2025-12-05 · 2025-12-05 14:48 UTC
    查看更新说明

    Full Changelog: https://github.com/OpenBMB/VoxCPM/compare/1.0.5...1.5.0

    What's New and What's Next? See our Release Note

用户点评

登录用户的真实使用记录,可按版本和硬件筛选
共 0 条

    还没有人点评。第一个说出真实体验的人最有价值。

    README 原文

    未翻译

    VoxCPM is a tokenizer-free Text-to-Speech system that directly generates continuous speech representations via an end-to-end diffusion autoregressive architecture, bypassing discrete tokenization to achieve highly natural and expressive synthesis.

    VoxCPM2 is the latest major release — a 2B parameter model trained on over 2 million hours of multilingual speech data, now supporting 30 languages, Voice Design, Controllable Voice Cloning, and 48kHz studio-quality audio output. Built on a MiniCPM-4 backbone.

    ✨ Highlights

    • 🌍 30-Language Multilingual — Input text in any of the 30 supported languages and synthesize directly, no language tag needed
    • 🎨 Voice Design — Create a brand-new voice from a natural-language description alone (gender, age, tone, emotion, pace …), no reference audio required
    • 🎛️ Controllable Cloning — Clone any voice from a short reference clip, with optional style guidance to steer emotion, pace, and expression while preserving the original timbre
    • 🎙️ Ultimate Cloning — Reproduce every vocal nuance: provide both reference audio and its transcript, and the model continues seamlessly from the reference, faithfully preserving every vocal detail — timbre, rhythm, emotion, and style (same as VoxCPM1.5)
    • 🔊 48kHz High-Quality Audio — Accepts 16kHz reference audio and directly outputs 48kHz studio-quality audio via AudioVAE V2's asymmetric encode/decode design, with built-in super-resolution — no external upsampler needed
    • 🧠 Context-Aware Synthesis — Automatically infers appropriate prosody and expressiveness from text content
    • ⚡ Real-Time Streaming — RTF as low as ~0.3 on NVIDIA RTX 4090, and ~0.13 accelerated by Nano-vLLM or vLLM-Omni — official vLLM omni-modal serving for VoxCPM2 with PagedAttention and an OpenAI-compatible API
    • 📜 Fully Open-Source & Commercial-Ready — Weights and code released under the Apache-2.0 license, free for commercial use

    🌍 Supported Languages (30)
    Arabic, Burmese, Chinese, Danish, Dutch, English, Finnish, French, German, Greek, Hebrew, Hindi, Indonesian, Italian, Japanese, Khmer, Korean, Lao, Malay, Norwegian, Polish, Portuguese, Russian, Spanish, Swahili, Swedish, Tagalog, Thai, Turkish, Vietnamese

    Chinese Dialect: 四川话, 粤语, 吴语, 东北话, 河南话, 陕西话, 山东话, 天津话, 闽南话

    News

    • [2026.04] 🔥 We release VoxCPM2 — 2B, 30 languages, Voice Design & Controllable Voice Cloning, 48kHz audio output! Weights | Docs | Playground | Technical Report
    • [2025.12] 🎉 Open-source VoxCPM1.5 weights with SFT & LoRA fine-tuning. (🏆 #1 GitHub Trending)
    • [2025.09] 🔥 Release VoxCPM Technical Report.
    • [2025.09] 🎉 Open-source VoxCPM-0.5B weights (🏆 #1 HuggingFace Trending)

    Contents


    🚀 Quick Start

    Installation

    pip install voxcpm
    

    Requirements: Python ≥ 3.10 (<3.13), PyTorch ≥ 2.5.0, CUDA ≥ 12.0. See Quick Start Docs for details.

    Python API

    🗣️ Text-to-Speech
    from voxcpm import VoxCPM
    import soundfile as sf
    
    model = VoxCPM.from_pretrained(
      "openbmb/VoxCPM2",
      load_denoiser=False,
    )
    
    wav = model.generate(
        text="VoxCPM2 is the current recommended release for realistic multilingual speech synthesis.",
        cfg_value=2.0,
        inference_timesteps=10,
        seed=42,
    )
    sf.write("demo.wav", wav, model.tts_model.sample_rate)
    print("saved: demo.wav")
    

    If you prefer downloading from ModelScope first, you can use:

    pip install modelscope
    
    from modelscope import snapshot_download
    snapshot_download("OpenBMB/VoxCPM2", local_dir='./pretrained_models/VoxCPM2') # specify the local directory to save the model
    
    from voxcpm import VoxCPM
    import soundfile as sf
    model = VoxCPM.from_pretrained("./pretrained_models/VoxCPM2", load_denoiser=False)
    
    wav = model.generate(
        text="VoxCPM2 is the current recommended release for realistic multilingual speech synthesis.",
        cfg_value=2.0,
        inference_timesteps=10,
        seed=42,
    )
    sf.write("demo.wav", wav, model.tts_model.sample_rate)
    
    🎨 Voice Design

    Create a voice from a natural-language description — no reference audio needed. Format: put the description in parentheses at the start of text(e.g. "(your voice description)The text to synthesize."):

    wav = model.generate(
        text="(A young woman, gentle and sweet voice)Hello, welcome to VoxCPM2!",
        cfg_value=2.0,
        inference_timesteps=10,
        seed=42,
    )
    sf.write("voice_design.wav", wav, model.tts_model.sample_rate)
    
    🎛️ Controllable Voice Cloning

    Upload a reference audio. The model clones the timbre, and you can still use control instructions to adjust speed, emotion, or style.

    wav = model.generate(
        text="This is a cloned voice generated by VoxCPM2.",
        reference_wav_path="path/to/voice.wav",
    )
    sf.write("clone.wav", wav, model.tts_model.sample_rate)
    
    wav = model.generate(
        text="(slightly faster, cheerful tone)This is a cloned voice with style control.",
        reference_wav_path="path/to/voice.wav",
        cfg_value=2.0,
        inference_timesteps=10,
        seed=42,
    )
    sf.write("controllable_clone.wav", wav, model.tts_model.sample_rate)
    
    🎙️ Ultimate Cloning

    Provide both the reference audio and its exact transcript for audio-continuation-based cloning with every vocal nuance reproduced. For maximum cloning similarity, pass the same reference clip to both reference_wav_path and prompt_wav_path as shown below:

    wav = model.generate(
        text="This is an ultimate cloning demonstration using VoxCPM2.",
        prompt_wav_path="path/to/voice.wav",
        prompt_text="The transcript of the reference audio.",
        reference_wav_path="path/to/voice.wav", # optional, for better simliarity 
    )
    sf.write("hifi_clone.wav", wav, model.tts_model.sample_rate)
    

    🔄 Streaming API

    import numpy as np
    
    chunks = []
    for chunk in model.generate_streaming(
        text="Streaming text to speech is easy with VoxCPM!",
    ):
        chunks.append(chunk)
    wav = np.concatenate(chunks)
    sf.write("streaming.wav", wav, model.tts_model.sample_rate)
    

    CLI Usage

    # Voice design (no reference audio needed)
    voxcpm design \
      --text "VoxCPM2 brings studio-quality multilingual speech synthesis." \
      --output out.wav
    
    # Controllable voice cloning with style control
    voxcpm design \
      --text "VoxCPM2 brings studio-quality multilingual speech synthesis." \
      --control "Young female voice, warm and gentle, slightly smiling" \
      --seed 42 \
      --output out.wav
    
    # Voice cloning (reference audio)
    voxcpm clone \
      --text "This is a voice cloning demo." \
      --reference-audio path/to/voice.wav \
      --output out.wav
    
    # Ultimate cloning (prompt audio + transcript)
    voxcpm clone \
      --text "This is a voice cloning demo." \
      --prompt-audio path/to/voice.wav \
      --prompt-text "reference transcript" \
      --reference-audio path/to/voice.wav \ # optional, for better simliarity
      --output out.wav
    
    # Batch processing
    voxcpm batch --input examples/input.txt --output-dir outs
    
    # Optional post-generation timestamps with stable-ts
    pip install "voxcpm[timestamps]"
    voxcpm design \
      --text "VoxCPM2 brings studio-quality multilingual speech synthesis." \
      --output out.wav \
      --timestamps \
      --timestamp-level word \
      --timestamp-language en
    
    # Character timestamps are best-effort and are derived from word alignment
    voxcpm design \
      --text "欢迎使用 VoxCPM2。" \
      --output out.wav \
      --timestamps \
      --timestamp-level char \
      --timestamp-language zh
    
    # Help
    voxcpm --help
    

    Web Demo

    python app.py --port 8808  # then open in browser: http://localhost:8808
    

    Use --device to choose the runtime device:

    python app.py --device auto
    

    Supported values are auto, cpu, mps, cuda, and cuda:N. On Apple Silicon Macs, auto uses MPS when available.

    🚢 Production Deployment (Nano-vLLM)

    For high-throughput serving, use Nano-vLLM-VoxCPM — a dedicated inference engine built on Nano-vLLM with concurrent request support and an async API.

    pip install nano-vllm-voxcpm
    
    from nanovllm_voxcpm import VoxCPM
    import numpy as np, soundfile as sf
    
    server = VoxCPM.from_pretrained(model="/path/to/VoxCPM", devices=[0])
    chunks = list(server.generate(target_text="Hello from VoxCPM!"))
    sf.write("out.wav", np.concatenate(chunks), 48000)
    server.stop()
    

    RTF as low as ~0.13 on NVIDIA RTX 4090 (vs ~0.3 with the standard PyTorch implementation), with support for batched concurrent requests and a FastAPI HTTP server. See the Nano-vLLM-VoxCPM repo for deployment details.

    🏭 Production Serving (vLLM-Omni)

    For production multi-tenant deployments, use vLLM-Omni — the official vLLM project's omni-modal extension with native VoxCPM2 support. PagedAttention KV cache, continuous batching, and a drop-in OpenAI-compatible /v1/audio/speech endpoint.

    # Install from source (latest main — vllm-omni is rapidly evolving)
    uv pip install vllm==0.19.0 --torch-backend=auto
    git clone https://github.com/vllm-project/vllm-omni.git && cd vllm-omni
    uv pip install -e .
    

    See the vLLM-Omni installation guide for other platforms (ROCm, XPU, MUSA, NPU) and Docker images.

    # Launch an OpenAI-compatible TTS server (--omni enables omni-modal serving)
    vllm serve openbmb/VoxCPM2 --omni --port 8000
    
    # Call it from any OpenAI client
    curl http://localhost:8000/v1/audio/speech \
      -H "Content-Type: application/json" \
      -d '{"model":"openbmb/VoxCPM2","input":"Hello from VoxCPM2 on vLLM-Omni!","voice":"default"}' \
      --output out.wav
    

    Built on the upstream vLLM scheduler, with batched concurrent requests, streaming chunk delivery, and multi-GPU deployment out of the box. See the VoxCPM2 example for full deployment recipes.

    📱 On-Device Inference (llama.cpp-omni)

    For on-device / edge deployment without Python, use llama.cpp-omni — a high-performance C++ inference engine built on llama.cpp, with native VoxCPM2 GGUF support on CPU / Metal / CUDA / Vulkan.

    1. Download GGUF weights from HuggingFace | ModelScope — you need one BaseLM (F16 or Q8_0) + the Acoustic file. Q8_0 halves the download with negligible quality loss.

    2. Build

    git clone https://github.com/tc-mb/llama.cpp-omni.git && cd llama.cpp-omni
    cmake -B build -DCMAKE_BUILD_TYPE=Release
    cmake --build build --target voxcpm2-cli -j
    

    CMake auto-detects Metal (macOS) or CUDA (Linux with NVIDIA GPU).

    3. Run

    # Basic TTS
    ./build/bin/voxcpm2-cli \
        -t "Hello, this is VoxCPM2 running through llama.cpp-omni." \
        -o output.wav VoxCPM2-BaseLM-Q8_0.gguf VoxCPM2-Acoustic-F16.gguf
    
    # Voice cloning (reference audio)
    ./build/bin/voxcpm2-cli \
        -t "Cloned voice." -r speaker.wav -o clone.wav \
        VoxCPM2-BaseLM-Q8_0.gguf VoxCPM2-Acoustic-F16.gguf
    
    # Ultimate cloning (reference audio + transcript)
    ./build/bin/voxcpm2-cli \
        -t "Target text." --prompt-wav speaker.wav --prompt-text "transcript of speaker.wav" \
        -o clone.wav VoxCPM2-BaseLM-Q8_0.gguf VoxCPM2-Acoustic-F16.gguf
    

    RTF ~1.76 (Q8_0) on Apple M4 Pro / Metal. Key flags: --cfg (guidance scale), --timesteps (CFM steps), --seed, --temperature, --stream. See the llama.cpp-omni repo and GGUF weights page for full details.

    Full parameter reference, multi-scenario examples, and voice cloning tips → Quick Start Guide | Usage Guide | Cookbook


    📦 Models & Versions

    VoxCPM2VoxCPM1.5VoxCPM-0.5B
    Status🟢 LatestStableLegacy
    Backbone Parameters2B0.6B0.5B
    Audio Sample Rate48kHz44.1kHz16kHz
    LM Token Rate6.25Hz6.25Hz12.5Hz
    Languages302 (zh, en)2 (zh, en)
    Cloning ModeIsolated Reference & ContinuationContinuation onlyContinuation only
    Voice Design✅——
    Controllable Voice Cloning✅——
    SFT / LoRA✅✅✅
    RTF (RTX 4090)~0.30~0.15~0.17
    RTF in Nano-VLLM (RTX 4090)~0.13~0.08~0.10
    VRAM~8 GB~6 GB~5 GB
    Weights🤗 HF / MS🤗 HF / MS🤗 HF / MS
    Technical ReportarXiv—arXiv ICLR 2026
    Demo PageAudio Samples—Audio Samples

    VoxCPM2 is built on a tokenizer-free, diffusion autoregressive paradigm. The model operates entirely in the latent space of AudioVAE V2, following a four-stage pipeline: LocEnc → TSLM → RALM → LocDiT, enabling rich expressiveness and 48kHz native audio output.

    For full architectural details, VoxCPM2-specific upgrades, and a model comparison table, see the Architecture Design.


    📊 Performance

    VoxCPM2 achieves state-of-the-art or comparable results on public zero-shot and controllable TTS benchmarks.

    Seed-TTS-eval

    Seed-TTS-eval WER(⬇)&SIM(⬆) Results (click to expand)

    ModelParametersOpen-Sourcetest-ENtest-ZHtest-Hard
    WER/%⬇SIM/%⬆CER/%⬇SIM/%⬆CER/%⬇SIM/%⬆
    MegaTTS30.5B❌2.7977.11.5279.0--
    DiTAR0.6B❌1.6973.51.0275.3--
    CosyVoice30.5B❌2.0271.81.1678.06.0875.8
    CosyVoice31.5B❌2.2272.01.1278.15.8375.8
    Seed-TTS-❌2.2576.21.1279.67.5977.6
    MiniMax-Speech-❌1.6569.20.8378.3--
    F5-TTS0.3B✅2.0067.01.5376.08.6771.3
    MaskGCT1B✅2.6271.72.2777.4--
    CosyVoice0.3B✅4.2960.93.6372.311.7570.9
    CosyVoice20.5B✅3.0965.91.3875.76.8372.4
    SparkTTS0.5B✅3.1457.31.5466.0--
    FireRedTTS0.5B✅3.8246.01.5163.517.4562.1
    FireRedTTS-21.5B✅1.9566.51.1473.6--
    Qwen2.5-Omni7B✅2.7263.21.7075.27.9774.7
    Qwen3-Omni30B-A3B✅1.39-1.07---
    OpenAudio-s1-mini0.5B✅1.9455.01.1868.523.3764.3
    IndexTTS21.5B✅2.2370.61.0376.57.1275.5
    VibeVoice1.5B✅3.0468.91.1674.4--
    HiggsAudio-v23B✅2.4467.71.5074.055.0765.6
    VoxCPM-0.5B0.6B✅1.8572.90.9377.28.8773.0
    VoxCPM1.50.8B✅2.1271.41.1877.07.7473.1
    MOSS-TTS✅1.8573.41.2078.8--
    Qwen3-TTS1.7B✅1.2371.71.2277.06.7674.8
    FishAudio S24B✅0.99-0.54-5.99-
    LongCat-Audio-DiT3.5B✅1.5078.61.0981.86.0479.7
    VoxCPM22B✅1.8475.30.9779.58.1375.3

    CV3-eval

    CV3-eval Multilingual WER/CER(⬇) Results (click to expand)

    Modelzhenhard-zhhard-enjakodeesfritru
    CosyVoice24.086.3212.5811.969.1319.7-----
    CosyVoice3-1.5B3.914.999.7710.557.575.696.434.4711.810.56.64
    Fish Audio S22.652.439.104.403.962.762.222.006.262.042.78
    VoxCPM23.655.008.558.485.965.694.773.809.854.255.21

    MiniMax-Multilingual-Test

    Minimax-MLS-test WER(⬇) Results (click to expand)

    LanguageMinimaxElevenLabsQwen3-TTSFishAudio S2VoxCPM2
    Arabic1.6651.666–3.50013.046
    Cantonese34.11151.513–30.67038.584
    Chinese2.25216.0260.9280.7301.136
    Czech3.8752.108–2.84024.132
    Dutch1.1430.803–0.9900.913
    English2.1642.3390.9341.6202.289
    Finnish4.6662.964–3.3302.632
    French4.0995.2162.8583.0504.534
    German1.9060.5721.2350.5500.679
    Greek2.0160.991–5.7402.844
    Hindi6.9625.827–14.64019.699
    Indonesian1.2371.059–1.4601.084
    Italian1.5431.7430.9481.2701.563
    Japanese3.51910.6463.8232.7604.628
    Korean1.7471.8651.7551.1801.962
    Polish1.4150.766–1.2601.141
    Portuguese1.8771.3311.5261.1401.938
    Romanian2.8781.347–10.74021.577
    Russian4.2813.8783.2122.4003.634
    Spanish1.0291.0841.1260.9101.438
    Thai2.70173.936–4.2302.961
    Turkish1.520.699–0.8700.817
    Ukrainian1.0820.997–2.3006.316
    Vietnamese0.8873.415–7.4103.307

    Minimax-MLS-test SIM(⬆) Results (click to expand)

    LanguageMinimaxElevenLabsQwen3-TTSFishAudio S2VoxCPM2
    Arabic73.670.6–75.079.1
    Cantonese77.867.0–80.583.5
    Chinese78.067.779.981.682.5
    Czech79.668.5–79.878.3
    Dutch73.868.0–73.080.8
    English75.661.377.579.785.4
    Finnish83.575.9–81.989.0
    French62.853.562.869.873.5
    German73.361.477.576.780.3
    Greek82.673.3–79.586.0
    Hindi81.873.0–82.185.6
    Indonesian72.966.0–76.380.0
    Italian69.957.981.774.778.0
    Japanese77.673.878.879.682.8
    Korean77.670.079.981.783.3
    Polish80.272.9–81.988.4
    Portuguese80.571.181.778.183.7
    Romanian80.969.9–73.379.7
    Russian76.167.679.279.081.1
    Spanish76.261.581.477.683.1
    Thai80.058.8–78.684.0
    Turkish77.959.6–83.587.1
    Ukrainian73.064.7–74.779.8
    Vietnamese74.336.9–74.080.6

    Internal 30-Language ASR Benchmark

    We additionally run an internal multilingual intelligibility benchmark with 30 languages × 500 samples. ASR transcription is evaluated via Gemini 3.1 Flash Lite API.

    Internal 30-Language ASR Benchmark (click to expand)

    LanguageMetricVoxCPM2Fish S2-Pro
    ar (Arabic)CER1.23%0.30%
    da (Danish)WER2.70%3.52%
    de (German)WER0.96%0.64%
    el (Greek)WER3.17%4.61%
    en (English)WER0.42%1.03%
    es (Spanish)WER1.33%0.64%
    fi (Finnish)WER2.24%2.80%
    fr (French)WER2.16%2.34%
    he (Hebrew)CER2.98%15.27%
    hi (Hindi)CER0.79%0.91%
    id (Indonesian)WER1.36%1.68%
    it (Italian)WER1.65%1.08%
    ja (Japanese)CER2.40%1.82%
    km (Khmer)CER2.05%75.15%
    ko (Korean)CER0.95%0.29%
    lo (Lao)CER1.90%87.40%
    ms (Malay)WER1.75%1.41%
    my (Burmese)CER1.42%85.27%
    nl (Dutch)WER1.25%1.68%
    no (Norwegian)WER2.49%3.76%
    pl (Polish)WER1.90%1.65%
    pt (Portuguese)WER1.48%1.49%
    ru (Russian)WER0.90%0.86%
    sv (Swedish)WER2.22%2.63%
    sw (Swahili)CER1.07%2.02%
    th (Thai)CER0.94%1.92%
    tl (Tagalog)WER2.63%4.00%
    tr (Turkish)WER1.65%1.65%
    vi (Vietnamese)WER1.56%5.56%
    zh (Chinese)CER0.92%1.02%
    Average (30 languages)1.68%-

    InstructTTSEval

    Instruction-Guided Voice Design Results (click to expand)

    ModelInstructTTSEval-ZHInstructTTSEval-EN
    APS⬆DSD⬆RP⬆APS⬆DSD⬆RP⬆
    Hume–––83.075.354.3
    VoxInstruct47.552.342.654.957.039.3
    Parler-tts-mini–––63.448.728.6
    Parler-tts-large–––60.045.931.2
    PromptTTS–––64.347.231.4
    PromptStyle–––57.446.430.9
    VoiceSculptor75.764.761.5–––
    Mimo-Audio-7B-Instruct75.774.361.580.677.659.5
    Qwen3TTS-12Hz-1.7B-VD85.281.165.182.982.468.4
    VoxCPM285.271.560.884.283.271.4

    ⚙️ Fine-tuning

    VoxCPM supports both full fine-tuning (SFT) and LoRA fine-tuning. With as little as 5–10 minutes of audio, you can adapt to a specific speaker, language, or domain.

    # LoRA fine-tuning (parameter-efficient, recommended)
    python scripts/train_voxcpm_finetune.py \
        --config_path conf/voxcpm_v2/voxcpm_finetune_lora.yaml
    
    # Full fine-tuning
    python scripts/train_voxcpm_finetune.py \
        --config_path conf/voxcpm_v2/voxcpm_finetune_all.yaml
    
    # WebUI for training & inference
    python lora_ft_webui.py   # then open http://localhost:7860
    

    Full guide → Fine-tuning Guide (data preparation, configuration, training, LoRA hot-swapping, FAQ)


    📚 Documentation

    Full documentation: voxcpm.readthedocs.io

    TopicLink
    Quick Start & InstallationQuick Start
    Usage Guide & CookbookUser Guide
    VoxCPM SeriesModels
    Fine-tuning (SFT & LoRA)Fine-tuning Guide
    FAQ & TroubleshootingFAQ

    🌟 Ecosystem & Community

    ProjectDescription
    Nano-vLLMHigh-throughput and Fast GPU serving
    vLLM-OmniOfficial vLLM omni-modal serving for VoxCPM2 — PagedAttention, OpenAI-compatible API
    llama.cpp-omniFull-duplex omni inference engine — VoxCPM2 GGUF on CPU / Metal / CUDA / Vulkan
    VoxCPM.cppGGML/GGUF: CPU, CUDA, Vulkan inference
    audio.cppggml-based unified C++ inference framework — CPU/CUDA/Vulkan/Metal, CLI & server, no Python
    VoxCPM-ONNXONNX export for CPU inference
    VoxCPMANEApple Neural Engine backend
    voxcpm_rsRust re-implementation
    ComfyUI-VoxCPMComfyUI node-based workflows
    ComfyUI_RH_VoxCPMFeature-complete ComfyUI workflow for VoxCPM 2 with multi-speaker generation, LoRA, and auto-ASR
    ComfyUI-VoxCPMTTSComfyUI TTS extension
    TTS WebUIBrowser-based TTS extension
    VoxaDubLocal-first video dubbing: speaker-aware VoxCPM2 dubbing, fully local Hy-MT2 translation, duration-aware synchronization, and experimental Live Dubbing

    See the full Ecosystem in the docs. Community projects are not officially maintained by OpenBMB. Built something cool? Open an issue or PR to add it!


    ⚠️ Risks and Limitations

    • Potential for Misuse: VoxCPM's voice cloning can generate highly realistic synthetic speech. It is strictly forbidden to use VoxCPM for impersonation, fraud, or disinformation. We strongly recommend clearly marking any AI-generated content.
    • Controllable Generation Stability: Voice Design and Controllable Voice Cloning results can vary between runs — you may try to generate 1~3 times to obtain the desired voice or style. We are actively working on improving controllability consistency.
    • Language Coverage: VoxCPM2 officially supports 30 languages. For languages not on the list, you are welcome to test directly or try fine-tuning on your own data. We plan to expand language coverage in future releases.
    • Usage: This model is released under the Apache-2.0 license. For production deployments, we recommend conducting thorough testing and safety evaluation tailored to your use case.

    📖 Citation

    If you find VoxCPM helpful, please consider citing our work and starring ⭐ the repository!

    @article{zhou2026voxcpm2,
      title   = {VoxCPM2 Technical Report},
      author  = {Zhou, Yixuan  and Zeng, Guoyang and Liu, Xin and Li, Xiang and Yu, Renjie and Gui, Jiancheng and Wu, Jiaheng and Wang, Ziyang and Shen, Xudong and Ye, Runchuan  and Zhang, Zhisheng and Zhou, Jiuyang and Bai, Bingsong and Sun, Weiyue and Deng, Mengyuan and Shi, Qundong and Wu, Zhiyong and Liu, Zhiyuan},
      journal = {arXiv preprint arXiv:2606.06928},
      year    = {2026},
    }
    
    @article{zhou2025voxcpm,
      title = {Voxcpm: Tokenizer-free TTS for context-aware speech generation and true-to-life voice cloning},
      author = {Zhou, Yixuan and Zeng, Guoyang and Liu, Xin and Li, Xiang and Yu, Renjie and Wang, Ziyang and Ye, Runchuan and Sun, Weiyue and Gui, Jiancheng and Li, Kehan and Wu, Zhiyong and Liu, Zhiyuan},
      journal = {arXiv preprint arXiv:2509.24650},
      year = {2025}
    }
    

    📄 License

    VoxCPM model weights and code are open-sourced under the Apache-2.0 license.

    🙏 Acknowledgments

    • DiTAR for the diffusion autoregressive backbone
    • MiniCPM-4 for the language model foundation
    • CosyVoice for the Flow Matching-based LocDiT implementation
    • DAC for the Audio VAE backbone
    • Our community users for trying VoxCPM, reporting issues, sharing ideas, and contributing—your support helps the project keep getting better

    Institutions

    ModelBest     THUHCSI

    ⭐ Star History

    Star History Chart