登录

wenet

wenet-e2eApache-2.0活跃维护

面向生产环境的端到端语音识别工具包，基于 PyTorch 提供 Conformer、Transformer 等模型，可直接训练和部署 ASR 服务。

Production First and Production Ready End-to-End Speech Recognition Toolkit

v3.1.0发布于 2024-05-23 03:25 UTC5,255star1,191fork同步于 2026-11-10 17:37 UTC
语音识别端到端推理服务ConformerPyTorch
对比 GitHub 官网asrautomatic-speech-recognitionconformere2e-modelsproduction-ready

怎么上手

由 README 提炼，执行前请确认
原文 
pip install git+https://github.com/wenet-e2e/wenet.git
wenet -m paraformer audio.wav
硬件要求按 README 说明

README 未给出具体显存要求；推理默认在 CPU 上运行，需显式传 --device cuda 才用 GPU，Turing 架构（如 Tesla T4）另有 dtype 与 attention-backend 说明；训练推荐 CUDA 12.1，另支持华为昇腾 NPU（需装 CANN）。

需要 CUDA昇腾 / Ascend
下载：源码 zip·GitHub Releases（安装包）本站不镜像安装包

项目说明

由 README 自动翻译提炼
没人工点评？写下你的经验

面向生产的端到端语音识别工具包，提供从训练到部署的全栈方案

面向生产环境的端到端语音识别工具包，基于 PyTorch 提供 Conformer、Transformer 等模型，可直接训练和部署 ASR 服务。

核心能力

  • 提供训练、推理到部署的全栈生产方案
  • 在多个公开语音数据集上达到 SOTA 效果
  • pip 安装即可用命令行或 Python API 转写音频
  • 内置 paraformer、firered、whisper 等预训练模型
  • 支持华为昇腾 NPU，并提供 torch-npu 依赖

容易踩的坑

pip 安装会拉取无版本上限的 torch，可能装到超出本机 GPU 驱动支持的新版本，装完需用 torch.cuda.is_available() 验证；命令行推理默认走 CPU，有 GPU 也要显式加 --device cuda；训练/部署需另建 conda 环境并装 sox，否则可能报 sox extension 不可用，x86 runtime 或 LM 还需用 cmake 3.14+ 单独编译。

内容由模型根据 README 生成，可能不完整或不准确，以官方文档为准。看原文 README

最近发版

5 个版本v3.1.0
全部 release
  1. v3.1.0最新正式版2024-05-23 · 2024-05-23 03:25 UTC

    ❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤

    What's Changed

  2. v3.0.1WeNet 3.0.12024-03-09 · 2024-03-09 06:50 UTC
    查看更新说明

    ❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤

    What's Changed

  3. v3.0.0WeNet 3.0.02024-01-25 · 2024-01-25 11:59 UTC
    查看更新说明

    ❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤

    New Features

    • Support full part of bestrq #1869, #2060
    • Support GPU Tlg streaming #1878
    • Support streaming ASR web demo #1888
    • Support k2 rnnt loss and delay penality #1909
    • Supports context biasing #1931, #1936
    • Support ZeroPrompt (not merged) #1943
    • Support M1 Mac onnxruntime #1953
    • Support ITN runtime #2001, #2042, #2246
    • Support wav2vec2 #2034, #2035
    • Support part of w2vbert training #2039
    • wenet cli #2047, #2054, #2075, #2082, #2088, #2087, https://github.com/wenet-e2e/wenet/pull/2098, https://github.com/wenet-e2e/wenet/pull/2101, #2122 (!! simple and fast !!) 🛫
    • Support E-Branchformer module #2013
    • Support deepspeed #1849, #2168, #2123 (!! big big big !!) 💯
    • LoRA support (not merged) #2049
    • support batch decoding for ctc_prefix_beam_search & attention_rescoring #2059 (!! simple and fast !!) 🛫
    • support ali-paraformer #2067, #2078, https://github.com/wenet-e2e/wenet/pull/2093, https://github.com/wenet-e2e/wenet/pull/2096, https://github.com/wenet-e2e/wenet/pull/2099, #2124, #2139, #2140, #2155, #2219, #2222, #2277, #2282, #2289, #2314, #2324
    • support Contrastive learning for unified models #2100
    • support context biasing with ac automaton #2128, #2136
    • support whisper arch #2141, #2157, #2196, #2313, #2322, #2323
    • Support gradient checkpointing for Conformer & Transformer (whisper) #2173, #2275
    • ssh-launcher for multi-node multi-gpu training #2180, #2265
    • u2++-lite training support #2202
    • support blank penalty #2278
    • support speaker in dataset #2292
    • Whisper inference support in cpp runtime #2320

    What's Changed

    • Upgrade libtorch CPU runtime with IPEX version #1893
    • Refine ctc alignment #1966
    • Use torchrun for distributed training #2020, #2021
    • Refine traning code #2055, #2103, #2123, #2248, #2252, #2253, #2270, #2286, #2288, #2312 (!! big changes !!) 🚀
    • mv all ctc functions to ctc_utils.py #2057 (!! big changes !!) 🚀
    • move search methods to search.py #2056 (!! big changes !!) 🚀
    • move all k2 related functions to k2 #2058
    • refactor and simplify decoding methods #2061, #2062
    • unify decode results of all decoding methods #2063
    • refactor(dataset): return dict instead of tuple #2106, #2111
    • init_model API changed #2116, #2216 (!! big changes !!) 🚀
    • move yaml saving to save_model() #2156
    • refine tokenizer #2165, #2186 (!! big changes !!) 🚀
    • deprecate wenetruntime #2194 (!! big changes !!) 🚀
    • use pre-commit to auto check and lint #2195
    • refactor(yaml): Config ctc/cmvn/tokenizer in train.yaml #2205, #2229, #2230, #2227, #2232 (!! big changes !!) 🚀
    • train with dict input #2242, #2243 (!! big changes !!) 🚀
    • [dataset] keep pcm for other task #2268
    • Updgrad torch to 2.x #2301 (!! big changes !!) 🚀
    • log everything to tensorboard #2307

    New Bug Fixes

    • Fix NST recipe #1863
    • Fix Librispeech fst dict #1929
    • Fix bug when make shard.list for *.flac #1933
    • Fix bug of transducer #1940
    • Avoid problem during model averaging when there is parameter-tying. #2113
    • [loss] set zero_infinity=True to ignore NaN or inf ctc_loss #2299
    • fix android #2303

    ❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤ Many thanks to all the contributors !!!!! I love u all.

  4. v2.2.1WeNet 2.2.12023-05-26 · 2023-05-26 05:08 UTC
    查看更新说明

    What's Changed

    • Add http server/client @aluminumbox #1670
    • Add Trt (Myelin) support for streaming ASR @yuekaizhang #1679
    • Support OpenVino @FionaZZ92 #1700
    • Support ONNX GPU export, add librispeech results, and fix V2 streaming decode issue for efficient conformer @zwglory #1701
    • Support ort backend in wenetruntime @xingchensong #1708
    • Support LFMMI @aluminumbox #1725
    • Support Paraformer @MrSupW & @robin1001 #1738 & #1749 & #1791 & #1795
    • Support part of bestrq @Mddct #1750 & #1754 & #1824
    • Remove concat after to simplify the code flow #1762 & #1763 & #1764
    • Add riva cuda tlg decoder @yuekaizhang #1773
    • Add CUDA TLG nbest and mbr decoding @yuekaizhang #1804
    • Support IPEX @ZailiWang #1816
    • Support Branchformer @kli017 #1845
    • Support GPU hotword @zwglory #1860
  5. v2.2.0WeNet 2.2.02023-01-15 · 2023-01-15 09:55 UTC
    查看更新说明

    What's Changed

    • support exporting squeezeformer to onnx (CPU & GPU) by @yygle in #1593 and #1634
    • support horizon x3 pi by @xingchensong in #1597
    • support noisy student training by @NevermoreCY in #1600
    • support efficient conformer by @zwglory in #1636
    • add blank scale for wfst decoding by @simonwang517 in #1646

用户点评

登录用户的真实使用记录，可按版本和硬件筛选
共 0 条

    还没有人点评。第一个说出真实体验的人最有价值。

    README 原文

    未翻译
    在 GitHub 打开

    WeNet

    License Python-Version WeChat

    Roadmap | Docs | Papers | Runtime | Pretrained Models | HuggingFace | Ask WeNet Guru

    We share Net together.

    Highlights

    • Production first and production ready: The core design principle, WeNet provides full stack production solutions for speech recognition.
    • Accurate: WeNet achieves SOTA results on a lot of public speech datasets.
    • Light weight: WeNet is easy to install, easy to use, well designed, and well documented.

    Install

    Install python package

    pip install git+https://github.com/wenet-e2e/wenet.git

    This pulls in torch>=1.13.0 with no upper bound, so pip may install a torch build newer than your installed GPU driver supports. If you plan to use a GPU, verify it after install:

    python -c "import torch; print(torch.cuda.is_available())"

    If this prints False on a machine with a GPU, torch silently falls back to CPU with no error. Reinstall a torch build matching your driver's CUDA version, e.g.:

    pip install torch==2.4.0+cu121 torchaudio==2.4.0+cu121 --index-url https://download.pytorch.org/whl/cu121 --force-reinstall

    Command-line usage (use -h for parameters):

    wenet -m paraformer audio.wav

    You can set -m with paraformer or firered or wenetspeech for chinese, and set it to whisper-large-v3 or whisper-large-v3-turbo for english.

    Note: this runs on CPU by default (--device defaults to cpu), even if a GPU is available. Pass --device cuda explicitly to use the GPU (Python API: wenet.load_model(model_name, device='cuda')). See python usage for GPU/Turing (e.g. Tesla T4) specific notes on dtype and attention-backend options.

    Python programming usage:

    import wenet

model = wenet.load_model('paraformer')
result = model.transcribe('audio.wav')
print(result.text)

    Please refer python usage for more command line and python programming usage.

    Install for training & deployment

    • Clone the repo
    git clone https://github.com/wenet-e2e/wenet.git

    conda create -n wenet python=3.10
conda activate wenet
conda install conda-forge::sox
    • Install CUDA: please follow this link, It's recommended to install CUDA 12.1
    • Install torch and torchaudio, It's recomended to use 2.2.2+cu121:
    pip install torch==2.2.2+cu121 torchaudio==2.2.2+cu121 -f https://download.pytorch.org/whl/torch_stable.html

    • Install CANN: please follow this link to install CANN toolkit and kernels.

    • Install WeNet with torch-npu dependencies:

    pip install -e .[torch-npu]
    • Related version control table:
    RequirementMinimumRecommend
    CANN8.0.RC2.alpha003latest
    torch2.1.02.2.0
    torch-npu2.1.02.2.0
    torchaudio2.1.02.2.0
    deepspeed0.13.2latest
    • Install other python packages
    pip install -r requirements.txt
pre-commit install  # for clean and tidy code
    • Frequently Asked Questions (FAQs)
    # If you encounter sox compatibility issues
RuntimeError: set_buffer_size requires sox extension which is not available.
# ubuntu
sudo apt-get install sox libsox-dev
# centos
sudo yum install sox sox-devel
# conda env
conda install  conda-forge::sox

    Build for deployment

    Optionally, if you want to use x86 runtime or language model(LM), you have to build the runtime as follows. Otherwise, you can just ignore this step.

    # runtime build requires cmake 3.14 or above
cd runtime/libtorch
mkdir build && cd build && cmake -DGRAPH_TOOLS=ON .. && cmake --build .

    Please see doc for building runtime on more platforms and OS.

    Discussion & Communication

    You can directly discuss on Github Issues.

    For Chinese users, you can also scan the QR code on the left to follow our official account of WeNet. We created a WeChat group for better discussion and quicker response. Please scan the personal QR code on the right, and the guy is responsible for inviting you to the chat group.

    Acknowledge

    1. We borrowed a lot of code from ESPnet for transformer based modeling.
    2. We borrowed a lot of code from Kaldi for WFST based decoding for LM integration.
    3. We referred EESEN for building TLG based graph for LM integration.
    4. We referred to OpenTransformer for python batch inference of e2e models.

    Citations

    @inproceedings{yao2021wenet,
title={WeNet: Production oriented Streaming and Non-streaming End-to-End Speech Recognition Toolkit},
author={Yao, Zhuoyuan and Wu, Di and Wang, Xiong and Zhang, Binbin and Yu, Fan and Yang, Chao and Peng, Zhendong and Chen, Xiaoyu and Xie, Lei and Lei, Xin},
  booktitle={Proc. Interspeech},
  year={2021},
  address={Brno, Czech Republic },
  organization={IEEE}
}

@article{zhang2022wenet,
  title={WeNet 2.0: More Productive End-to-End Speech Recognition Toolkit},
  author={Zhang, Binbin and Wu, Di and Peng, Zhendong and Song, Xingchen and Yao, Zhuoyuan and Lv, Hang and Xie, Lei and Yang, Chao and Pan, Fuping and Niu, Jianwei},
  journal={arXiv preprint arXiv:2203.15455},
  year={2022}
}

    相关项目

    同分类里 star 最高的项目
    对比全部 →
    VoiceStudio
    debpalash·Python

    本地运行的 ElevenLabs 开源替代品，支持声音克隆、语音设计、视频配音、听写、转写和有声书制作，覆盖 646 种语言。

    语音克隆+4语音推理
    107004+388012kv0.5.72026-11-10 16:38 UTC
    voicebox
    jamiepine·Python

    开源 AI 语音工作室，支持声音克隆、语音听写和语音生成，基于 Qwen3-TTS 与 Whisper，可在 CUDA 或 MLX 上本地运行。

    语音克隆+4语音推理
    60376+6237.6kv0.5.02026-10-07 01:17 UTC
    whisper.cpp
    ggml-org·C++

    OpenAI Whisper 语音识别模型的 C/C++ 移植版，无需 Python 和 GPU 依赖，在普通 CPU 上就能跑语音转文字。

    语音识别+4语音推理
    55071+846.3kv1.9.52026-11-10 12:09 UTC
    faster-whisper
    SYSTRAN·Python

    基于 CTranslate2 重实现的 Whisper 语音转文字推理引擎，通过量化与高效推理大幅提速并降低显存占用，适合本地部署转录。

    语音识别+4语音推理
    26650+782.2kv1.2.12026-11-10 10:13 UTC