whisper.net

Whisper.net

Open-Source Whisper.net

Dotnet bindings for OpenAI Whisper made possible by whisper.cpp

Build Status

Build type	Build Status
CI Status (Native + dotnet)

Getting Started

To install Whisper.net with all the available runtimes, run the following command in the Package Manager Console:

    PM> Install-Package Whisper.net.AllRuntimes

Or add a package reference in your .csproj file:

    <PackageReference Include="Whisper.net.AllRuntimes" Version="1.9.1" />

Whisper.net is the main package that contains the core functionality but does not include any runtimes. Whisper.net.AllRuntimes includes all available Whisper and Parakeet runtimes, including CUDA 13 and CUDA 12 GPU builds.

Installing Specific Runtimes

To install a specific runtime, you can install them individually and combine them as needed. For example, to install the CPU runtime, add the following package references:

    <PackageReference Include="Whisper.net" Version="1.9.1" />

    <PackageReference Include="Whisper.net.Runtime" Version="1.9.1" />

GPT for Whisper

We also have a custom-built GPT inside ChatGPT, which can help you with information based on this code, previous issues, and releases. Available here.

Please try to ask it before publishing a new question here, as it can help you a lot faster.

Runtimes Description

Whisper.net comes with multiple runtimes to support different platforms and hardware acceleration. Below are the available runtimes:

Whisper.net.Runtime

The default runtime that uses the CPU for inference. It is available on all platforms and does not require any additional dependencies.

Examples:

• Simple usage example
• Simple usage example (without Async processing)
• NAudio integration for mp3
• NAudio integration for resampled wav
• Simple channel diarization
• Blazor example

Pre-requisites

• Windows: Microsoft Visual C++ Redistributable for at least Visual Studio 2022 (x64) Download Link
• Windows 11 or Windows Server 2022 (or newer) is required
• Linux: libstdc++6, glibc 2.31
• macOS: TBD
• For x86/x64 platforms, the CPU must support AVX, AVX2, FMA and F16C instructions. If your CPU does not support these instructions, you'll need to use the Whisper.net.Runtime.NoAvx runtime instead.

Supported Platforms

• Windows x86, x64, ARM64
• Linux x64, ARM64, ARM
• macOS x64, ARM64 (Apple Silicon)
• Android
• iOS
• MacCatalyst
• tvOS
• WebAssembly

Whisper.net.Runtime.NoAvx

For CPUs that do not support AVX instructions.

Pre-requisites

• Windows: Microsoft Visual C++ Redistributable for at least Visual Studio 2022 (x64) Download Link
• Windows 11 or Windows Server 2022 (or newer) is required
• Linux: libstdc++6, glibc 2.31
• macOS: TBD

Supported Platforms

• Windows x86, x64, ARM64
• Linux x64, ARM64, ARM

Whisper.net.Runtime.Cuda

Contains the native whisper.cpp library with NVidia CUDA support enabled (built with the CUDA 13 toolchain).

Examples

• CUDA usage example

Pre-requisites

• Everything from Whisper.net.Runtime pre-requisites
• NVidia GPU with CUDA support
• CUDA Toolkit (>= 13.0.1)

Supported Platforms

• Windows x64
• Linux x64

Whisper.net.Runtime.Cuda12

Contains the native whisper.cpp library with NVidia CUDA support enabled, built against the CUDA 12 toolchain for systems that only provide CUDA 12.x drivers.

Examples

• CUDA usage example

Pre-requisites

• Everything from Whisper.net.Runtime pre-requisites
• NVidia GPU with CUDA support
• CUDA Toolkit (>= 12.4.1)

Supported Platforms

• Windows x64
• Linux x64

Whisper.net.Runtime.CoreML

Contains the native whisper.cpp library with Apple CoreML support enabled.

Examples:

• CoreML example

Supported Platforms

• macOS x64, ARM64 (Apple Silicon)
• iOS
• MacCatalyst

Whisper.net.Runtime.OpenVino

Contains the native whisper.cpp library with Intel OpenVino support enabled.

Examples

• OpenVino Example

Pre-requisites

• Everything from Whisper.net.Runtime pre-requisites
• OpenVino Toolkit (>= 2024.4)

Supported Platforms

• Windows x64
• Linux x64

Whisper.net.Runtime.Vulkan

Contains the native whisper.cpp library with Vulkan support enabled.

Examples

• Vulkan Example

Pre-requisites

• Everything from Whisper.net.Runtime pre-requisites
• Vulkan Toolkit (>= 1.4.321.1)]

Supported Platforms

• Windows x64

Parakeet runtimes

Parakeet uses a separate native engine and standalone package family. Install Whisper.net.Runtime.Parakeet for CPU inference, or the matching accelerated package:

Backend	Package
CPU	Whisper.net.Runtime.Parakeet
CPU without AVX	Whisper.net.Runtime.Parakeet.NoAvx
CUDA 13	Whisper.net.Runtime.Parakeet.Cuda
CUDA 12	Whisper.net.Runtime.Parakeet.Cuda12
Vulkan	Whisper.net.Runtime.Parakeet.Vulkan

Parakeet supports the same Windows, Linux, macOS, Android, Apple mobile, and WebAssembly platforms as the corresponding Whisper backend. CoreML and OpenVINO are Whisper-only.

Use the normal factory, builder, and processor workflow, but select the model family explicitly:

using var factory = WhisperFactory.FromPath(modelPath, new()
{
    ModelFamily = WhisperModelFamily.Parakeet,
});
using var processor = factory.CreateBuilder().Build();

Whisper remains the default because Whisper and Parakeet model files cannot be reliably distinguished from their file header. Translation, language selection/detection, prompts, beam search, temperature fallback, token timestamps, native print options, Flash Attention, DTW, CoreML, and OpenVINO are not supported by the Parakeet engine; explicitly configuring them throws NotSupportedException. Voice activity detection (WhisperVadFactory) is a Whisper-only feature that always loads the Whisper native engine regardless of WhisperFactoryOptions.ModelFamily, so it cannot be paired with the Parakeet engine.

Parakeet results use the shared SegmentData and WhisperToken types. SegmentData.Language is empty and NoSpeechProbability is float.NaN; Whisper-only token fields are zero.

See the Parakeet example for a complete transcription flow.

Multiple Runtimes Support

You can install and use multiple runtimes in the same project. The runtime will be automatically selected based on the platform you are running the application on and the availability of the native runtime.

The following order of priority will be used by default:

1. Whisper.net.Runtime.Cuda (NVidia devices with CUDA 13 drivers installed)
2. Whisper.net.Runtime.Cuda12 (NVidia devices with CUDA 12 drivers installed)
3. Whisper.net.Runtime.Vulkan (Windows x64 with Vulkan installed)
4. Whisper.net.Runtime.CoreML (Apple devices)
5. Whisper.net.Runtime.OpenVino (Intel devices)
6. Whisper.net.Runtime (CPU inference)
7. Whisper.net.Runtime.NoAvx (CPU inference without AVX support)

The loader automatically probes the CUDA runtimes in this order and validates the installed driver via cudaRuntimeGetVersion, so machines with only CUDA 12 drivers will transparently fall back to Whisper.net.Runtime.Cuda12.

To change the preferred runtime order, set RuntimeLibraryOrder on RuntimeOptions:

RuntimeOptions.RuntimeLibraryOrder =
[
    RuntimeLibrary.CoreML,
    RuntimeLibrary.OpenVino,
    RuntimeLibrary.Cuda,
    RuntimeLibrary.Cuda12,
    RuntimeLibrary.Cpu
];

Setting RuntimeLibraryOrder changes the preferred order, but compatibility checks are still performed before a runtime is loaded. If your application performs its own hardware and software detection, you can force a runtime and bypass those checks:

RuntimeOptions.ForcedRuntimeLibrary = RuntimeLibrary.Cpu;

Parakeet runtime selection is independent, so both engines can use different backends in the same process:

RuntimeOptions.ParakeetRuntimeLibraryOrder =
[
    RuntimeLibrary.Vulkan,
    RuntimeLibrary.Cpu
];
RuntimeOptions.ForcedParakeetRuntimeLibrary = RuntimeLibrary.Cpu;

Set ForcedRuntimeLibrary before creating any WhisperFactory. Only the specified runtime will be considered; native library path resolution and loading will still occur normally, but automatic checks such as CPU instruction-set and CUDA availability detection will be skipped. The application is responsible for ensuring the selected runtime is compatible. Forcing an incompatible runtime can cause native library loading failures or process crashes due to unsupported CPU instructions.

This override can also be used by Native AOT applications that perform their own runtime hardware detection, without raising the managed application's instruction-set baseline.

Pluggable native runtimes

• Whisper.net can run with any compatible compilation of the native whisper.cpp libraries; the package Whisper.net.Runtime is just one of the possible builds we publish.
• You may build your own native binaries (CPU, CUDA, CoreML, OpenVINO, Vulkan, NoAvx) and use them with Whisper.net as long as their files are arranged under ./runtimes in the same layout as our NuGet packages. The NativeLibraryLoader will probe them at runtime.
• For reproducible builds, you can use the attached GitHub workflows as references or entry points to produce artifacts: .github/workflows/ (e.g., dotnet.yml, dotnet-noavx.yml, dotnet-maui.yml). These workflows compile and package native libraries across platforms and can be adapted for your needs.

Versioning

Whisper.net follows semantic versioning.

Starting from version 1.8.0, Whisper.net does not follow the same versioning scheme as whisper.cpp, which creates releases based on specific commits in their master branch (e.g., b2254, b2255).

To track the whisper.cpp version used in a specific Whisper.net release, you can check the whisper.cpp submodule. The commit hash for the tag associated with the release will indicate the corresponding whisper.cpp version.

Ggml Models

Whisper.net uses Ggml models to perform speech recognition and translation. You can find more about Ggml models here.

For easier integration, Whisper.net provides a Downloader using Hugging Face.

var modelName = "ggml-base.bin";
if (!File.Exists(modelName))
{
    using var modelStream = await WhisperGgmlDownloader.Default.GetGgmlModelAsync(GgmlType.Base);
    using var fileWriter = File.OpenWrite(modelName);
    await modelStream.CopyToAsync(fileWriter);
}

The same downloader can fetch the ggml Silero VAD model used by WhisperVadFactory:

var vadModelName = "ggml-silero-v6.2.0.bin";
if (!File.Exists(vadModelName))
{
    using var modelStream = await WhisperGgmlDownloader.Default.GetGgmlSileroVadModelAsync();
    using var fileWriter = File.OpenWrite(vadModelName);
    await modelStream.CopyToAsync(fileWriter);
}

using var vadFactory = WhisperVadFactory.FromPath(vadModelName);

Parakeet TDT 0.6B v3 models are mirrored in sandrohanea/whisper.net at v5 in F32, F16, Q8_0, Q4_0, and Q4_K variants. All Whisper, Parakeet, encoder, and VAD downloads use this shared fixed version rather than an upstream main branch. Existing Whisper, encoder, and VAD model bytes are unchanged from v4, which remains available for older Whisper.net releases.

The mirrors preserve the converted files from ggml-org/parakeet-GGUF at 35156454d1a39de06863303dd209fd2bed6ee079. NVIDIA's original Parakeet TDT 0.6B v3 weights are licensed under CC-BY-4.0; the conversion repository declares MIT, which does not replace the original weights' attribution obligations. The mirror's provenance and checksum manifest records both sources. For example, the Q4_0 model is approximately 356 MB:

using var modelStream = await WhisperGgmlDownloader.Default.GetGgmlParakeetModelAsync(
    ParakeetModelType.Tdt0_6B_V3,
    ParakeetQuantizationType.Q4_0);
using var fileWriter = File.Create("ggml-parakeet-tdt-0.6b-v3-q4_0.bin");
await modelStream.CopyToAsync(fileWriter);

Models can also be supplied from managed sources without writing native code. Use FromStream, FromBuffer, or implement IWhisperModelLoader when the model comes from a custom source:

using var whisperModelStream = await GetWhisperModelStreamAsync();
using var whisperFactory = WhisperFactory.FromStream(whisperModelStream);

using var vadModelStream = await GetVadModelStreamAsync();
using var vadFactory = WhisperVadFactory.FromStream(vadModelStream);

The Silero VAD model can also be built from the whisper.cpp submodule:

python -m venv venv
.\venv\Scripts\Activate.ps1
pip install silero-vad
python .\whisper.cpp\models\convert-silero-vad-to-ggml.py --output .\whisper.cpp\models\silero.bin

The conversion script names the output with the Silero package version, for example silero-v6.2.0-ggml.bin. The current v5 snapshot includes this model as vad/ggml-silero-v6.2.0.bin. Future model updates require a new Hugging Face version tag and an update to the downloader's shared model version; do not move existing tags.

Environment variables for model downloads

• HF_TOKEN
  • Optional. If set, Whisper.net will add an Authorization header when downloading models from Hugging Face to avoid rate limiting.
  • Example:
    • Bash: export HF_TOKEN=hf_xxx
    • PowerShell: $env:HF_TOKEN = "hf_xxx"

Usage

using var whisperFactory = WhisperFactory.FromPath("ggml-base.bin");

using var processor = whisperFactory.CreateBuilder()
    .WithLanguage("auto")
    .Build();

using var fileStream = File.OpenRead(wavFileName);

await foreach (var result in processor.ProcessAsync(fileStream))
{
    Console.WriteLine($"{result.Start}->{result.End}: {result.Text}");
}

Grammar constraints

Use WithGrammar to guide Whisper decoding with GBNF rules, for example when recognizing a fixed set of voice commands:

using var processor = whisperFactory.CreateBuilder()
    .WithLanguage("ro")
    .WithGrammar("""
        root ::= " "* ("pornește" | "oprește" | "continuă") [ .!?]*
        """)
    .Build();

WithGrammar(gbnf, startRule: "root", penalty: 100f) accepts literals, Unicode character classes and ranges (including negated classes), rule references, groups, alternatives, and *, +, ? repetition. Rules are separated by newlines; # starts a comment. Invalid syntax, undefined rules, and left recursion are rejected before processing. The processor owns and releases native grammar memory.

Whisper subtracts the penalty from logits of incompatible tokens; this does not guarantee that every result is complete or valid. Restricting characters also does not guarantee a language when languages share an alphabet. Use a multilingual model and WithLanguage for foreign-language transcription; WithTranslate() requests English translation. Grammar constraints are unavailable for Parakeet.

Documentation

You can find the documentation and code samples here.

• Development environment setup notes are available in DEVELOPMENT.md.

Running tests

For instructions on running the test suites locally (including required .NET SDKs, optional environment variables like HF_TOKEN), see tests/README.md.

• Offline/local alternative: You can run tests fully locally without network by pre-downloading all ggml models required by tests and pointing tests to them via WHISPER_TEST_MODEL_PATH.
• MAUI tests use the Dotnet XHarness CLI to drive emulators/simulators. Docs: https://github.com/dotnet/xharness
• Native runtimes: By default, tests and are using the locally built native binaries instead, see “Building The Runtime” in DEVELOPMENT.md and ensure the output matches the expected runtimes layout.

License

MIT License. See LICENSE for details.