silero-vad

Silero VAD - pre-trained enterprise-grade Voice Activity Detector (also see our STT models).

https://user-images.githubusercontent.com/36505480/144874384-95f80f6d-a4f1-42cc-9be7-004c891dd481.mp4

Please note, that video loads only if you are logged in your GitHub account.

System requirements to run python examples on x86-64 systems:

• python 3.8+;
• 1G+ RAM;
• A modern CPU with AVX, AVX2, AVX-512 or AMX instruction sets.

Required:

• torch>=1.12.0.

Everything else is optional and installed via extras:

Extra	Installs	Needed for
silero-vad[audio]	torchaudio>=0.12.0,<2.10, plus torchcodec on Python 3.9+	read_audio / save_audio
silero-vad[codec]	torchcodec	read_audio / save_audio with no torchaudio at all
silero-vad[onnx-cpu]	onnxruntime>=1.16.1, numpy	ONNX and sequence models
silero-vad[onnx-gpu]	onnxruntime-gpu>=1.16.1, numpy	ONNX on GPU
silero-vad[all]	all of the above	everything

[audio] pulls in torchcodec on Python 3.9+ because torchaudio>=2.9 hands decoding over to it and does not work without it; on Python 3.8 pip resolves an older torchaudio that decodes on its own. torchcodec itself needs FFmpeg (4-9) on the system.

The model itself needs only torch — if you already load audio yourself, pass a 1-D float32 torch.Tensor straight to get_speech_timestamps and install nothing extra.

The bundled I/O helpers (read_audio, save_audio) work with either backend: torchaudio if present, otherwise torchcodec. Note that torchcodec resamples through FFmpeg rather than torchaudio.transforms.Resample, so results can differ by ~1e-3 on files that need resampling.

With torchaudio, a proper audio backend is required:

• Option №1 - FFmpeg backend. conda install -c conda-forge 'ffmpeg<7';
• Option №2 - sox_io backend. apt-get install sox, TorchAudio is tested on libsox 14.4.2;
• Option №3 - soundfile backend. pip install soundfile.

If you are planning to run the VAD using solely the onnx-runtime, it will run on any other system architectures where onnx-runtume is supported. In this case please note that:

• You will have to implement the I/O;
• You will have to adapt the existing wrappers / examples / post-processing for your use-case.

Using pip: pip install silero-vad[audio] (or plain pip install silero-vad if you load audio yourself)

from silero_vad import load_silero_vad, read_audio, get_speech_timestamps
model = load_silero_vad()
wav = read_audio('path_to_audio_file')
speech_timestamps = get_speech_timestamps(
  wav,
  model,
  return_seconds=True,  # Return speech timestamps in seconds (default is samples)
)

Using torch.hub:

import torch
torch.set_num_threads(1)

model, utils = torch.hub.load(repo_or_dir='snakers4/silero-vad', model='silero_vad')
(get_speech_timestamps, _, read_audio, _, _) = utils

wav = read_audio('path_to_audio_file')
speech_timestamps = get_speech_timestamps(
  wav,
  model,
  return_seconds=True,  # Return speech timestamps in seconds (default is samples)
)

• Stellar accuracy

  Silero VAD has excellent results on speech detection tasks.

• Fast

  One audio chunk (30+ ms) takes less than 1ms to be processed on a single CPU thread. Using batching or GPU can also improve performance considerably. Under certain conditions ONNX may even run up to 4-5x faster.

• Lightweight

  JIT model is around two megabytes in size.

• General

  Silero VAD was trained on huge corpora that include over 6000 languages and it performs well on audios from different domains with various background noise and quality levels.

• Flexible sampling rate

  Silero VAD supports 8000 Hz and 16000 Hz sampling rates.

• Highly Portable

  Silero VAD reaps benefits from the rich ecosystems built around PyTorch and ONNX running everywhere where these runtimes are available.

• No Strings Attached

  Published under permissive license (MIT) Silero VAD has zero strings attached - no telemetry, no keys, no registration, no built-in expiration, no keys or vendor lock.

• Voice activity detection for IOT / edge / mobile use cases
• Data cleaning and preparation, voice detection in general
• Telephony and call-center automation, voice bots
• Voice interfaces

• Examples and Dependencies
• Quality Metrics
• Performance Metrics
• Versions and Available Models
• Further reading
• FAQ

Try our models, create an issue, start a discussion, join our telegram chat, email us, read our news.

Please see our wiki for relevant information and email us directly.

Citations

@misc{Silero VAD,
  author = {Silero Team},
  title = {Silero VAD: pre-trained enterprise-grade Voice Activity Detector (VAD), Number Detector and Language Classifier},
  year = {2024},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/snakers4/silero-vad}},
  commit = {insert_some_commit_here},
  email = {[email protected]}
}

• Example of VAD ONNX Runtime model usage in C++

• Example of VAD ExecuTorch model usage in C++

• Voice activity detection for the browser using ONNX Runtime Web

• Rust, Rust (wavekat-vad), Go, Java, C++, C# and other community examples

• OpenVINO conversion guidelines

• Optional offline ONNX sequence inference for long recordings

• A tinygrad model with a pico example in the docsting + separate weights in safetensors format (for simplicity we provided just the 16k model)