Chatterbox-TTS-Server

Chatterbox TTS Server: OpenAI-Compatible API with Web UI, Large Text Handling & Built-in Voices

Self-host Resemble AI's Chatterbox open-source TTS family (Original + Multilingual + Turbo) behind an OpenAI‑compatible API and a modern Web UI. The complete lineup includes the original high-quality model, multilingual support for 23 languages, and Chatterbox‑Turbo—a streamlined 350M-parameter model with dramatically improved throughput and native paralinguistic tags like [laugh], [cough], and [chuckle] for more expressive voice agents and narration. Features voice cloning, large text processing via intelligent chunking, audiobook generation, and consistent, reproducible voices using built-in ready-to-use voices and a generation seed feature.

🚀 Try it now! Test the full TTS server with voice cloning and audiobook generation in Google Colab - no installation required! To use it, please run cells 1 through 4 one at a time. After running cell 4, click on the "https://localhost:8004" link that appears in the output, and your web browser will open the UI from the .colab.dev domain. Read the instructions here.

This server is based on the architecture and UI of our Dia-TTS-Server project but uses the distinct chatterbox-tts engine. Runs accelerated on NVIDIA (CUDA), AMD (ROCm), and Apple Silicon (MPS) GPUs, with a fallback to CPU. Make sure you also check our Kitten-TTS-Server project.

📦 Portable Mode (Windows): This application supports a fully portable installation — the entire folder, including Python and all dependencies, is self-contained. Copy it to a USB drive, share it as a zip, or move it anywhere. Just double-click start.bat — no Python installation needed on the target machine. Learn more →

---

🆕 What's New

🚀 v2.0.0 highlights (new)

v2.0 ships the complete Chatterbox family on every major GPU stack behind one OpenAI-compatible API and Web UI. The headline themes:

• DGX Spark / sm_121 support via the new docker-compose-cu130.yml (CUDA 13.0, PyTorch 2.10). RTX 30/40/50 keep using cu121 / cu128.
• AMD Strix Halo support via docker-compose-strixhalo.yml (ROCm 7.2, HSA_OVERRIDE_GFX_VERSION=11.0.0).
• Streaming /tts endpoint — opt-in stream: true parameter returns a StreamingResponse that flushes WAV bytes per chunk with 20 ms crossfades. Default behavior unchanged.
• Voice conditioning cache — repeated requests against the same reference voice skip re-encoding. Real latency win for batch / OpenAI-endpoint workflows.
• Opt-in BF16 inference — TTS_BF16=on (or =auto) converts T3 to bfloat16 and runs under autocast for ~40% throughput on bf16-capable GPUs. Default off to preserve existing behavior on upgrade.
• HTTPS / SSL — optional ssl_certfile and ssl_keyfile in config.yaml for direct HTTPS without a reverse proxy.
• Security: CWE-22 path traversal fixed on /tts and /v1/audio/speech voice file parameters. Traversal attempts return HTTP 400.
• New endpoints — /api/unload (release GPU memory without restart) and /v1/audio/voices (OpenAI-compatible voice listing).
• Dynamic language selector — the UI populates the language dropdown from SUPPORTED_LANGUAGES exposed by the multilingual engine.
• Chunker fix — stray dashes in narrative text no longer get treated as bullet items that swallow the rest of the paragraph (#144).

See the v2.0.0 release notes for the full list with contributor credits.

📦 Portable Mode for Windows (new)

• The launcher now offers Portable Mode for all Windows users during first-time setup — selected by default.
• Creates a fully self-contained installation: the entire project folder can be copied to a USB drive, zipped and shared, or moved anywhere on the filesystem.
• The recipient just double-clicks start.bat — no Python installation required on the target machine.
• Works with any system Python 3.10+, but the embedded runtime always uses Python 3.10 — the only fully supported version. If your system Python is 3.11+, Portable Mode is the easiest way to avoid dependency issues on Windows.
• Use --portable to skip the prompt and install in portable mode directly, or --no-portable for a standard virtual environment.
• Linux and macOS use standard virtual environments with Python 3.10. Portable Mode is not available on these platforms, so Python 3.10 must be installed on the system.

🌍 Chatterbox Multilingual support (new)

• Added full support for Chatterbox Multilingual, completing support for all three models in Resemble AI's Chatterbox family.
• Multilingual brings 23-language support including Arabic, Chinese, Danish, Dutch, English, Finnish, French, German, Greek, Hebrew, Hindi, Italian, Japanese, Korean, Malay, Norwegian, Polish, Portuguese, Russian, Spanish, Swedish, Swahili, and Turkish.
• Built on the same 0.5B-parameter architecture as the original Chatterbox with emotion exaggeration control and zero-shot voice cloning.
• Perfect for international projects, multilingual audiobooks, and voice agents serving global audiences.

⚡ Chatterbox‑Turbo support (new)

• Added full support for Chatterbox‑Turbo, Resemble AI's latest efficiency-focused Chatterbox model.
• Turbo is built on a streamlined 350M‑parameter architecture, designed to use less compute/VRAM while keeping high-fidelity output.
• Turbo distills the speech-token-to-mel "audio diffusion decoder" from 10 steps → 1 step, removing a major inference bottleneck.
• Resemble positions Turbo for real-time/agent workflows and highlights significantly faster-than-real-time performance on GPU (performance varies by hardware/settings).

🔁 Hot‑swappable TTS engines (UI)

• Added a new engine selector dropdown at the top of the Web UI.
• Instantly hot-swap between Original Chatterbox, Chatterbox Multilingual, and Chatterbox‑Turbo; the backend auto-loads the selected engine.
• All three models are hot-swappable—simply select from the dropdown and the backend automatically loads your choice without requiring restarts or configuration changes.
• All UI + API requests route through the active engine so you can A/B test quality, language support, and latency without changing client code.

🎭 Paralinguistic tags (Turbo)

• Turbo adds native paralinguistic tags you can write directly into your text, e.g. …calling you back [chuckle]….
• Supported tags include [laugh], [cough], and [chuckle], plus text-based prompting for reactions like sigh, gasp, and cough.
• Added new presets in ui/presets.yaml demonstrating paralinguistic prompting for agent-style scripts and expressive reads.

✅ Original Chatterbox remains first‑class

• The original Chatterbox model remains available, with support for high quality English language output, a 0.5B LLaMA backbone, emotion exaggeration control, and training on 0.5M hours of cleaned data.

🎯 Complete Chatterbox family support

You now have access to the entire Chatterbox lineup:

• Original Chatterbox — High-quality English output with emotion control (0.5B parameters, 0.5M hours training data)
• Chatterbox Multilingual — 23-language support with voice cloning and emotion control (0.5B parameters)
• Chatterbox Turbo — Fastest inference with paralinguistic tags like [laugh] and [cough] (350M parameters, 1-step diffusion)

Switching models is effortless: Simply select your preferred model from the engine selector dropdown at the top of the Web UI. No restarts, no configuration changes required—just instant hot-swapping to test quality, speed, and language support across the complete Chatterbox family.

🖥️ Installation fixes across all platforms

• All platforms: Chatterbox is now installed with --no-deps across all installation paths (CPU, NVIDIA, cu128, ROCm). This eliminates ONNX source build failures, torch version conflicts, and CMake errors that affected many users. Chatterbox's dependencies (conformer, diffusers, transformers, s3tokenizer, etc.) are now listed explicitly in each requirements file with onnx==1.16.0 pinned to guarantee pre-built wheels.
• Apple Silicon / MPS: Fixed Turbo model crash ("Cannot convert a MPS Tensor to float64 dtype") by forcing float32 in s3tokenizer and voice_encoder. Fix applied in the chatterbox-v2 fork and also as an automatic post-install patch in start.py for users of other chatterbox versions. Thanks to @jonas3245 (#93).
• Docker CPU: New lightweight Dockerfile.cpu based on python:3.10-slim instead of the 4GB+ NVIDIA CUDA base image. docker-compose-cpu.yml now uses this smaller image. Removed deprecated version tags from all docker-compose files.
• config.yaml: Default device changed from cuda to auto for correct auto-detection on all hardware (CUDA, MPS, CPU).
• Python version: Python 3.10 is required — it is the only version with pre-built wheels for all dependencies (torch, torchvision, ONNX). Python 3.11+ may fail due to missing wheels. The Windows launcher's Portable Mode handles this automatically by using an embedded Python 3.10 runtime.
• Blackwell (CUDA 12.8): Fixed requirements-nvidia-cu128.txt to properly install PyTorch 2.9.0 with CUDA 12.8 (sm_120 support) for RTX 5060 Ti, 5070, 5070 Ti, 5080, and 5090 GPUs. The Dockerfile.cu128 now correctly installs chatterbox with --no-deps to prevent PyTorch downgrade.
• AMD ROCm: Fixed ROCm installation by switching to PyTorch's official ROCm 6.1 wheel index (torch==2.5.1+rocm6.1), which resolves the previous torch==2.6.0 / torchaudio==2.5.1 version conflict. A new requirements-rocm-init.txt installs the ROCm PyTorch stack before other dependencies. Both Dockerfile.rocm and start.py now use a two-step install to prevent pip from replacing ROCm torch wheels with CPU-only versions.
• Thanks to community contributors in issues #20, #23, #44, #58, #64, #79, #89, #92, #93, #98, #105, #107, #109, #113, #114, #121, and #122 for testing and reporting solutions.

🧰 Automated launcher + easy updates

• New Automated Launcher (Windows + Linux) that creates/activates a venv, installs the right dependencies, downloads model files, starts the server, and opens the Web UI.
• Easy maintenance commands:
  • --upgrade to update code + dependencies.
  • --reinstall for a clean reinstall when environments get messy.

---

🗣️ Overview: Enhanced Chatterbox TTS Generation

The Chatterbox TTS model by Resemble AI provides capabilities for generating high-quality speech. This project builds upon that foundation by providing a robust FastAPI server that makes Chatterbox significantly easier to use and integrate.

🚀 Want to try it instantly? Launch the live demo in Google Colab - no installation needed!

The server expects plain text input for synthesis and we solve the complexity of setting up and running the model by offering:

• A modern Web UI for easy experimentation, preset loading, reference audio management, and generation parameter tuning.
• Multi-engine support (Original + Turbo): Choose the TTS engine directly in the Web UI, then generate via the same UI/API surface.
• Paralinguistic prompting (Turbo): Native tags like [laugh], [cough], and [chuckle] for natural non-speech reactions inside the same generated voice.
• Original Chatterbox strengths: High quality English output plus unique "emotion exaggeration control" and 0.5B LLaMA backbone.
• Multi-Platform Acceleration: Full support for NVIDIA (CUDA), AMD (ROCm), and Apple Silicon (MPS) GPUs, with an automatic fallback to CPU, ensuring you can run on any hardware.
• Large Text Handling: Intelligently splits long plain text inputs into manageable chunks based on sentence structure, processes them sequentially, and seamlessly concatenates the audio.
• 📚 Audiobook Generation: Perfect for creating complete audiobooks - simply paste an entire book's text and the server automatically processes it into a single, seamless audio file with consistent voice quality throughout.
• Predefined Voices: Select from curated, ready-to-use synthetic voices for consistent and reliable output without cloning setup.
• Voice Cloning: Generate speech using a voice similar to an uploaded reference audio file.
• Consistent Generation: Achieve consistent voice output across multiple generations or text chunks by using the "Predefined Voices" or "Voice Cloning" modes, optionally combined with a fixed integer Seed.
• Docker support for easy, reproducible containerized deployment on any platform.

This server is your gateway to leveraging Chatterbox's TTS capabilities seamlessly, with enhanced stability, voice consistency, and large text support for plain text inputs.

✨ Key Features of This Server

🔥 Live Demo Available:

• 🚀 One-Click Google Colab Demo: Try the full server with voice cloning and audiobook generation instantly in your browser - no local installation required!

This server application enhances the underlying chatterbox-tts engine with the following:

🚀 Core Functionality:

• Multi-Engine Support:
  • Choose between Original Chatterbox, Chatterbox Multilingual, and Chatterbox‑Turbo via a hot-swappable engine selector in the Web UI.
  • Original Chatterbox provides high-quality English output with emotion exaggeration control (0.5B parameters).
  • Chatterbox Multilingual offers 23-language support with voice cloning and emotion control (0.5B parameters).
  • Chatterbox Turbo delivers significantly faster inference with a streamlined 350M-parameter architecture and paralinguistic tags.
  • All three models are hot-swappable—simply select from the dropdown without restarts or config changes.
• Paralinguistic Tags (Turbo):
  • Write native tags like [laugh], [cough], and [chuckle] directly in your text when using Chatterbox‑Turbo.
  • New presets demonstrate paralinguistic prompting for agent-style scripts and expressive narration.
• Large Text Processing (Chunking):
  • Automatically handles long plain text inputs by intelligently splitting them into smaller chunks based on sentence boundaries.
  • Processes each chunk individually and seamlessly concatenates the resulting audio, overcoming potential generation limits of the TTS engine.
  • Ideal for audiobook generation - paste entire books and get professional-quality audiobooks with consistent narration.
  • Configurable via UI toggle ("Split text into chunks") and chunk size slider.
• Predefined Voices:
  • Allows usage of curated, ready-to-use synthetic voices stored in the ./voices directory.
  • Selectable via UI dropdown ("Predefined Voices" mode).
  • Provides reliable voice output without manual cloning setup.
• Voice Cloning:
  • Supports voice cloning using a reference audio file (.wav or .mp3).
  • The server processes the reference audio for the engine.
• Generation Seed: Added seed parameter to UI and API for influencing generation results. Using a fixed integer seed in combination with Predefined Voices or Voice Cloning helps maintain consistency.
• API Endpoint (/tts):
  • The primary API endpoint, offering fine-grained control over TTS generation.
  • Supports parameters for text, voice mode (predefined/clone), reference/predefined voice selection, chunking control (split_text, chunk_size), generation settings (temperature, exaggeration, CFG weight, seed, speed factor, language), and output format.
• UI Configuration Management: Added UI section to view/edit config.yaml settings (server, model, paths) and save generation defaults.
• Configuration System: Uses config.yaml for all runtime configuration, managed via config.py (YamlConfigManager). If config.yaml is missing, it's created with default values from config.py.
• Audio Post-Processing (Optional): Includes utilities for silence trimming, internal silence reduction, and (if parselmouth is installed) unvoiced segment removal to improve audio quality. These are configurable.
• UI State Persistence: Web UI now saves/restores text input, voice mode selection, file selections, and generation parameters (seed, chunking, sliders) in config.yaml (ui_state section).

🔧 General Enhancements:

• Easy Installation & Management:
  • 🚀 Automated Launcher (start.bat / start.sh) - One-command setup with automatic hardware detection
  • 🔧 Multiple GPU Support - NVIDIA CUDA 12.1, NVIDIA CUDA 12.8 (Blackwell), AMD ROCm, Apple MPS
  • 🔄 Easy Updates - Simple --upgrade and --reinstall commands
  • 📦 Portable Mode (Windows) - Self-contained, movable installation — copy to USB, share as zip, run anywhere without Python
  • 🎯 Skip Menu Options - Direct installation with --cpu, --nvidia, --nvidia-cu128, --rocm, --portable flags
• Performance: Optimized for speed and efficient VRAM usage on GPU.
• Web Interface: Modern, responsive UI for plain text input, parameter adjustment, preset loading, reference/predefined audio management, and audio playback.
• Model Loading: Uses ChatterboxTTS.from_pretrained() for robust model loading from Hugging Face Hub, utilizing the standard HF cache.
• Dependency Management: Clear requirements.txt.
• Utilities: Comprehensive utils.py for audio processing, text handling, and file management.

✅ Features Summary

• Core Chatterbox Capabilities (via Resemble AI Chatterbox):
  • 🗣️ High-quality single-speaker voice synthesis from plain text.
  • 🎤 Perform voice cloning using reference audio prompts.
  • 🎯 Complete model family: Original Chatterbox (English, emotion control), Chatterbox Multilingual (23 languages), and Chatterbox‑Turbo (fastest, paralinguistic tags).
  • 🔄 Hot-swappable engines: Switch between all three models instantly via dropdown—no restarts needed.
• Enhanced Server & API:
  • ⚡ Built with the high-performance FastAPI framework.
  • ⚙️ Custom API Endpoint (/tts) as the primary method for programmatic generation, exposing all key parameters.
  • 📄 Interactive API documentation via Swagger UI (/docs).
  • 🩺 Health check endpoint (/api/ui/initial-data also serves as a comprehensive status check).
• Advanced Generation Features:
  • 🔁 Hot-Swappable Engines: Switch between Original Chatterbox, Chatterbox Multilingual, and Chatterbox‑Turbo directly in the Web UI—no restarts required.
  • 🌍 Multilingual Support: 23 languages including Arabic, Chinese, French, German, Japanese, Spanish, and more via Chatterbox Multilingual.
  • 🎭 Paralinguistic Tags (Turbo): Native support for [laugh], [cough], [chuckle] and other expressive tags.
  • 📚 Large Text Handling: Intelligently splits long plain text inputs into chunks based on sentences, generates audio for each, and concatenates the results seamlessly. Configurable via split_text and chunk_size.
  • 📖 Audiobook Creation: Perfect for generating complete audiobooks from full-length texts with consistent voice quality and automatic chapter handling.
  • 🎤 Predefined Voices: Select from curated synthetic voices in the ./voices directory.
  • ✨ Voice Cloning: Simple voice cloning using an uploaded reference audio file.
  • 🌱 Consistent Generation: Use Predefined Voices or Voice Cloning modes, optionally with a fixed integer Seed, for consistent voice output.
  • 🔇 Audio Post-Processing: Optional automatic steps to trim silence, fix internal pauses, and remove long unvoiced segments/artifacts (configurable via config.yaml).
• Intuitive Web User Interface:
  • 🖱️ Modern, easy-to-use interface.
  • 🔁 Engine Selector: Hot-swap between Original Chatterbox, Chatterbox Multilingual, and Chatterbox‑Turbo with a simple dropdown—no restarts needed.
  • 💡 Presets: Load example text and settings dynamically from ui/presets.yaml.
  • 🎤 Reference/Predefined Audio Upload: Easily upload .wav/.mp3 files.
  • 🗣️ Voice Mode Selection: Choose between Predefined Voices or Voice Cloning.
  • 🎛️ Parameter Control: Adjust generation settings (Temperature, Exaggeration, CFG Weight, Speed Factor, Seed, etc.) via sliders and inputs.
  • 💾 Configuration Management: View and save server settings (config.yaml) and default generation parameters directly in the UI.
  • 💾 Session Persistence: Remembers your last used settings via config.yaml.
  • ✂️ Chunking Controls: Enable/disable text splitting and adjust approximate chunk size.
  • ⚠️ Warning Modals: Optional warnings for chunking voice consistency and general generation quality.
  • 🌓 Light/Dark Mode: Toggle between themes with preference saved locally.
  • 🔊 Audio Player: Integrated waveform player (WaveSurfer.js) for generated audio with download option.
  • ⏳ Loading Indicator: Shows status during generation.
• Flexible & Efficient Model Handling:
  • ☁️ Downloads models automatically from Hugging Face Hub using ChatterboxTTS.from_pretrained().
  • 🔄 Easily specify model repository via config.yaml.
  • 📄 Optional download_model.py script available to pre-download specific model components to a local directory (this is separate from the main HF cache used at runtime).
• Performance & Configuration:
  • 💻 GPU Acceleration: Automatically uses NVIDIA CUDA, Apple MPS, or AMD ROCm if available, falls back to CPU.
  • ⚙️ All configuration via config.yaml.
  • 📦 Uses standard Python virtual environments.
  • 📦 Portable Mode (Windows): Self-contained installation that can be copied, moved, or shared — no Python needed on the target machine.
• Docker Support:
  • 🐳 Containerized deployment via Docker and Docker Compose.
  • 🔌 NVIDIA GPU acceleration with Container Toolkit integration.
  • 💾 Persistent volumes for models (HF cache), custom voices, outputs, logs, and config.
  • 🚀 One-command setup and deployment (docker compose up -d).

🔩 System Prerequisites

• Operating System: Windows 10/11 (64-bit) or Linux (Debian/Ubuntu recommended).
• Python: Version 3.10 required (Download). Python 3.10 is the only version with pre-built wheels for all dependencies (torch, torchvision, ONNX, ONNXRuntime). Python 3.11+ is not supported — key dependencies lack pre-built wheels, causing build failures. On Windows, the launcher's Portable Mode automatically uses an embedded Python 3.10 runtime regardless of your system Python version. When using Portable Mode, Python is only needed on the machine where you first set up the application.
• Git: For cloning the repository (Download).
• Internet: For downloading dependencies and models from Hugging Face Hub.
• Disk Space: 10GB+ recommended (for dependencies and model cache).
• (Optional but HIGHLY Recommended for Performance):
  • NVIDIA GPU (CUDA 12.1): CUDA-compatible (Maxwell architecture or newer, RTX 20/30/40 series). Check NVIDIA CUDA GPUs.
  • NVIDIA GPU (CUDA 12.8): RTX 5090 or other Blackwell-based GPUs, driver version 570+.
  • NVIDIA Drivers: Latest version for your GPU/OS (Download).
  • AMD GPU: ROCm-compatible (e.g., RX 6000/7000 series). Check AMD ROCm GPUs.
  • AMD Drivers: Latest ROCm-compatible drivers for your GPU/OS (Linux only).
  • Apple Silicon: M1, M2, M3, M4, or newer Apple Silicon chips with macOS 12.3+ for MPS acceleration.
• (Linux Only):
  • libsndfile1: Audio library needed by soundfile. Install via package manager (e.g., sudo apt install libsndfile1).
  • ffmpeg: For robust audio operations (optional but recommended). Install via package manager (e.g., sudo apt install ffmpeg).

Hardware Compatibility Matrix

Hardware	Installation Option	Requirements File	Driver Requirement
CPU Only	--cpu	requirements.txt	None
NVIDIA RTX 20/30/40	--nvidia	requirements-nvidia.txt	525+
NVIDIA RTX 5090 / Blackwell (sm_120)	--nvidia-cu128	requirements-nvidia-cu128.txt (torch 2.9, CUDA 12.8)	570+
NVIDIA DGX Spark / GB10 (sm_121)	Docker only	requirements-nvidia-cu130.txt (torch 2.10, CUDA 13.0)	580+
AMD RX 6000/7000 (Linux)	--rocm	requirements-rocm.txt	ROCm 6.4+
AMD Strix Halo (Ryzen AI MAX+)	Docker only	requirements-strixhalo.txt (ROCm 7.2)	ROCm 7.2+
AMD RX 9000 series / RDNA4 (Linux)	Docker only	requirements-rdna4-init.txt (ROCm 7.2)	ROCm 7.2+
Apple Silicon (M1/M2/M3/M4)	Manual install	See Option 4	macOS 12.3+

---

💻 Installation and Setup

This project uses specific dependency files to ensure a smooth installation for your hardware. You can choose between the automated launcher (recommended for most users) or manual installation (for advanced users).

1. Clone the Repository

git clone https://github.com/devnen/Chatterbox-TTS-Server.git
cd Chatterbox-TTS-Server

---

🚀 Quick Start with Automated Launcher (Recommended)

The automated launcher handles virtual environment creation, hardware detection, dependency installation, and server startup - all in one step.

Windows

# Double-click start.bat or run from command prompt:
start.bat

Linux / macOS

# Make the launcher executable and run it
chmod +x start.sh
./start.sh

What Happens

1. The launcher checks your Python installation (3.10 required; 3.10+ needed to bootstrap Portable Mode on Windows)
2. On Windows: Offers Portable Mode (recommended) — creates a fully self-contained, movable installation. See Portable Mode for details. Use --portable to skip this prompt or --no-portable for a standard virtual environment.
3. Sets up the Python environment (portable or standard virtual environment)
4. Detects your GPU hardware (NVIDIA, AMD, or CPU-only)
5. Shows an installation menu with recommended option pre-selected:

══════════════════════════════════════════════════════════════
   Hardware Detection
══════════════════════════════════════════════════════════════

   NVIDIA GPU: Detected (NVIDIA GeForce RTX 4090)
   AMD GPU:    Not detected

══════════════════════════════════════════════════════════════
   Select Installation Type
══════════════════════════════════════════════════════════════

   [1] CPU Only
       No GPU acceleration - works on any system

   [2] NVIDIA GPU (CUDA 12.1) [DEFAULT]
       Standard for RTX 20/30/40 series

   [3] NVIDIA GPU (CUDA 12.8)
       For RTX 5090 / Blackwell GPUs only

   [4] AMD GPU (ROCm 6.4)
       For AMD GPUs on Linux

   Enter choice [2]: 

6. Press Enter to accept the recommended default, or type a number to select a different option
7. Dependencies are installed automatically (this may take several minutes on first run)
8. The server starts and displays the access URLs

Launcher Command-Line Options

Option	Description
--reinstall or -r	Remove existing installation and reinstall fresh (shows menu)
--upgrade or -u	Upgrade to latest version (keeps current hardware selection)
--cpu	Install CPU-only version (skip menu)
--nvidia	Install NVIDIA CUDA 12.1 version (skip menu)
--nvidia-cu128	Install NVIDIA CUDA 12.8 version for RTX 5090/Blackwell (skip menu)
--rocm	Install AMD ROCm version (skip menu)
--portable	Use portable Python environment on Windows (skip prompt)
--no-portable	Use standard virtual environment on Windows (skip prompt)
--verbose or -v	Show detailed installation output
--help or -h	Show help message

Examples:

# Skip menu and install NVIDIA CUDA 12.1 directly
python start.py --nvidia

# Reinstall with fresh dependencies
python start.py --reinstall

# Upgrade to latest version (keeps your hardware selection)
python start.py --upgrade

```bash
## Install with verbose output for troubleshooting
python start.py --reinstall --nvidia --verbose

## Install in portable mode (Windows) - skip prompt
python start.py --portable

## Switch from standard to portable mode
python start.py --reinstall --portable

## Force standard virtual environment (skip portable prompt)
python start.py --no-portable

Subsequent Runs

##### Subsequent Runs

After the first installation, simply run the launcher again to start the server:

```bash
# Windows
start.bat

# Linux/macOS
./start.sh

The launcher detects the existing installation and starts the server directly without reinstalling.

---

📦 Portable Mode (Windows)

On Windows, the launcher offers Portable Mode — a fully self-contained installation where the entire project folder, including its own private Python runtime and all dependencies, lives in one place. Unlike standard Python virtual environments (which use hardcoded absolute paths and break when moved), the portable installation uses only relative paths and works from any location.

What You Can Do With It

Once the first-time setup completes, the entire project folder can be:

• Copied to a different directory on the same machine — it still works.
• Copied to a USB drive and run on another Windows PC — no Python needed on that machine.
• Zipped up and shared with someone else — they unzip, double-click start.bat, and it runs.
• Moved anywhere on the filesystem without breaking anything.

The recipient doesn't need Python installed, doesn't need to run any setup, and doesn't need an internet connection to launch the server (see note about models below).

First Run vs. Subsequent Runs

• First run: The launcher sets up the portable Python environment and installs all dependencies (PyTorch, CUDA libraries, etc.). This requires an internet connection and takes a few minutes. The resulting folder will be several GB due to PyTorch and GPU libraries.
• Subsequent runs: The launcher detects the existing portable environment and starts the server in seconds. No internet needed, no setup repeated.

Note About Model Downloads

The TTS models (~2+ GB) are downloaded from Hugging Face Hub into a separate cache folder the first time the server starts on any machine. This is a one-time download per machine, independent of the portable application folder.

If you're sharing a portable installation, the recipient will need an internet connection for this initial model download on their first run. To pre-include models in the portable package, you can set the model_cache path in config.yaml to a folder inside the project directory, then include it when sharing.

When Does the Portable Prompt Appear?

During first-time setup on Windows, the launcher offers a choice:

• Portable Mode (recommended, default): Creates a self-contained python_embedded/ environment inside the project folder using Python 3.10. Any system Python 3.10+ can bootstrap this process, but the embedded runtime is always Python 3.10.
• Standard installation: Uses a regular Python virtual environment (venv/). Requires system Python 3.10 — Python 3.11+ will fail due to missing pre-built wheels for key dependencies.

If your system Python is 3.11+, Portable Mode is required on Windows because it uses an embedded Python 3.10 runtime, bypassing the dependency issues entirely.

You can skip the prompt entirely with command-line flags:

• python start.py --portable — go straight to portable mode
• python start.py --no-portable — go straight to standard venv
• python start.py --reinstall --portable — switch an existing installation to portable mode

Not Available on Linux / macOS

Portable Mode is Windows-specific. On Linux and macOS, the standard virtual environment is used. Python 3.10 is required on all platforms — the dependency compatibility issues (missing wheels for torchvision, ONNX) affect all operating systems, not just Windows.

GPU Drivers Still Required

Portable Mode includes Python and all Python packages, but GPU drivers are not included. If the target machine has an NVIDIA GPU and you want GPU acceleration, the appropriate NVIDIA drivers must be installed on that machine. CPU mode works without any drivers.

How it works under the hood:

• Portable Mode uses the official CPython 3.10.11 embeddable distribution from python.org — a minimal, self-contained Python runtime (~8 MB before dependencies are installed).
• The embeddable distribution's python310._pth file is patched to use relative paths (., .., Lib\site-packages). The .. entry resolves to the project root since python_embedded/ is always one level deep. This is what makes portability work — no absolute paths are written anywhere.
• There is no virtual environment activation step. No activate.bat, no activate.ps1, no hardcoded paths. The launcher simply runs python_embedded/python.exe directly.
• The launcher bootstraps pip via get-pip.py, explicitly installs setuptools (needed by the perth watermarking library at runtime), generates a sitecustomize.py for DLL search path configuration, and patches the TTS engine's watermarker initialization for resilience.
• The start.bat batch file finds any system Python to launch start.py, which then detects the existing python_embedded/ directory and uses it — regardless of which system Python version invoked the launcher.

---

📋 Manual Installation

For users who prefer manual control over the installation process.

2. Create a Python Virtual Environment

Using a virtual environment is crucial to avoid conflicts with other projects.

• Windows (PowerShell):

  python -m venv venv
  .\venv\Scripts\activate
  

• Linux (Bash):

  python3 -m venv venv
  source venv/bin/activate
  

  Your command prompt should now start with (venv).

3. Choose Your Installation Path

Pick one of the following commands based on your hardware. This single command will install all necessary dependencies with compatible versions.

---

Option 1: CPU-Only Installation

This is the most straightforward option and works on any machine without a compatible GPU.

# Make sure your (venv) is active
pip install --upgrade pip
pip install -r requirements.txt
pip install --no-deps git+https://github.com/devnen/chatterbox-v2.git@master

---

Option 2: NVIDIA GPU Installation (CUDA 12.1)

For users with NVIDIA GPUs. This provides the best performance for RTX 20/30/40 series.

Prerequisite: Ensure you have the latest NVIDIA drivers installed. Python 3.10 required (3.11+ is not supported — pre-built wheels for torchvision and ONNX are unavailable).

# Make sure your (venv) is active
pip install --upgrade pip
pip install -r requirements-nvidia.txt
pip install --no-deps git+https://github.com/devnen/chatterbox-v2.git@master

After installation, verify that PyTorch can see your GPU:

python -c "import torch; print(f'PyTorch version: {torch.__version__}'); print(f'CUDA available: {torch.cuda.is_available()}'); print(f'Device name: {torch.cuda.get_device_name(0) if torch.cuda.is_available() else None}')"

If CUDA available: shows True, your setup is correct!

---

Option 2b: NVIDIA GPU with CUDA 12.8 (RTX 5090 / Blackwell)

Note: Only use this if you have a Blackwell-based GPU (RTX 5060 Ti, 5070, 5070 Ti, 5080, 5090). For RTX 2000/3000/4000 series, use Option 2 above.

For users with NVIDIA Blackwell architecture GPUs that require CUDA 12.8 and sm_120 support.

Prerequisites:

• NVIDIA RTX 5060 Ti, 5070, 5070 Ti, 5080, 5090 or other Blackwell-based GPU
• CUDA 12.8+ drivers (driver version 570+)

Using Docker (Recommended for RTX 5090):

# Build and start with CUDA 12.8 support
docker compose -f docker-compose-cu128.yml up -d

# Access the web UI at http://localhost:8004

Manual Installation:

# Make sure your (venv) is active
pip install --upgrade pip

# Step 1: Install dependencies with PyTorch 2.9.0+cu128 (includes sm_120 support)
pip install -r requirements-nvidia-cu128.txt

# Step 2: Install chatterbox without dependencies (prevents PyTorch downgrade)
pip install --no-deps git+https://github.com/devnen/chatterbox-v2.git@master

⚠️ Critical: The --no-deps flag is required to prevent PyTorch from being downgraded to a version that doesn't support Blackwell GPUs.

After installation, verify that PyTorch supports sm_120:

python -c "import torch; print(f'PyTorch: {torch.__version__}'); print(f'CUDA: {torch.cuda.is_available()}'); print(f'GPU: {torch.cuda.get_device_name(0)}'); print(f'Architectures: {torch.cuda.get_arch_list()}')"

You should see sm_120 in the architectures list!

NVIDIA's Blackwell GPUs (RTX 5060 Ti, 5070, 5070 Ti, 5080, 5090) use compute capability sm_120. PyTorch 2.9.0 with CUDA 12.8 includes support for this architecture. Earlier versions (including CUDA 12.1) will fail with the error: CUDA error: no kernel image is available for execution on the device.

See README_CUDA128.md for detailed setup instructions and troubleshooting.

---

Option 2c: NVIDIA GPU with CUDA 13.0 (DGX Spark / sm_121)

Note: Use this for NVIDIA DGX Spark / GB10 hardware (compute capability sm_121). RTX 5090 stays on cu128 (Option 2b); RTX 30/40 stays on cu121 (Option 2).

For users on the very latest NVIDIA stack who need CUDA 13.0 and PyTorch 2.10.

Prerequisites:

• NVIDIA DGX Spark / GB10 or other sm_121-capable hardware
• CUDA 13.0+ drivers (driver version 580+)

Using Docker (recommended):

docker compose -f docker-compose-cu130.yml up -d

# Access the web UI at http://localhost:8004

The Dockerfile.cu130 installs PyTorch 2.10.0+cu130 with chatterbox-v2 via --no-deps so PyTorch is not downgraded.

---

Option 3: AMD GPU Installation (ROCm)

For users with modern, ROCm-compatible AMD GPUs.

Prerequisite: Ensure you have the latest ROCm drivers installed on a Linux system.

# Make sure your (venv) is active
pip install --upgrade pip

# Step 1: Install ROCm PyTorch stack first
pip install -r requirements-rocm-init.txt

# Step 2: Install remaining dependencies
pip install -r requirements-rocm.txt

# Step 3: Install chatterbox without dependencies (prevents ROCm torch overwrite)
pip install --no-deps git+https://github.com/devnen/chatterbox-v2.git@master

⚠️ Critical: The --no-deps flag on chatterbox-tts is required to prevent pip from replacing the ROCm PyTorch wheels with CPU-only versions from PyPI. The start.py launcher handles this automatically.

After installation, verify that PyTorch can see your GPU:

python -c "import torch; print(f'PyTorch version: {torch.__version__}'); print(f'ROCm available: {torch.cuda.is_available()}'); print(f'Device name: {torch.cuda.get_device_name(0) if torch.cuda.is_available() else None}')"

If ROCm available: shows True, your setup is correct!

ROCm installation uses a two-step process:

1. requirements-rocm-init.txt installs PyTorch from the official ROCm 6.1 wheel index (torch==2.5.1+rocm6.1), ensuring you get AMD GPU-accelerated builds.
2. requirements-rocm.txt installs remaining server dependencies without touching PyTorch.
3. Chatterbox is installed with --no-deps to prevent pip's dependency resolver from replacing the ROCm torch with a CPU-only version.

For APU/iGPU users: If you encounter "HIP error: invalid device function", you may need to set HSA_OVERRIDE_GFX_VERSION. See AMD ROCm Support Details below.

---

Option 4: Apple Silicon (MPS) Installation

For users with Apple Silicon Macs (M1, M2, M3, M4, etc.).

Prerequisite: Ensure you have macOS 12.3 or later for MPS support.

Step 1: Install PyTorch with MPS support first

# Make sure your (venv) is active
pip install --upgrade pip
pip install torch torchvision torchaudio

Step 2: Configure the server to use MPS Update your config.yaml to use MPS instead of CUDA:

tts_engine:
  device: mps  # Changed from 'cuda' to 'mps'

Step 3: Install remaining dependencies

# Install chatterbox-tts without its dependencies to avoid conflicts
pip install --no-deps git+https://github.com/devnen/chatterbox-v2.git@master

# Install core server dependencies
pip install fastapi 'uvicorn[standard]' librosa safetensors soundfile pydub audiotsm praat-parselmouth python-multipart requests aiofiles PyYAML watchdog unidecode inflect tqdm

# Install missing chatterbox dependencies
pip install conformer==0.3.2 diffusers==0.29.0 resemble-perth==1.0.1 transformers==4.46.3

# Install s3tokenizer without its problematic dependencies
pip install --no-deps s3tokenizer

# Install a compatible version of ONNX and audio codec
pip install onnx==1.16.0 descript-audio-codec

After installation, verify that PyTorch can see your GPU:

python -c "import torch; print(f'PyTorch version: {torch.__version__}'); print(f'MPS available: {torch.backends.mps.is_available()}'); print(f'Device will use: {\"mps\" if torch.backends.mps.is_available() else \"cpu\"}')"

If MPS available: shows True, your setup is correct!

---

Option 5: AMD Strix Halo (Ryzen AI MAX+) Installation

Note: Use this for AMD Strix Halo APUs (Ryzen AI MAX+ 395 / "Ryzen AI Max" with integrated Radeon 8060S, GFX 11.5.0). Standard discrete Radeon GPUs use Option 3 (ROCm).

Prerequisites:

• AMD Strix Halo APU on Linux
• ROCm 7.2+ stack installed on the host

Using Docker (recommended):

docker compose -f docker-compose-strixhalo.yml up -d

## Access the web UI at http://localhost:8004

The compose file sets HSA_OVERRIDE_GFX_VERSION=11.0.0 and HSA_XNACK=1 so the ROCm 7.2 wheels work on Strix Halo's GFX 11.5.0 silicon, and enables TTS_BF16=on for the throughput win on this hardware.

---

Option 6: AMD RDNA4 (RX 9070 / RX 9070 XT / R9700) Installation

Note: For AMD RDNA4 discrete GPUs (RX 9070, RX 9070 XT, Radeon AI PRO R9700, gfx1201). ROCm 6.1 wheels do not support gfx1201 — ROCm 7.2 is required. Standard discrete Radeon GPUs (RX 6000/7000) use Option 3.

Prerequisites:

• AMD RDNA4 GPU (RX 9070 / 9070 XT / R9700) on Linux
• ROCm 7.2+ stack installed on the host (see ROCm install guide)
• User in video and render groups: sudo usermod -aG video,render $USER

Using Docker (recommended):

docker compose -f docker-compose-rdna4.yml up -d

## Access the web UI at http://localhost:8004

The compose file sets ROCBLAS_USE_HIPBLASLT=0 (required for stability on gfx1201) and enables TTS_BF16=on for throughput gain. gfx1201 is natively supported by ROCm 7.0+ — no HSA_OVERRIDE_GFX_VERSION needed.

---

🚀 Live Demo - Try It Now! (Google Colab)

Want to test Chatterbox TTS Server immediately without any installation?

Why Try the Demo?

• ✅ Full Web UI with all controls and features
• ✅ Voice cloning with uploaded audio files
• ✅ Predefined voices included
• ✅ Large text processing with chunking (perfect for audiobooks)
• ✅ Free GPU acceleration (T4 GPU)
• ✅ No installation or setup required
• ✅ Works on any device with a web browser

Quick Start:

1. Click the badge above to open the notebook in Google Colab
2. Select GPU runtime: Runtime → Change runtime type → T4 GPU → Save
3. Run Cell 1: Click the play button to install dependencies (~1-5 minutes)
4. Run Cell 2: Start the server and access the Web UI via the provided links
5. Wait for "Server ready! Click below" message: Locate the "localhost:8004" link and click. This starts the Web UI in your browser
6. Generate speech: Use the web interface to create high-quality TTS audio

Notes:

• First run: Takes a few minutes to download models (one-time only)
• Session limits: Colab free tier has usage limits; sessions may timeout after inactivity
• For production: Use the local installation or Docker deployment methods below

---

Prefer local installation? Continue reading below for full setup instructions.

⚙️ Configuration

The server relies exclusively on config.yaml for runtime configuration.

• config.yaml: Located in the project root. This file stores all server settings, model paths, generation defaults, and UI state. It is created automatically on the first run (using defaults from config.py) if it doesn't exist. This is the main file to edit for persistent configuration changes.
• UI Configuration: The "Server Configuration" and "Generation Parameters" sections in the Web UI allow direct editing and saving of values into config.yaml.

Key Configuration Areas (in config.yaml or UI):

• server: host, port, logging settings.
• model: repo_id (e.g., "ResembleAI/chatterbox").
• tts_engine: device ('auto', 'cuda', 'mps', 'cpu'), predefined_voices_path, reference_audio_path, default_voice_id.
• paths: model_cache (for download_model.py), output.
• generation_defaults: Default UI values for temperature, exaggeration, cfg_weight, seed, speed_factor, language.
• audio_output: format, sample_rate, max_reference_duration_sec.
• ui_state: Stores the last used text, voice mode, file selections, etc., for UI persistence.
• ui: title, show_language_select, max_predefined_voices_in_dropdown.
• debug: save_intermediate_audio.

⭐ Remember: Changes made to server, model, tts_engine, or paths sections in config.yaml (or via the UI's Server Configuration section) require a server restart to take effect. Changes to generation_defaults or ui_state are applied dynamically or on the next page load.

Model selection: set model.repo_id to one of:

• chatterbox (or original) — Original 0.5B English model with emotion exaggeration.
• chatterbox-turbo (or turbo) — 350M Turbo model with paralinguistic tags ([laugh], [cough], [chuckle]).
• chatterbox-multilingual (or multilingual) — 0.5B multilingual model with 23-language support.

All three are hot-swappable from the Web UI engine dropdown without a server restart.

🔐 Security

• Voice file parameters on /tts (predefined_voice_id, reference_audio_filename) and /v1/audio/speech (voice) are sandboxed under their configured directories using utils.safe_resolve_within().
• Path traversal attempts (.., absolute paths, symlinks pointing outside the sandbox) return HTTP 400 with no filesystem access. The fix addresses CWE-22 and shipped in v2.0.0.
• Vulnerability reports: please use the repo's Security tab → Report a vulnerability for private disclosure rather than opening a public issue.

⚡ Performance tuning

The server defaults are tuned for safety and broad compatibility. The following knobs trade safety for speed when you understand your hardware:

• BF16 inference — set environment variable TTS_BF16=on (or =auto for "enable only if the GPU reports is_bf16_supported()") to convert T3 to bfloat16 and run generate() under autocast. Roughly 40% throughput on bf16-capable GPUs (RTX 30/40/50, A100, H100, Strix Halo). Default is off to preserve existing behavior on upgrade. Output is numerically slightly different from float32 but typically inaudible.
• Voice conditioning cache — repeated requests against the same reference voice skip re-encoding. Cache is keyed by (path, mtime, exaggeration) and is automatically cleared on reload_model() / /api/unload. No config needed.
• Chunk size — chunk_size parameter on /tts (50–500, default 120). Larger chunks = fewer requests but more VRAM per call. The chunker respects sentence boundaries either way.
• Streaming — stream: true on /tts for long-form input (audiobooks, multi-paragraph content). See the API section above for the chunk-level caveat.
• HTTPS — set server.ssl_certfile and server.ssl_keyfile in config.yaml for direct HTTPS without putting a reverse proxy in front.

▶️ Running the Server

Important Note on Model Downloads (First Run): The very first time you start the server, it needs to download the chatterbox-tts model files from Hugging Face Hub. This is an automatic, one-time process (per model version, or until your Hugging Face cache is cleared).

• ⏳ Please be patient: This download can take several minutes, depending on your internet speed and the size of the model files (typically a few gigabytes).
• 📝 Monitor your terminal: You'll see progress indicators or logs related to the download. The server will only become fully operational and accessible after these essential model files are successfully downloaded and loaded.
• ✔️ Subsequent starts will be much faster as the server will use the already downloaded models from your local Hugging Face cache.

You can optionally use the python download_model.py script to pre-download specific model components to the ./model_cache directory defined in config.yaml. However, please note that the runtime engine (engine.py) primarily loads the model from the main Hugging Face Hub cache directly, not this specific local model_cache directory.

Using the Automated Launcher (Recommended)

The easiest way to run the server is using the automated launcher:

Windows:

start.bat

Linux / macOS:

./start.sh

The launcher automatically:

• Activates the virtual environment
• Verifies the installation is complete
• Starts the server
• Waits for the server to be ready (including model download on first run)
• Displays the access URLs when ready

Manual Server Start

If you prefer to start the server manually:

Steps to Run:

1. Activate the virtual environment (if not activated):
   • Linux/macOS: source venv/bin/activate
   • Windows: .\venv\Scripts\activate
2. Run the server:

   python server.py
   

3. Access the UI: After the server starts (and completes any initial model downloads), it should automatically attempt to open the Web UI in your default browser. If it doesn't, manually navigate to http://localhost:PORT (e.g., http://localhost:8004 if your configured port is 8004).
4. Access API Docs: Open http://localhost:PORT/docs for interactive API documentation.
5. Stop the server: Press CTRL+C in the terminal where the server is running.

### 🔄 Updating to the Latest Version

Follow these steps to update your local installation to the latest version from GitHub. This guide provides multiple methods: using the automated launcher, the recommended `git stash` workflow, and a manual backup alternative. All methods preserve your local `config.yaml`.

**First, Navigate to Your Project Directory**

Before starting, open your terminal and go to the project folder.

```bash
cd Chatterbox-TTS-Server

---

Method 1: Using the Automated Launcher (Easiest)

The launcher provides simple upgrade functionality that handles everything automatically.

Upgrade (keeps your hardware selection):

# First, pull the latest code
git pull origin main

# Then upgrade dependencies using the launcher
# Windows
python start.py --upgrade

# Linux/macOS
python3 start.py --upgrade

Full Reinstall (choose new hardware option):

git pull origin main

# Windows
python start.py --reinstall

# Linux/macOS
python3 start.py --reinstall

The --upgrade flag preserves your current hardware selection (CPU, NVIDIA, etc.) and reinstalls dependencies.

The --reinstall flag removes the existing installation completely and shows the hardware selection menu again.

Changing Hardware Configuration:

To switch to a different hardware configuration (e.g., from CPU to NVIDIA, or from CUDA 12.1 to CUDA 12.8):

# Shows menu to select new hardware
python start.py --reinstall

# Or specify directly
python start.py --reinstall --nvidia
python start.py --reinstall --nvidia-cu128
python start.py --reinstall --cpu
python start.py --reinstall --rocm

---

Method 2: Stash and Pop (Recommended for Manual Installation)

If you installed manually without using the launcher, this is the standard and safest way to update using Git. It automatically handles your local changes (like to config.yaml) without needing to manually copy files.

First, activate your virtual environment:

# On Windows (PowerShell):
.\venv\Scripts\activate

# On Linux (Bash):
source venv/bin/activate

• Step 1: Stash Your Local Changes This command safely stores your modifications on a temporary "shelf."

  git stash
  

• Step 2: Pull the Latest Version Now that your local changes are safely stored, you can download the latest code from GitHub.

  git pull origin main
  

• Step 3: Re-apply Your Changes This command takes your changes from the shelf and applies them back to the updated code.

  git stash pop
  

  Your config.yaml will now have your settings, and the rest of the project files will be up-to-date. You can now proceed to the "Final Steps" section below.

---

Method 3: Manual Backup (Alternative)

This method involves manually backing up and restoring your configuration file.

First, activate your virtual environment:

# On Windows (PowerShell):
.\venv\Scripts\activate

# On Linux (Bash):
source venv/bin/activate

• Step 1: Backup Your Configuration ⚠️ Important: Create a backup of your config.yaml to preserve your custom settings.

  # Create a backup of your current configuration
  cp config.yaml config.yaml.backup
  

• Step 2: Update the Repository Choose one of the following commands based on your needs:

  • Standard Update (recommended):

    git pull origin main
    

    If you encounter merge conflicts with config.yaml, you may need to resolve them manually.
  • Force Update (if you have conflicts or want to ensure a clean update):

    # Fetch latest changes and reset to match remote exactly
    git fetch origin
    git reset --hard origin/main
    

• Step 3: Restore Your Configuration

  # Restore your backed-up configuration