Kokoro-FastAPI

Dockerized FastAPI wrapper for Kokoro-82M text-to-speech model. Generate hours of high quality speech in minutes.

• OpenAI-compatible Speech + Realtime (TTS only) endpoints, multi-language support
  • English (US/GB), Spanish, French, Hindi, Italian, Japanese, Brazilian Portuguese, Mandarin Chinese
• ElevenLabs and Cartesia compatible websockets for voice agents (LiveKit, Pipecat)
• Custom voicepacks via Inno Clone-Tuner
• Optional integrated WebUI; read-along long-generation
• Inline multi-speaker generation, weighted voice mixing with short aliases, SSML support
• Per-word, or per-chunk timestamped caption generation
• Phoneme endpoints: generate phonemes from text, or generate audio from phonemes
• Prebuilt multiplatform images
  • CPU and NVIDIA GPU (CUDA): linux/amd64 + linux/arm64
  • AMD GPU (ROCm, experimental): linux/amd64 only
• Apple Silicon (MPS) supported when running directly via UV (no image)

Integration & Guides

• Home Assistant
  • wyoming_openai, openai_tts, Kokoro-TTS, Kokoro-FastAPI App
• App stores and templates
  • Umbrel, Unraid Apps, GPUStack, jetson-containers
• Readers and audiobooks
  • openreader, epub_to_audiobook, audiobook-creator, Zotero-OpenReader
• Assistants and agents
  • xiaozhi-esp32-server, call-me, agent-cli, voice-chat-ai
• Browser
  • kokoro-extension, customtts

Get Started

Pre-built multi-arch images with models baked in.

:latest is available, but please pin to a release tag for stable usage.

No GPU (laptop, CPU-only server)

docker run -p 8880:8880 ghcr.io/remsky/kokoro-fastapi-cpu:latest

NVIDIA (GTX 900-series through RTX 40; ships cu126)

docker run --gpus all -p 8880:8880 ghcr.io/remsky/kokoro-fastapi-gpu:latest

NVIDIA RTX 50-series / Blackwell (ships cu128)

docker run --gpus all -p 8880:8880 ghcr.io/remsky/kokoro-fastapi-gpu:latest-cu128

NVIDIA arm64 (Jetson, GH200; same tag, ships cu129)

docker run --gpus all -p 8880:8880 ghcr.io/remsky/kokoro-fastapi-gpu:latest

AMD GPU (ROCm, experimental, x86_64 only)

docker run --device=/dev/kfd --device=/dev/dri -p 8880:8880 ghcr.io/remsky/kokoro-fastapi-rocm:latest

Apple Silicon (native MPS clone; the CPU image also works)

./start-gpu_mac.sh

gpu:latest is the same image as gpu:latest-cu126. Configuration via environment variables, see the configuration guide.

1. Install prerequisites, and start the service using Docker Compose (Full setup including UI):
   • Install Docker
   • Clone the repository:

     git clone https://github.com/remsky/Kokoro-FastAPI.git
     cd Kokoro-FastAPI
     
     cd docker/gpu   # For NVIDIA GPU support
     # or cd docker/cpu   # For CPU support
     # or cd docker/rocm  # For AMD GPU (ROCm, experimental, amd64 only)
     docker compose up --build
     
     # *Note for Apple Silicon (M1/M2/M3) users:
     # The Docker GPU image is CUDA-only and won't run on Apple Silicon. With Docker, use `docker/cpu`.
     # For native MPS (Apple GPU) acceleration, run directly via UV with `./start-gpu_mac.sh`.
     
     cd ../..  # back to repo root for the paths below
     
     # Models will auto-download, but if needed you can manually download:
     python docker/scripts/download_model.py --output api/src/models/v1_0
     

Configuration guide covers image vs build, the volume mounts, and env vars.

1. Install prerequisites:

   • Install astral-uv

   • Install espeak-ng in your system if you want it available as a fallback for unknown words/sounds. The upstream libraries may attempt to handle this, but results have varied.

   • Clone the repository:

     git clone https://github.com/remsky/Kokoro-FastAPI.git
     cd Kokoro-FastAPI
     

     Run the model download script if you haven't already

     Start directly via UV (with hot-reload)

     Linux and macOS

     ./start-cpu.sh OR
     ./start-gpu.sh 
     

     Windows

     .\start-cpu.ps1 OR
     .\start-gpu.ps1 
     

Run locally as an OpenAI-Compatible Speech Endpoint

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8880/v1", api_key="not-needed"
)

with client.audio.speech.with_streaming_response.create(
    model="kokoro",
    voice="af_sky+af_bella", #single or multiple voicepack combo
    input="Hello world!"
  ) as response:
      response.stream_to_file("output.mp3")

• The API will be available at http://localhost:8880

• API Documentation: http://localhost:8880/docs

• Web Interface: http://localhost:8880/web

Features

Core

# Using OpenAI's Python library
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8880/v1", api_key="not-needed")
response = client.audio.speech.create(
    model="kokoro",  
    voice="af_bella+af_sky", # see /api/src/core/openai_mappings.json to customize
    input="Hello world!",
    response_format="mp3"
)

response.stream_to_file("output.mp3")

Or Via Requests:

import requests


response = requests.get("http://localhost:8880/v1/audio/voices")
voices = [v["id"] for v in response.json()["voices"]]

# Generate audio
response = requests.post(
    "http://localhost:8880/v1/audio/speech",
    json={
        "model": "kokoro",  
        "input": "Hello world!",
        "voice": "af_bella",
        "response_format": "mp3",  # Supported: mp3, wav, opus, flac, aac, pcm
        "speed": 1.0
    }
)

# Save audio
with open("output.mp3", "wb") as f:
    f.write(response.content)

Quick tests (run from another terminal):

python examples/assorted_checks/test_openai/test_openai_tts.py # Test OpenAI Compatibility
python examples/assorted_checks/test_voices/test_all_voices.py # Test all available voices

For Realtime TTS clients (wyoming_openai TTS_REALTIME_MODELS etc). Voice names map the same as on the Speech endpoint.

import asyncio
import base64
import wave

from openai import AsyncOpenAI


async def main():
    client = AsyncOpenAI(api_key="x", base_url="http://localhost:8880/v1")
    audio = bytearray()
    async with client.realtime.connect(model="kokoro") as conn:
        await conn.session.update(session={"type": "realtime", "audio": {"output": {"voice": "af_heart"}}})
        await conn.response.create(response={"conversation": "none", "input": [
            {"type": "message", "role": "user", "content": [{"type": "input_text", "text": "Hello there."}]}
        ]})
        async for event in conn:
            if event.type == "response.output_audio.delta":
                audio += base64.b64decode(event.delta)
            elif event.type == "response.done":
                break

    with wave.open("output.wav", "wb") as f:
        f.setnchannels(1)
        f.setsampwidth(2)
        f.setframerate(24000)
        f.writeframes(audio)


asyncio.run(main())

• Text is read as written, instructions are ignored
• Speech endpoint fields go in session.update or response.create:
  • voice_aliases
  • lang_code
  • volume_multiplier
  • normalization_options
• Audio: PCM at 24000 Hz only
• response.cancel stops a response. Audio input and conversation events get an error back
• OpenAI SDK 2.x forces wss://, and requires you pass websocket_base_url="ws://localhost:8880/v1"

ElevenLabs (multi-stream-input) and Cartesia (/tts/websocket) compatible websockets, so voice agent frameworks can use their existing plugins. Voice ids are Kokoro voice names, API keys are ignored unless API_KEYS is set.

# LiveKit Agents
from livekit.plugins import cartesia, elevenlabs
tts = elevenlabs.TTS(base_url="http://localhost:8880/v1", api_key="x", voice_id="af_heart")
tts = cartesia.TTS(base_url="http://localhost:8880", api_key="x", voice="am_michael")

# Pipecat
from pipecat.services.cartesia.tts import CartesiaTTSService
from pipecat.services.elevenlabs.tts import ElevenLabsTTSService
tts = ElevenLabsTTSService(url="ws://localhost:8880", api_key="x", voice_id="af_heart")
tts = CartesiaTTSService(url="ws://localhost:8880/tts/websocket", api_key="x", voice_id="am_michael")

• Audio: PCM at 8000 to 48000 Hz, plus MP3 on the ElevenLabs route
• Word timestamps where Kokoro has them (English, espeak languages)
• Cartesia cancel stops generation for that context. ElevenLabs close_context finishes what was already sent
• Speech endpoint fields go on the message that opens a context (voice tags are not supported):
  • voice_aliases
  • lang_code
  • volume_multiplier
  • normalization_options
• ENABLE_WEBSOCKET_TTS=false turns these and the Realtime route off

# OpenAI-compatible streaming
from openai import OpenAI
client = OpenAI(
    base_url="http://localhost:8880/v1", api_key="not-needed")

# Stream to file
with client.audio.speech.with_streaming_response.create(
    model="kokoro",
    voice="af_bella",
    input="Hello world!"
) as response:
    response.stream_to_file("output.mp3")

# Stream to speakers (requires PyAudio)
import pyaudio
player = pyaudio.PyAudio().open(
    format=pyaudio.paInt16, 
    channels=1, 
    rate=24000, 
    output=True
)

with client.audio.speech.with_streaming_response.create(
    model="kokoro",
    voice="af_bella",
    response_format="pcm",
    input="Hello world!"
) as response:
    for chunk in response.iter_bytes(chunk_size=1024):
        player.write(chunk)

Or via requests:

import requests

response = requests.post(
    "http://localhost:8880/v1/audio/speech",
    json={
        "input": "Hello world!",
        "voice": "af_bella",
        "response_format": "pcm"
    },
    stream=True
)

for chunk in response.iter_content(chunk_size=1024):
    if chunk:
        # Process streaming chunks
        pass

Key Streaming Metrics:

• First token latency @ chunksize
  • ~300ms (GPU) @ 400
  • ~3500ms (CPU) @ 200 (older i7)
  • ~<1s (CPU) @ 200 (M3 Pro)
• Adjustable chunking settings for real-time playback

Note: Artifacts in intonation can increase with smaller chunks

• mp3
• wav
• opus
• flac
• aac
• pcm

Voices

• Weighted voice combinations using ratios (e.g., "af_bella(2)+af_heart(1)" for 67%/33% mix)
• Ratios are automatically normalized to sum to 100%
• Available through any endpoint by adding weights in parentheses
• Saves generated voicepacks for future use

Combine voices and generate audio:

import requests
response = requests.get("http://localhost:8880/v1/audio/voices")
voices = [v["id"] for v in response.json()["voices"]]

# Weighted voice combination (67%/33% mix)
response = requests.post(
    "http://localhost:8880/v1/audio/speech",
    json={
        "input": "Hello world!",
        "voice": "af_bella(2)+af_sky(1)",  # 2:1 ratio = 67%/33%
        "response_format": "mp3"
    }
)

# Download combined voice as .pt file
response = requests.post(
    "http://localhost:8880/v1/audio/voices/combine",
    json="af_bella(2)+af_sky(1)"  # 2:1 ratio = 67%/33%
)

# Save the .pt file
with open("combined_voice.pt", "wb") as f:
    f.write(response.content)

# Use the downloaded voice file
response = requests.post(
    "http://localhost:8880/v1/audio/speech",
    json={
        "input": "Hello world!",
        "voice": "combined_voice",  # Use the saved voice file
        "response_format": "mp3"
    }
)

Weighted mixes can get long fast. voice_aliases maps a short name per request, for both the voice field and [voice:...] tags:

• Names match case-insensitively, but an exact match wins. Keep them lowercase to avoid surprises.
• An alias pointing at a nonexistent voice returns a 400.
• The web UI's cast exports as {"voice_aliases": {...}}, which can be sent as-is in API calls.

from openai import OpenAI
client = OpenAI(base_url="http://localhost:8880/v1", api_key="not-needed")

client.audio.speech.create(
    model="kokoro",
    voice="narrator",
    input="[voice:narrator] Once upon a time. [voice:villain] Never!",
    extra_body={
        "allow_voice_tags": True,
        "voice_aliases": {"narrator": "af_bella(2)+af_sky", "villain": "am_michael"},
    },
)

POST /dev/tune takes a 3 to 30 s clip of one English speaker and returns speech in a voice tuned toward it, via inno-kokoro. It is a tuner, not a cloner. Expect a similar voice, not a match. Only tune voices you have permission to use.

Off by default. ENABLE_INNO_TUNER=true enables the route and a Tune tab in the web player.

import json
import requests

url = "http://localhost:8880/dev/tune"

# Speak text in the tuned voice
with open("ref.wav", "rb") as ref:
    response = requests.post(
        url,
        files={"audio": ref},
        data={"request": json.dumps({"input": "Hello there."})},
    )
with open("out.mp3", "wb") as f:
    f.write(response.content)

# Download the tuned voice pack
with open("ref.wav", "rb") as ref:
    response = requests.post(url, files={"audio": ref}, data={"return_voice_pack": "true"})
with open("ref.pt", "wb") as f:
    f.write(response.content)

# Save it on the server as am_ref_tuned
with open("ref.wav", "rb") as ref:
    response = requests.post(url, files={"audio": ref}, data={"save_voice": "am_ref"})
print(response.json())  # {"voice": "am_ref_tuned"}

• request is the /v1/audio/speech body without voice
• save_voice keeps the pack in VOICES_DIR as <name>_tuned, usable like any other voice. Off unless ALLOW_LOCAL_VOICE_SAVING=true. Without it, the pack is deleted after the response
• Saved names use the usual prefix (am_, bf_, ax_, etc.). Bundled tuned voices end in _inno
• Optional: prosody_head (default on), fmax pitch ceiling in Hz (default auto)
• Full reference in docs/inno-tune.md

• [voice:...] tags switch speakers inline, anywhere input is accepted
• /v1/audio/speech needs allow_voice_tags: true per request; /dev/dialogue allows them by default
• ENABLE_VOICE_TAGS=false opts out server-wide: the parameter is refused and /dev/dialogue 403s

curl -X POST http://localhost:8880/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{
    "model": "kokoro",
    "voice": "af_heart",
    "input": "The narrator opens. [voice:af_bella] Did it land? [pause:0.3s] [voice:am_michael] It did.",
    "allow_voice_tags": true,
    "response_format": "mp3"
  }' --output dialogue.mp3

With the official OpenAI client, pass the param in extra_body:

client.audio.speech.create(
    model="kokoro",
    voice="af_jadzia",
    input="The narrator opens. [voice:af_bella] Did it land?",
    extra_body={"allow_voice_tags": True},
)

POST /dev/dialogue takes structured turns instead of tags, with an optional pause_between_turns in seconds.

import requests

response = requests.post(
    "http://localhost:8880/dev/dialogue",
    json={
        "turns": [
            {"voice": "af_bella", "text": "Did the multi speaker support land?"},
            {"voice": "am_michael", "text": "It did. Turns switch voices inline."},
        ],
        "pause_between_turns": 0.4,
        "response_format": "mp3",
    },
)
with open("dialogue.mp3", "wb") as f:
    f.write(response.content)

Notes:

• Any text before the first tag uses the request's voice/default.
• Each speaker keeps its own language pipeline based on voice prefix. An explicit lang_code will override every speaker.
• Consecutive turns sharing a voice are merged automatically.
• Tags accept short names from Voice Aliases above, instead of full weighted mixes.

The number of voices has little impact on generation speed. Turns shorter than about 2 sentences do slow it down, from extra chunking. The cost is flat and does not grow with text length. Regenerate with examples/assorted_checks/test_dialogue/.

Text Control

Four tokens can be embedded in the input text and are parsed server-side (API, WebUI, or any client):

• Pause: [pause:1.5s] inserts that much silence. Must be exactly this form (colon, trailing s, case-insensitive). [pause=1.5] and [PAUSE 1.0] are not recognized and get read aloud.
• Pronunciation: [Worcester](https://github.com/remsky/Kokoro-FastAPI/blob/HEAD/wˈʊstər/) speaks the IPA between the slashes instead of the word. English only; use /dev/phonemize to find the IPA.
• Voice: [voice:am_michael] switches speaker for everything that follows.
  • Requires allow_voice_tags: true per request, and ENABLE_VOICE_TAGS server-side (on by default). Otherwise the tag is spoken as written.
  • Accepts the same combine syntax as the voice parameter ([voice:af_bella(2)+af_sky]).
  • Short names/aliases can be defined in voice_aliases.
  • Unknown values return a 400.
• Rate: [rate:1.5] scales the speaking voice's pace until the next rate tag or voice change; [rate:1.0] reverts. Applies on top of the request speed, clamped to 0.25-4.0. Same gating as voice tags.
  • A voice alias can carry a natural pace: {"grandpa": {"voice": "am_michael", "rate": 0.8}} applies that rate whenever the alias speaks, as the voice parameter or in tags. Useful for voices that read fast or slow, and for named presets over one voice (narrator_fast, narrator_slow).
  • Rate belongs to the voice speaking it. A [rate:] tag scales the speaker's own pace instead of replacing it, so an alias set to 0.8 stays slower under [rate:1.1]. Each [voice:...] tag resets to the new voice's own rate. Use speed to set the pace for the whole request.

The city of [Worcester](https://github.com/remsky/Kokoro-FastAPI/blob/HEAD/wˈʊstər/) is easy. [pause:1s] See?

Send ssml: true with allow_voice_tags: true on /v1/audio/speech or /dev/captioned_speech to translate and speak in one call. Both flags are needed, since the translation emits [voice:] and [rate:] spans that would otherwise be read aloud; ssml without them is a 400.

{
  "model": "kokoro",
  "input": "<speak>Hi<break time=\"750ms\"/>there</speak>",
  "voice": "af_bella",
  "allow_voice_tags": true,
  "ssml": true
}

POST /dev/ssml does the translation on its own when you want the tokens back as text rather than audio, or want to inspect them before synthesis. Send text, plus voice if your speech request uses one, then pass the result back with allow_voice_tags: true. Without a voice, <voice>/<prosody> are stripped and their content kept.

• <break time="750ms"/> becomes [pause:0.75s]. strength= instead of time= gives none/x-weak 0s, weak 0.25s, medium 0.5s, strong 1s, x-strong 1.5s
• <voice name="am_michael"> becomes [voice:am_michael], reverts at the closing tag
• <prosody rate="slow"> becomes [rate:0.75], and takes 80% or 1.2 too. Scales the speaking voice's pace on top of the request speed, clamped 0.25-4.0, reverts. pitch/volume ignored
• <phoneme alphabet="ipa" ph="wˈʊstər">Worcester</phoneme> becomes [Worcester](https://github.com/remsky/Kokoro-FastAPI/blob/HEAD/wˈʊstər/), IPA and English only
• <sub alias="World Wide Web">WWW</sub> speaks the alias
• <desc> is dropped with its text, an audio description is not speech
• <emphasis>, <say-as>, <p>, <s>, <lang>, etc: markup dropped, text spoken
• Malformed SSML is a 400, non-SSML passes through unchanged
• DTDs are refused and nesting past SSML_MAX_DEPTH (10) is a 400; no dialect uses a DTD, real documents nest 2-5
• Prefixed names (google:style, mstts:express-as, amazon:effect) need their xmlns: on <speak>, vendor docs often omit it
• GET /dev/ssml serves these tables as data, read off the translator itself

import requests

ssml = '<speak>The city of <phoneme alphabet="ipa" ph="wˈʊstər">Worcester</phoneme> is easy.<break time="1s"/>See?</speak>'
response = requests.post("http://localhost:8880/dev/ssml", json={"text": ssml})
print(response.json())  # {"text": "The city of [Worcester](https://github.com/remsky/Kokoro-FastAPI/blob/HEAD/wˈʊstər/) is easy. [pause:1.0s] See?"}

• Automatically splits and stitches at sentence boundaries
• Reduces artifacts, and allows long-form output from a base model configured for roughly 30s at a time

The model takes up to 510 phonemized tokens per chunk, but running it that long tends to produce 'rushed' speech and other artifacts. The server adds its own chunking layer on top, sized by TARGET_MIN_TOKENS, TARGET_MAX_TOKENS, and ABSOLUTE_MAX_TOKENS (175, 250, 450 by default, set via environment variables).

Convert text to phonemes and/or generate audio directly from phonemes:

import requests

def get_phonemes(text: str, language: str = "a"):
    """Get phonemes and tokens for input text"""
    response = requests.post(
        "http://localhost:8880/dev/phonemize",
        json={"text": text, "language": language}  # "a" for American English
    )
    response.raise_for_status()
    result = response.json()
    return result["phonemes"], result["tokens"]

def generate_audio_from_phonemes(phonemes: str, voice: str = "af_bella"):
    """Generate audio from phonemes"""
    response = requests.post(
        "http://localhost:8880/dev/generate_from_phonemes",
        json={"phonemes": phonemes, "voice": voice},
        headers={"Accept": "audio/wav"}
    )
    if response.status_code != 200:
        print(f"Error: {response.text}")
        return None
    return response.content

# Example usage
text = "Hello world!"
try:
    # Convert text to phonemes
    phonemes, tokens = get_phonemes(text)
    print(f"Phonemes: {phonemes}")  # e.g. ðɪs ɪz ˈoʊnli ɐ tˈɛst
    print(f"Tokens: {tokens}")      # Token IDs including start/end tokens

    # Generate and save audio
    if audio_bytes := generate_audio_from_phonemes(phonemes):
        with open("speech.wav", "wb") as f:
            f.write(audio_bytes)
        print(f"Generated {len(audio_bytes)} bytes of audio")
except Exception as e:
    print(f"Error: {e}")

See examples/phoneme_examples/generate_phonemes.py for a sample script.

Captions

Generate audio with word-level timestamps without streaming:

import requests
import base64
import json

response = requests.post(
    "http://localhost:8880/dev/captioned_speech",
    json={
        "model": "kokoro",
        "input": "Hello world!",
        "voice": "af_bella",
        "speed": 1.0,
        "response_format": "mp3",
        "stream": False,
    },
    stream=False
)

with open("output.mp3","wb") as f:

    audio_json=json.loads(response.content)
    chunk_audio=base64.b64decode(audio_json["audio"].encode("utf-8"))
    f.write(chunk_audio)
    print(audio_json["timestamps"])

Generate audio with word-level timestamps with streaming:

import requests
import base64
import json

response = requests.post(
    "http://localhost:8880/dev/captioned_speech",
    json={
        "model": "kokoro",
        "input": "Hello world!",
        "voice": "af_bella",
        "speed": 1.0,
        "response_format": "mp3",
        "stream": True,
    },
    stream=True
)

f=open("output.mp3","wb")
for chunk in response.iter_lines(decode_unicode=True):
    if chunk:
        chunk_json=json.loads(chunk)
        chunk_audio=base64.b64decode(chunk_json["audio"].encode("utf-8"))
        f.write(chunk_audio)
        print(chunk_json["timestamps"])

With "allow_voice_tags": true, each timestamp also carries the voice that spoke the word, so multi-speaker captions can be labelled without re-deriving the split client side. Without it the field is absent.

With stream, return_download_link, and return_timing set, the response carries an X-Timing-Path header pointing at a JSON sidecar of per-chunk timings. The audio body is unchanged and nothing extra is computed; this is what drives the web UI's read along.

response = requests.post(
    "http://localhost:8880/v1/audio/speech",
    json={
        "input": "Hello world! [pause:1s] Again.",
        "voice": "af_bella",
        "stream": True,
        "return_download_link": True,
        "return_timing": True,
    },
    stream=True,
)
audio = b"".join(response.iter_content(1024))

timings = requests.get(f"http://localhost:8880/v1{response.headers['x-timing-path']}").json()
# {"chunks": [{"text": "Hello world!", "start": 0.0, "end": 0.64}, ...]}

• Chunk-level (a sentence group, roughly 10-20s of audio), not word-level. For word-level use /dev/captioned_speech.
• The header arrives up front, but the file is written when generation finishes. Fetch it after the stream ends; earlier is a 404.
• start/end are seconds in the final audio, so pauses and speed are already accounted for. [pause:Ns] gaps appear as {"text": ""} entries.
• text is normalized (numbers etc expanded), so align by words rather than exact match.
• With allow_voice_tags, each spoken chunk also carries its voice, same as captioned timestamps. Absent otherwise.
• The sidecar sits next to the download file and shares its temp lifetime.

Performance & Operations

Throughput

Generation through the local API, text lengths up to feature-length books (~1.5 hours output), measuring processing time and realtime factor. Run on:

• Windows 11 Home w/ WSL2
• NVIDIA 4060Ti 16gb GPU @ CUDA 12.1
• 11th Gen i7-11700 @ 2.5GHz
• 64gb RAM
• WAV native output
• H.G. Wells - The Time Machine (full text)

Key Performance Metrics:

• Realtime Speed: Ranges between 35x-100x (generation time to output audio length)
• Average Processing Rate: 137.67 tokens/second (cl100k_base)

Model Unload / VRAM Reclaim

POST /dev/unload frees the model from VRAM and reloads lazily on the next request. The memory freed grows with load (the activation pool, not just weights), but levels off because chunks cap at 450 tokens. The long-form workload is ~30 paragraphs. Same setup as above.

Workload	Loaded	Floor	Reclaimed	Reload
Short (6s audio)	3.11 GB	2.37 GB	758 MiB	+4.9s
Long-form (7.5m)	3.98 GB	2.37 GB	1,656 MiB	+5.1s

Floor is host + CUDA context. Reproduce with uv run --extra benchmarks assorted_checks/benchmarks/benchmark_model_unload.py from examples/.

To automatically unload the model after an idle timeout, set MODEL_AUTO_UNLOAD_TIMEOUT_SECONDS to a positive number of seconds. The default 0 disables auto-unload.

Transcription roundtrip (WER/CER)

End-to-end roundtrip: synthesize with Kokoro, transcribe the result back with faster-whisper, compare to the source text. Scripts and data live under examples/assorted_checks/test_transcription/.

Long-form English (full book, A Journey to the Centre of the Earth, Project Gutenberg, voice af_heart, base.en Whisper on CUDA float16, baseline captured on cu126 GPU build):

Run	Input chars	Audio length	Synth speedup	Transcribe speedup	WER
Short (~ch.7)	64,996	66m 06s	36.4x rt	62.4x rt	0.047
Full book	502,766	507m 52s	45.7x rt	65.1x rt	0.033

See examples/assorted_checks/test_transcription/BASELINE.md for the full regression bands.

Per-language check (single-sentence per voice, multilingual Whisper small. WER for Latin scripts, CER for ja/zh/hi):

Language	Voice	Metric	Score
English	af_heart	WER	0.000
English (UK)	bf_emma	WER	0.111
Spanish	ef_dora	WER	0.000
French	ff_siwis	WER	0.000
Italian	if_sara	WER	0.000
Portuguese	pf_dora	WER	0.000
Hindi	hf_alpha	CER	0.059
Japanese	jf_alpha	CER	0.000
Chinese	zf_xiaobei	CER	0.143

Caveat: these are single short sentences, not a comprehensive per-language quality benchmark. They confirm each voice produces transcribable audio in its target language; deeper quality evaluation per language is open work.

To reproduce, see examples/assorted_checks/test_transcription/README.md.

Every setting is an environment variable, or a line in a .env file at the project root. Full reference in the configuration guide.

Off by default. Set API_KEYS to a JSON list and API routes answer 401 Unauthorized without one of the keys.

docker run --env 'API_KEYS=["key-one","key-two"]' -p 8880:8880 ghcr.io/remsky/kokoro-fastapi-cpu:latest

Clients send it as their usual api_key, in the header they already use (Authorization: Bearer for OpenAI, xi-api-key for ElevenLabs, X-API-Key for Cartesia):

client = OpenAI(base_url="http://localhost:8880/v1", api_key="key-one")

/health, /v1/test and /docs stay open. /web does too for now, but the player won't load voices or generate audio.

This is a basic shared-key check, not a full auth layer (no rate limiting, scoped keys etc). Keys are sent in plain text over HTTP, so if it's reachable over a network, put it behind a reverse proxy with HTTPS.

System state and resource usage, for debugging exhaustion or performance issues. The /debug/* routes expose host and process internals, so they are off by default; set ENABLE_DEBUG_ENDPOINTS=true to enable.

• /debug/threads - Get thread information and stack traces
• /debug/storage - Disk usage per mounted partition
• /debug/system - Get system information (CPU, memory, GPU)
• /dev/model - Get model load state and auto-unload timing. Off by default; set ALLOW_DEV_UNLOAD=true to enable
• POST /dev/unload - Release model from VRAM; reloads lazily on next request. Off by default; set ALLOW_DEV_UNLOAD=true to enable. Returns 409 while speech is generating
• POST /dev/reload - Load the model into VRAM. Off by default; set ALLOW_DEV_UNLOAD=true to enable. Returns 409 while speech is generating

Stability: the /v1/* OpenAI-compatible routes are the stable API. /dev/* and /debug/* are operational helpers, and may change or move behind flags between minor releases.

Global API loguru logging level can be set using the API_LOG_LEVEL environment variable. Defaults to DEBUG. Per run method in the configuration guide.

Known Issues & Troubleshooting

The API normalizes input text, which can incorrectly remove or change some phrases. Disable it with "normalization_options":{"normalize": false} in the request json, the full field list is in Text normalization:

import requests

response = requests.post(
    "http://localhost:8880/v1/audio/speech",
    json={
        "input": "Hello world!",
        "voice": "af_heart",
        "response_format": "pcm",
        "normalization_options":
        {
            "normalize": False
        }
    },
    stream=True
)

for chunk in response.iter_content(chunk_size=1024):
    if chunk:
        # Process streaming chunks
        pass

See docs/troubleshooting.md#linux-gpu-permissions for container group, host group, and device permission options.

See docs/troubleshooting.md#amd-gpu-rocm for HSA overrides, MIOpen warmup, hipBLAS fallback, and native Linux host requirement.

WAV responses use a streaming-sentinel (0xFFFFFFFF) for the size fields in the header. Most readers handle this fine: soundfile, pydub/ffmpeg, browsers, OS players. Python's stdlib wave does not, and reports a bogus duration. For exact length use soundfile.info(path).duration or ffprobe.

Project

Branching Strategy:

• release branch: Contains the latest stable build, recommended for production use. Docker images tagged with specific versions are built from this branch.
• master branch: Active development. Experimental features, ongoing changes, and fixes not yet released. Use it for the newest code, expect less stability. No images are published from here; :latest is built from release like every other tag.

Note: This is a development focused project at its core.

If something breaks, roll back to an earlier release tag, or build from source and submit a PR.

Free and open source is a community effort, and there's only really so many hours in a day. If you'd like to support the work, feel free to open a PR, buy me a coffee, or report any bugs/features/etc you find during use.

Working on the code, or pointing an AI agent at it? AGENTS.md covers the repo layout, commands, and conventions.

This API uses the Kokoro-82M model from HuggingFace.

Visit the model page for more details about training, architecture, and capabilities. I have no affiliation with any of their work, and produced this wrapper for ease of use and personal projects.

• The Kokoro model weights are licensed under Apache 2.0 (see model page)
• The FastAPI wrapper code in this repository is licensed under Apache 2.0 to match
• The inference code adapted from StyleTTS2 is MIT licensed

The full Apache 2.0 license text can be found at: https://www.apache.org/licenses/LICENSE-2.0

Contributors

Made with contrib.rocks.