Self-Hosted TTS with Kokoro ONNX: What CPU-Only Inference Actually Gets You

Why Self-Hosted TTS?

The project started from a narrow, practical need: text-to-speech for smart home automation, running on hardware that was already sitting in the rack. No GPU budget, no interest in sending every doorbell announcement and morning briefing through a third-party API, and no appetite for per-character billing on something that should just work locally.

That constraint — CPU-only, no external dependency — shaped every decision that followed. Cloud TTS APIs work, but they send every utterance to an external server. For a home automation system that runs locally, that’s a non-starter. And we didn’t have a GPU in the rack, so the choice was simple: make TTS work on existing CPU hardware or don’t bother.

The privacy argument was sufficient for smart home use. But the customisation is what made it interesting — 54 built-in voices across 8 languages, plus the ability to blend any combination into unique voices.

Kokoro ONNX: What It Is

Kokoro ONNX is a small, open-weight TTS model — notably lightweight compared to the diffusion- and transformer-heavy TTS stacks that assume a GPU is available. Exported to ONNX, it runs through ONNX Runtime’s CPU execution provider, which means no CUDA, no ROCm, no driver stack to maintain — just a .onnx file and a runtime that runs anywhere ONNX Runtime does.

The model ships in three quantisation variants:

RTF (Real-Time Factor) tells you how many seconds of CPU time produce one second of audio. An RTF of 1.4x means 1.4 seconds of CPU work per 1 second of speech. Below 1.0 is real-time. Above 1.0 is slower than real-time.

The practical decision that matters most on CPU-only hardware is precision. On GPU, FP16 gives you a proportional speedup because GPUs have native FP16 compute paths. On CPU, most consumer and server CPUs lack native FP16 compute — so the speed gain comes mainly from reduced memory bandwidth, not faster arithmetic. That’s why FP16 and FP32 end up close in RTF on CPU, while INT8 (which actually does get faster arithmetic on CPU) jumps ahead on speed despite being the smallest model.

On our 4-core Xeon, FP16 hits 1.4x — usable for smart home commands where a short delay is acceptable. The default is INT8 (88 MB, fastest load) but FP16 is the sweet spot for quality vs speed. All three models are auto-downloaded from GitHub Releases on first boot — no manual setup required.

Architecture: Two Containers, One Volume

The system runs as two Docker containers managed by Docker Compose:

# docker-compose.yml (simplified)
services:
  tts-api:
    build: .
    container_name: kokoro_tts_api
    volumes:
      - tts-models:/app/models
    ports:
      - "8111:8000"
    restart: always

  gradio-ui:
    build: .
    container_name: kokoro_gradio_ui
    ports:
      - "8112:7860"
    environment:
      - API_URL=http://tts-api:8000
    volumes:
      - tts-models:/app/models
    command: ["python", "gradio_app.py", "--server-name", "0.0.0.0"]
    depends_on:
      - tts-api
    restart: always

volumes:
  tts-models:

The tts-api container runs the FastAPI server with uvicorn (single worker). The gradio-ui container runs the Gradio web interface. Both share a tts-models volume so models are downloaded once and reused.

Models auto-download on first run rather than being baked into the image, which keeps the image lean and makes swapping model versions a config change rather than a rebuild. Total startup time: about 15 seconds.

The Dialogue Scripting System

This is the feature that turns Kokoro from “type text, get audio” into something closer to a lightweight audio production tool.

Scripts are defined as arrays of lines, where each line independently specifies a voice (or a blend), speed, delay before the line plays, and optional per-line blend weight if multiple voices are mixed. That’s enough to script a back-and-forth exchange between two distinct voices, insert pauses for pacing, and vary delivery speed per line — without touching code.

{
  "script": [
    {
      "text": "Good morning! Here's your daily briefing.",
      "voice": "af_sky",
      "speed": 1.0,
      "delay": 0.0
    },
    {
      "text": "The temperature outside is 22 degrees.",
      "voice": "am_eric",
      "speed": 0.9,
      "delay": 0.5
    },
    {
      "text": "And now for the news headlines.",
      "blend_components": [
        {"voice": "am_adam", "weight": 0.7},
        {"voice": "af_nova", "weight": 0.3}
      ],
      "blend_method": "slerp"
    }
  ]
}

Each line in the script goes through four stages:

1. Parse — extract text, voice/blend, speed, delay
2. Phonemize — convert text to phonemes using phonemizer-fork
3. Voice resolve — apply single voice or blend voices (additive or SLERP)
4. Infer — run Kokoro ONNX to generate audio samples

The output is concatenated into a single WAV file with the specified delays between lines. For smart home use, this means you can script multi-character conversations — a narrator, a weather reporter, a news anchor — all in one request. A multi-step announcement (“garage door open… did you mean to leave that?”) can sound like two distinct system voices rather than one flat TTS drone reading a paragraph.

The streaming endpoint (/synthesize-stream) splits text on sentence boundaries and uses an asyncio.Queue producer-consumer pattern to stream audio chunks as they’re generated, reducing perceived latency for longer scripts.

Voice Blending: Additive + SLERP

Each of the 54 voices is a 256-dimensional embedding vector. Voice blending operates in this space to create new, unique voices.

result = normalize(weight_a * voice_a + weight_b * voice_b)

# Iterative pairwise blending for 3+ voices
t = weight_b / (weight_a + weight_b)
result = slerp(voice_a, voice_b, t)

After blending, the system finds the closest existing voice name by Euclidean distance — useful for debugging or when you want to know what the blend “sounds like” relative to known voices.

The random voice endpoints (/random-speaker and /random-custom-voice) use Dirichlet-distributed weights for random blending — great for exploring the voice space.

Gradio UI: Five Tabs

The Gradio interface provides visual access to every feature:

The benchmark tab is particularly useful — it runs all three model variants and recommends the best one based on your hardware. On our Xeon, it recommended FP16 as “best balanced” and INT8 as “smallest footprint”.

Benchmark Results: Honest Numbers

Test hardware: Intel Xeon E5620 @ 2.40GHz, 4 cores / 8 threads, 16GB RAM. No GPU. This is a decade-plus-old server CPU, not a modern chip — worth keeping in mind when reading these numbers, since RTF is going to look meaningfully better on anything newer.

That’s the headline number, and it’s not a flattering one — but it’s also the honest one. Nobody self-hosting TTS on an E5620 should expect real-time conversational latency. What this hardware is good for is pre-generated or asynchronous announcements: garage door status, morning briefings queued a few seconds ahead, notification audio rendered on trigger rather than on demand mid-conversation.

The surprise: INT8 is actually the slowest on CPU. Quantisation reduces model size but increases inference time because the CPU has to dequantise during computation. FP16 is the clear winner for speed — 1.4x RTF means a 5-second utterance takes about 7 seconds of CPU time.

For reference, cloud TTS APIs typically hit 0.3–0.5x RTF — but they run on dedicated GPU clusters. We’re running on a decade-old Xeon from the rack. Different league, different constraints.

What’s Missing: Honest Limitations

Being direct about the gaps, since a smart-home proof of concept and a production API are different things:

None of these are bugs. They’re the result of scoping the project honestly for what it needed to do — run TTS for one household’s smart home triggers — rather than over-building for a use case that didn’t exist yet.

What’s Next

Two concrete next steps:

• Request queue — even a simple in-process queue would remove the worst failure mode (two requests degrading each other) without needing to touch the auth or multi-tenancy question at all.
• MCP integration — trigger.json and _signal.json already exist as a lightweight hook point. Wiring these into an MCP server would let agent workflows trigger speech synthesis directly, rather than needing a bespoke integration per automation.

Neither of those requires new hardware. Both are software-only next steps on the same E5620 — which is, in a way, the whole point of the project: figuring out how far CPU-only inference stretches before you actually need to reach for a GPU.

Final Thoughts

Kokoro TTS Service started as a smart home experiment: can CPU-only TTS produce usable speech? The answer is yes — with the right model (FP16), the right expectations (not real-time), and the right use case (local automation, not public API).

The dialogue scripting system turned out to be the most interesting feature. Per-line voice control, speed adjustments, delays between speakers, and voice blending — it’s more than a TTS wrapper. It’s a mini audio production tool.

The service is published as-is: no auth, no queue, no scaling. That’s intentional. It’s a starting point for anyone who wants to run TTS locally on CPU. Fork it, add a queue, add auth, make it yours.

If you’re building self-hosted AI infrastructure — TTS, LLM inference, or custom models — let’s talk. We’ve been through the CPU constraints, the ONNX optimisations, and the Docker deployment puzzles.

Related Reading

• Self-Hosted AI vs Cloud APIs — The business case for running AI on your own hardware
• Bringing GPU Support to NeuTTS-Air — Another TTS project, this time with GPU acceleration
• VoxNemesis Supertonic — Local-first TTS browser extension with GPU awareness
• Self-Hosted CI/CD on a Home Rack — The deployment pipeline that ships this service
• Self-Hosted AI Services — Run AI models on your own infrastructure