TTS model benchmarks¶

Date: 2026-07-31. Hardware: NVIDIA GeForce RTX 4090, PyTorch 2.8.0, CUDA 12.8.

Five of 34 registered TTS providers have real-checkpoint, end-to-end results with at least 10 seconds of audio. The other rows are explicit about what was and was not run.

Read the labels¶

real-checkpoint-end-to-end: published weights produced a long waveform.
tiny-graph: a small synthetic graph ran; this is not a checkpoint benchmark.
static-plan: lazy construction and optimization planning only.
blocked, not-run, and unsupported never carry invented performance.
rejected means a measured candidate failed the quality policy.

Every measured profile used seed 1234, a fresh process and compiler cache, CUDA-synchronized timing, warm-ups, repeated runs, and a 10-second minimum. Exact waveform equality was required for numerically invariant changes. Non-exact regional kernels also needed unchanged duration/transcript metrics and high waveform SNR. One fixture is not a general listening-quality claim.

Results to use¶

VITS: use eager FP32 plus weight-norm-cache. It was bit-exact and 1.048x faster by mean latency (1.027x by median), but used 53.92 MiB more peak allocated memory. AUTO inference now stays eager; inference compile is quality-unsafe.
F5-TTS: regional compile passed this fixture at 1.111x steady speed and 4.01% less peak allocated memory. Cold inference rose from 6.33 to 130.16 seconds. It needs 206 additional requests (207 total) to break even. Use it only for a long-lived, fixed workload and recheck quality on your data.
NeuTTS: keep eager. SDPA was already native; regional compile was 1.342x faster but changed duration and doubled CER, so it is rejected.
Supertonic: keep eager. Compile was bit-exact but only 0.589x as fast; reduced step counts raised WER.
Vui: keep eager. Both compile policies changed sequence or transcript quality and now fail closed for inference.

Real-checkpoint baselines¶

Model	Audio	Mean latency	RTF	Peak allocated	Warm-up / runs
Vui	21.827 s	8.523 s	0.39049	1,897.76 MiB	2 / 5
F5-TTS	14.207 s	5.836 s	0.41080	1,600.56 MiB	1 / 3
NeuTTS	12.440 s	20.270 s	1.62941	3,487.07 MiB	1 / 3
Supertonic	28.369 s	0.478 s	0.01685	433.73 MiB	3 / 10
VITS	27.312 s	0.064 s	0.00235	487.73 MiB	5 / 15

Optimization candidates¶

Peak allocated change uses − for less memory and + for more. “Paired” means the candidate used the separately recorded baseline from the same run.

Model / profile	Mean	Speed	Peak allocated change	Quality check	Decision
Vui / compile dynamic	5.266 s	1.619x	+7.12%	WER 0.516 → 2.359; length changed	`rejected`
Vui / compile specialized	4.316 s	1.942x	−0.22%	WER 0.516 → 0.578; transcript changed	`rejected`
F5 / BF16	2.042 s	2.859x	−49.66%	WER 0.485 → 1.455	`rejected`
F5 / SDPA	5.916 s	0.986x	0.00%	bit-exact	`no-benefit`
F5 / Triton	5.814 s	1.004x	0.00%	same WER; 54.07 dB SNR	`no-benefit`
F5 / SDPA + Triton	5.860 s	0.996x	0.00%	same WER; 54.07 dB SNR	`no-benefit`
F5 / NFE 16	3.088 s	1.890x	−1.36%	WER 0.485 → 0.606	`rejected`
F5 / DBCache 0.05	5.997 s	0.973x	+4.01%	exact; 0% cache hits	`no-benefit`
F5 / DBCache 0.12	2.904 s	2.010x	+4.01%	same WER; changed waveform lacks paired SNR	`rejected`
F5 / outer compile	—	—	—	terminated after more than 376 s	`blocked`
F5 / regional compile (paired)	5.406 s	1.111x	−4.01%	same WER; 51.31 dB SNR	`accepted`
NeuTTS / SDPA	20.486 s	0.989x	0.00%	bit-exact	`no-benefit`
NeuTTS / regional compile (paired)	15.225 s	1.342x	+0.001%	duration −0.46 s; CER doubled	`rejected`
Supertonic / compile	0.811 s	0.589x	−0.19%	bit-exact	`no-benefit`
Supertonic / 3 steps	0.357 s	1.339x	+0.0002%	WER 0.031 → 0.063	`rejected`
Supertonic / 2 steps	0.297 s	1.612x	+0.0002%	WER 0.031 → 0.266	`rejected`
VITS / weight-norm cache (paired)	0.068 s	1.048x	+11.06%	bit-exact; replicated	`accepted`
VITS / Triton	0.064 s	0.999x	0.00%	not bit-exact	`rejected`
VITS / compile	0.070 s	0.914x	+32.45%	not bit-exact	`rejected`
VITS / Triton + compile	0.058 s	1.099x	+32.45%	not bit-exact	`rejected`
VITS / FP16 cache	0.055 s	1.166x	−38.45%	duration and waveform changed	`rejected`
VITS / BF16 cache	0.044 s	1.464x	−38.45%	substantial waveform change	`rejected`

Full model evidence¶

All 34 rows passed the offline static-plan audit. “Evidence” below is the highest tier reached, not a claim that every lower-level test was end to end. Full checkpoint revisions and evidence paths are in the JSON artifact.

Model type	Evidence	Real checkpoint	What was actually established
`orpheustts`	`tiny-graph`	`blocked`	Tiny round trip; gated weights and voice fixture unavailable
`dia`	`tiny-graph`	`not-run`	Tiny local generation/training/export
`vui`	`real-checkpoint-end-to-end`	measured	21.827-second eager and compile runs
`chatterbox`	`tiny-graph`	`not-run`	Tiny T3 inference and component training
`kokoro`	`tiny-graph`	`not-run`	Tiny acoustic/frontend graph
`echo`	`static-plan`	`blocked`	No immutable default-checkpoint revision recorded
`conversationtts`	`tiny-graph`	`not-run`	Tiny cached decoder and training graph
`llasa`	`tiny-graph`	`not-run`	Tiny codec/generation/training graph
`cosyvoice`	`tiny-graph`	`blocked`	Missing speaker-embedding and prompt-token fixture
`f5tts`	`real-checkpoint-end-to-end`	measured	14.207-second baseline and nine candidates
`gptsovits`	`tiny-graph`	`not-run`	Prepared-input native graph
`melotts`	`tiny-graph`	`not-run`	Tiny inference/export/training graph
`openvoice`	`tiny-graph`	`blocked`	Missing matched base and target-speaker audio
`outetts`	`tiny-graph`	`not-run`	Tiny language-model objective/export
`parlertts`	`tiny-graph`	`not-run`	Tiny acoustic-token and DAC graph
`styletts2`	`tiny-graph`	`blocked`	No default checkpoint; reviewed artifact and phonemes required
`mosstts`	`tiny-graph`	`not-run`	Tiny codec and buffered generation
`qwen3tts`	`tiny-graph`	`not-run`	Tiny talker/codec/cache/LoRA graph
`irodoritts`	`tiny-graph`	`not-run`	Tiny RF-DiT/duration graph
`zonos`	`tiny-graph`	`not-run`	Tiny sampling/codec/training graph
`zonos2`	`tiny-graph`	`not-run`	Tiny dense/MoE sampling graph
`voxcpm`	`tiny-graph`	`not-run`	Tiny generation/SFT/LoRA graph
`omnivoice`	`tiny-graph`	`not-run`	Tiny token generation and codec graph
`higgstts`	`tiny-graph`	`not-run`	Tiny constrained generation and codec graph
`xtts`	`tiny-graph`	`not-run`	Tiny token generation/conditioning graph
`vibevoice`	`tiny-graph`	`unsupported`	Low-level stages only; high-level waveform parity fails closed
`fishtts`	`tiny-graph`	`not-run`	Tiny semantic/cache and codec graph
`csm`	`tiny-graph`	`blocked`	Tiny graph; gated checkpoint access unavailable
`neutts`	`real-checkpoint-end-to-end`	measured	12.440-second official Emily fixture
`supertonic`	`real-checkpoint-end-to-end`	measured	28.369-second baseline and three candidates
`inflecttts`	`tiny-graph`	`not-run`	Tiny VITS/frontend graph
`bark`	`tiny-graph`	`not-run`	Tiny end-to-end generation and codec
`speecht5`	`tiny-graph`	`not-run`	Tiny acoustic/vocoder generation
`vits`	`real-checkpoint-end-to-end`	measured	27.312-second baseline and six candidates

Reproduce a measured pair¶

python scripts/benchmark_tts_inference.py \
  --model-type vits \
  --model facebook/mms-tts-eng \
  --profiles baseline,weight-norm-cache \
  --config-kwargs '{"revision":"c71de0fe7204c83f1c10820a7d696d0b450048ba"}' \
  --seed 1234 \
  --minimum-audio-seconds 10 \
  --output result.json

Keep the eager baseline, pin every checkpoint, and compare one change at a time. The machine-readable ledger is benchmarks/tts_optimization_rtx4090_2026-07-31.json.