XTTS¶

Usage¶

python -m pip install voicehub

Install VoiceHub and the provider extra shown above.
Choose a checkpoint that matches this integration.
Provide an authorized reference.wav and an exact reference transcript when the example requests them.
Generate audio and inspect the returned sample rate and metadata.

from pathlib import Path

REFERENCE_AUDIO = Path("reference.wav")
REFERENCE_TEXT = "This transcript must exactly match the authorized reference audio."


from voicehub import AutoModelForTextToSpeech, TTSGenerationConfig

model = AutoModelForTextToSpeech.from_pretrained(
    'coqui/XTTS-v2',
    model_type='xtts',
    device="cuda",
    lazy_load=True,
)
generation_kwargs = {
    "speaker_audio_path": str(REFERENCE_AUDIO),
    "language": "en",
}
output = model.generate(
    "VoiceHub keeps model integrations consistent and easy to extend.",
    generation_config=TTSGenerationConfig(
        seed=42,
        output_file=Path("output.wav"),
    ),
    **generation_kwargs,
)
print(output.file_path, output.sample_rate)

Use only authorized recordings for reference voice, transcription, detection, or evaluation. The example selects a concrete device; verify checkpoint-specific hardware needs and pin an immutable revision before production use.

Overview¶

XTTS uses the canonical model type xtts and is a VoiceHub text to speech integration. This page is generated from the model registry and its executable data and training contracts, so the documented support stays aligned with code. Open the xtts Colab notebook.

Property	Value
Task	Text to speech
Architecture	`xtts2`
Runtime	`VoiceHub-native`
Languages	17 enumerated languages
Capabilities	`text-to-speech`, `voice-cloning`, `multilingual`, `fine-tuning`, `safetensors`, `voicehub-native`, `native-runtime`, `preencoded-code-fine-tuning`, `gpt-fine-tuning`, `restricted-pickle-conversion`
Reusable components	—
Normalized output	`TTSOutput`

Language support¶

17 documented languages

en, es, fr, de, it, pt, pl, tr, ru, nl, cs, ar, zh-CN, hu, ko, ja, hi

Configuration¶

Load the registered configuration without constructing the model. The canonical key remains serializable even though the page uses a presentation label.

from voicehub import AutoConfig

config = AutoConfig.for_model('xtts')
print(config.model_type)

Property	Value
Canonical model type	`xtts`
Configuration class	`XTTSConfig`
Architecture class	`XTTSForTextToSpeech`

Processing¶

AutoProcessor resolves the processor declared by the registered model. Creating the processor does not allocate model weights.

from voicehub import AutoProcessor

processor = AutoProcessor.from_pretrained(
    'coqui/XTTS-v2',
    model_type='xtts',
)
print(type(processor).__name__)

Processor behavior remains model-owned when text normalization, audio loading, feature extraction, or reference speech requires provider-specific semantics.

Inference¶

The Usage example returns TTSOutput through AutoModelForTextToSpeech. Inputs are validated against the task and data contracts below before model-specific execution.

Input and output contract¶

Property	Value
Readiness	`preprocessed`
Data architecture	`hybrid`
Sample rate	22,050 Hz
Contract getter	`get_tts_dataset_spec('xtts')`

Variant	Required fields	One of	Boundary	Other rules
`native-gpt-tokens`	`text_inputs`, `text_lengths`, `audio_codes`, `wav_lengths`	cond_mels / cond_latents	Prepared	—
`native-gpt-waveform`	`text_inputs`, `text_lengths`	wav / audio_values; cond_mels / cond_latents	Prepared	at most one: wav / audio_values; forbidden: audio_codes

Multi-component language-model, diffusion, acoustic, or GAN data. Follow the shared data workflow for manifest loading, audio validation, leakage-safe splits, and model-owned preprocessing.

Training and optimization¶

All public optimizations enter this model through the shared BaseSpeechModel lifecycle. Use available_optimization_passes() to discover the public pass registry, then apply, inspect, serialize, or restore a plan through the common model API. Application remains fail-closed when the active runtime or hardware cannot satisfy a pass.

Training contract¶

Property	Value
Support	`preprocessed`
Family	`composite`
Recipe	`single-phase`
Default phase	`language_model`
Training checkpoint	`coqui/XTTS-v2`
Native training graph	`yes`

Phase	Kind	Components	Required inputs	Loss keys
`language_model`	objective	`model.gpt`	`text_inputs`, `text_lengths`, `audio_codes`, `wav_lengths`	`loss`, `loss_text_ce`, `loss_mel_ce`

Prepare the exact tensors listed in the data contract before this step. Call model.validate_training_support() before constructing a trainer. Follow the shared training workflow for a one-step smoke test, validation, checkpoint resume, optimization, and portable export.

Checkpoints, provenance, license, and limitations¶

Property	Value
Default checkpoint	`coqui/XTTS-v2`
Checkpoint status	Registry default; pin an immutable revision for production and reproducible evidence
Optional dependency extra	Core package
Hardware and runtime	Usage selects `cuda`; verify checkpoint-specific requirements
Real-checkpoint evidence	Release evidence; a registry default alone is not execution evidence
Implementation	`voicehub.models.xtts.modeling_xtts.XTTSForTextToSpeech`
Configuration	`voicehub.models.xtts.configuration_xtts.XTTSConfig`
Source provenance	`voicehub/models/xtts/source/SOURCE.json`
License	CPML

XTTS checkpoint terms are separate from the MPL-2.0 runtime source. Commercial use: review required.

The default checkpoint identifies the expected family, not every compatible variant. Confirm the selected checkpoint's revision, access terms, provenance, and license before downloading or redistributing it.

Limitations¶

No integration-specific checkpoint limitation is registered. Verify the selected checkpoint revision and its documented runtime requirements.
The Usage example selects cuda; validate memory, precision, and optional dependency requirements on the target system.
Public optimizations fail closed when the runtime or hardware cannot satisfy their validation contract; an unavailable pass is not reported as applied.
Contract tests do not substitute for released-checkpoint evidence. Consult the linked release record before treating a checkpoint path as verified.

Public API¶

The stable configuration and model facades keep source inspection local while the task auto class owns pretrained loading and normalized output behavior.

`XTTSConfig`¶

View XTTSConfig source

XTTSConfig(**config_kwargs)

`XTTSForTextToSpeech`¶

View XTTSForTextToSpeech source

AutoModelForTextToSpeech.from_pretrained(
    pretrained_model_name_or_path,
    *,
    model_type='xtts',
    config=None,
    **model_kwargs,
)

The loader returns XTTSForTextToSpeech through the shared task-specific factory.

from voicehub import get_model_spec

spec = get_model_spec('xtts')
print(spec.display_name, spec.task.value)

Purpose	Public object
Discover	`get_model_spec('xtts')`
Load and run	`AutoModelForTextToSpeech`
Configure	`XTTSConfig`
Process	`AutoProcessor`
Model implementation	`XTTSForTextToSpeech`
Normalized output	`TTSOutput`
Training contract	`get_training_spec('xtts')`
Optimization lifecycle	`available_optimization_passes`, `apply_optimization_plan`, `optimization_manifest`, `restore_optimization_plan`