跳转至

FunAudioLLMFun-CosyVoice3-0.5B-2512

CosyVoice

Loads the required 192-value speaker embedding from a reviewable JSON file.

Text to speechVoiceHub-nativecosyvoice-nativeParameters: 859.2MLanguages: zh, en +7Training: customLicense: Checkpoint-specific

Parameter metadata: Exact learned-parameter total for VoiceHub's audited native primary graph at the registered default selection; separately loaded auxiliary models are excluded.

Use this model Checkpoint
Resources
Paper Upstream GitHub VoiceHub source Open in Colab

Usage

Complete the VoiceHub installation once, then run this repository-authored example. Model pages intentionally contain no package-install command.

This example is maintained against VoiceHub's public API; it is not copied from an upstream demo or package README.

Model-specific path: Loads the required 192-value speaker embedding from a reviewable JSON file.

Inputs and controls: The native boundary intentionally does not run an unverified speaker encoder behind the caller's back.

from pathlib import Path
import json

from voicehub import AutoModelForTextToSpeech, TTSGenerationConfig

SPEAKER_EMBEDDING_FILE = Path("speaker_embedding.json")
SPEAKER_EMBEDDING = json.loads(SPEAKER_EMBEDDING_FILE.read_text(encoding="utf-8"))
if len(SPEAKER_EMBEDDING) != 192:
    raise ValueError("CosyVoice expects exactly 192 speaker-embedding values.")

model = AutoModelForTextToSpeech.from_pretrained(
    'FunAudioLLM/Fun-CosyVoice3-0.5B-2512',
    model_type='cosyvoice',
    device="cuda",
    lazy_load=True,
)
output = model.generate(
    'VoiceHub keeps model integrations explicit and reproducible.',
    generation_config=TTSGenerationConfig(
        seed=42,
        output_file=Path("output.wav"),
    ),
    speaker_embedding=SPEAKER_EMBEDDING,
    instruction="Speak clearly.",
    flow_steps=10,
)
print(output.file_path, output.sample_rate, output.metadata)

Use authorized recordings. Verify hardware needs and pin a revision in production.

Overview

cosyvoice is a VoiceHub text to speech integration. This page is generated from its registry contract. Open the cosyvoice Colab notebook.

Property Value
Task Text to speech
Architecture cosyvoice-native
Runtime VoiceHub-native
Languages zh, en, ja, ko, … complete audited list below
Capabilities text-to-speech, voice-cloning, multilingual, fine-tuning, flow-matching, adversarial-vocoder-training, safetensors, voicehub-native, native-runtime, precomputed-speaker-embedding, preencoded-speech-token-fine-tuning
Reusable components conformer
Normalized output TTSOutput

Language support

Supported language abbreviations

zh, en, ja, ko, de, es, fr, it, ru

The card additionally names Guangdong, Minnan, Sichuan, Dongbei, Shan3xi, Shan1xi, Shanghai, Tianjin, Shandong, Ningxia, and Gansu Chinese dialects or accents.

Paper and GitHub

Configuration

Load configuration without constructing the model:

from voicehub import AutoConfig

config = AutoConfig.for_model('cosyvoice')
print(config.model_type)
Property Value
Canonical model type cosyvoice
Configuration class CosyVoiceConfig
Architecture class CosyVoiceForTextToSpeech

Processing

Create the registered processor without allocating model weights:

from voicehub import AutoProcessor

processor = AutoProcessor.from_pretrained(
    'FunAudioLLM/Fun-CosyVoice3-0.5B-2512',
    model_type='cosyvoice',
)
print(type(processor).__name__)

Inference

The Usage example returns TTSOutput through AutoModelForTextToSpeech.

Input and output contract

Property Value
Readiness integrated-raw
Data architecture hybrid
Sample rate 24,000 Hz
Contract getter get_tts_dataset_spec('cosyvoice')
Variant Required fields One of Boundary Other rules
llm-raw-audio text speech_audio / audio / waveform / audio_path Source at most one: speech_audio / audio / waveform / audio_path; forbidden: speech_tokens; speech_audio requires one of speech_sampling_rate, sampling_rate, sample_rate; audio requires one of speech_sampling_rate, sampling_rate, sample_rate; waveform requires one of speech_sampling_rate, sampling_rate, sample_rate
llm-record text, speech_tokens — Prepared forbidden: speech_audio, audio, waveform, audio_path

Multi-component language-model, diffusion, acoustic, or GAN data. See the data workflow.

Training and optimization

Use available_optimization_passes() to discover reversible public passes. Unsupported runtime or hardware fails closed before mutation.

Training contract

Property Value
Support custom
Family composite
Recipe adversarial
Default phase llm
Training checkpoint FunAudioLLM/Fun-CosyVoice3-0.5B-2512
Native training graph yes
Phase Kind Components Required inputs Loss keys
llm objective model.llm — language_model_loss
flow objective model.flow — flow_matching_loss
hifigan_generator generator model.hift — adversarial_loss, feature_matching_loss, pitch_loss, spectral_reconstruction_loss
hifigan_discriminator discriminator model.hifigan.discriminator — discriminator_loss

This profile uses model-specific phases; inspect and honor each phase boundary. Call model.validate_training_support() first, then follow the training workflow.

Checkpoints, provenance, license, and limitations

Property Value
Default checkpoint FunAudioLLM/Fun-CosyVoice3-0.5B-2512
Hugging Face ID FunAudioLLM/Fun-CosyVoice3-0.5B-2512
Repository availability verified through the Hugging Face model API on 2026-08-11; pin a revision before production use.
Checkpoint status Registry default; pin an immutable revision for production and reproducible evidence
Optional dependency extra Core package
Hardware and runtime Usage selects cuda; verify checkpoint-specific requirements
Real-checkpoint evidence Release evidence; a registry default alone is not execution evidence
Implementation voicehub.models.cosyvoice.modeling_cosyvoice.CosyVoiceForTextToSpeech
Configuration voicehub.models.cosyvoice.configuration_cosyvoice.CosyVoiceConfig
Source provenance voicehub/models/cosyvoice/source/SOURCE.json
License Checkpoint-specific

No VoiceHub-specific license override is registered. Verify the checkpoint and upstream source terms before use.

Confirm the checkpoint revision, access terms, provenance, and license.

Limitations

  • No integration-specific checkpoint limitation is registered. Verify the selected checkpoint revision and its documented runtime requirements.
  • Validate memory, precision, and optional dependencies on the target system.
  • Public optimizations fail closed when the runtime or hardware cannot satisfy their validation contract; an unavailable pass is not reported as applied.
  • Contract tests do not replace the linked released-checkpoint evidence.

Public API

Use the stable configuration, processor, and task-model facades below.

Configuration

CosyVoiceConfig

View source

CosyVoiceConfig(**config_kwargs)

Parameters

  • **config_kwargs — Configuration fields validated by CosyVoiceConfig.

Model

CosyVoiceForTextToSpeech

View source

AutoModelForTextToSpeech.from_pretrained(
    pretrained_model_name_or_path,
    *,
    model_type='cosyvoice',
    config=None,
    **model_kwargs,
)

Parameters

  • pretrained_model_name_or_path — Hub ID or compatible local directory.
  • model_type — Canonical model type; use 'cosyvoice'.
  • config — Optional preloaded CosyVoiceConfig instance.
  • **model_kwargs — Model-specific loading arguments.
from voicehub import get_model_spec

spec = get_model_spec('cosyvoice')
print(spec.display_name, spec.task.value)
Purpose Public object
Discover get_model_spec('cosyvoice')
Load and run AutoModelForTextToSpeech
Configure CosyVoiceConfig
Process AutoProcessor
Model implementation CosyVoiceForTextToSpeech
Normalized output TTSOutput
Training contract get_training_spec('cosyvoice')
Optimization lifecycle available_optimization_passes, apply_optimization_plan, optimization_manifest, restore_optimization_plan

See all model guides, inference, and the training matrix.