How plachtaa/seed-vc works
Seed-VC provides zero-shot voice, singing-voice, and real-time conversion, plus V2 style conversion and custom-data fine-tuning. The main workflow takes source speech and a target voice reference through a selected model runtime and produces converted audio. The graph separates V1/V2 inference, shared audio and model components, interfaces, and training. Some UI and training details are represented from documented capabilities and the file tree; their internal wiring is not asserted where source excerpts do not show it.
Main entry points: inference.py V1 inference · vc_wrapper.py V2 conversion wrapper · seed_vc_wrapper.py V1 conversion wrapper · commons.py Shared model operations
Components 27
User interfaces
- Voice conversion UI
app_vc.py - V1/V2 demo UI
app.py - Real-time UI
real-time-gui.py
Conversion runtime
- V1 inference
inference.py - V1 conversion wrapper
seed_vc_wrapper.py - V2 inference
inference_v2.py - V2 conversion wrapper
modules/v2/vc_wrapper.py
Model components
- Audio features
modules/audio.py - Pitch extraction
modules/rmvpe.py - Speaker embedding
modules/campplus/DTDNN.py - Shared model operations
modules/commons.py - V1 diffusion model
modules/diffusion_transformer.py - Flow matching
modules/flow_matching.py - Length regulation
modules/length_regulator.py - ASTRAL content encoding
modules/astral_quantization/bsq.py - Style AR model
modules/v2/ar.py - V2 flow model
modules/v2/cfm.py - V2 DiT model
modules/v2/dit_model.py
Audio synthesis
- BigVGAN vocoder
modules/bigvgan/bigvgan.py - HiFiGAN vocoder
modules/hifigan/generator.py - Encodec codec
modules/encodec.py - Converted audio
Fine-tuning
- V1 fine-tuning
train.py - V2 fine-tuning
train_v2.py - Fine-tuning dataset
data/ft_dataset.py
Other components
- Audio user
- Reference speech
Connections 35
- Audio user → V1 inference: provides audio
README.md - Reference speech → V1 inference: provides voice reference
README.md - V1 inference → Audio features: extracts mel features
inference.py - V1 inference → Pitch extraction: extracts pitch
inference.py - V1 inference → Speaker embedding: loads speaker model
inference.py - V1 inference → BigVGAN vocoder: loads vocoder
inference.py - V1 inference → HiFiGAN vocoder: loads vocoder
inference.py - V1 inference → Converted audio: writes conversion
README.md - Audio user → V2 inference: provides audio
README.md - Reference speech → V2 inference: provides voice reference
README.md - V2 inference → V2 conversion wrapper: runs conversion
inference_v2.py - V2 conversion wrapper → ASTRAL content encoding: uses content encoder
modules/v2/vc_wrapper.py - V2 conversion wrapper → Style AR model: uses style model
modules/v2/vc_wrapper.py - V2 conversion wrapper → V2 flow model: uses flow model
modules/v2/vc_wrapper.py - V2 conversion wrapper → Converted audio: streams audio
modules/v2/vc_wrapper.py - Voice conversion UI → V1 conversion wrapper: invokes conversion
README.md - V1/V2 demo UI → V1 conversion wrapper: invokes conversion
app.py - V1/V2 demo UI → V2 conversion wrapper: invokes conversion
app.py - Real-time UI → Converted audio: streams conversion
- V1 conversion wrapper → Audio features: uses audio processing
seed_vc_wrapper.py - V1 conversion wrapper → Pitch extraction: uses pitch extractor
seed_vc_wrapper.py - V1 conversion wrapper → Speaker embedding: uses speaker model
seed_vc_wrapper.py - V1 conversion wrapper → BigVGAN vocoder: uses vocoder
seed_vc_wrapper.py - V1 inference → Shared model operations: uses model utilities
inference.py - Shared model operations → Flow matching: imports flow matching
modules/commons.py - Shared model operations → Length regulation: imports length regulation
modules/commons.py - V1 inference → V1 diffusion model: constructs model
inference.py - V1 inference → Converted audio: synthesizes speech
README.md - V2 conversion wrapper → BigVGAN vocoder: uses vocoder
- V1 fine-tuning → Fine-tuning dataset: loads training data
- V2 fine-tuning → Fine-tuning dataset: loads training data
- Audio user → V1 fine-tuning: fine-tunes model
README.md - Audio user → V2 fine-tuning: fine-tunes model
README.md - V2 conversion wrapper → V2 DiT model: uses DiT model
- V1 inference → Encodec codec: optional codec path