How corentinj/real-time-voice-cloning works
Real-Time Voice Cloning implements the three-stage SV2TTS workflow: derive a speaker embedding from reference audio, synthesize a mel spectrogram from text and that embedding, then vocode it into speech. Users can run the workflow from a CLI or the toolbox; the README also documents preprocessing and training capabilities. The graph shows relationships supported by excerpts, the SOURCE INDEX, or README statements; unsampled subsystem internals are represented only where the tree supports a path.
Main entry points: synthesizer/inference.py Mel synthesis · __init__.py Toolbox coordinator · encoder/inference.py Speaker embedding · vocoder/inference.py Waveform inference
Components 22
User applications
- CLI workflow
demo_cli.py - Toolbox coordinator
toolbox/__init__.py - Toolbox UI
toolbox/ui.py
Speaker encoding
- Speaker embedding
encoder/inference.py - Speaker encoder model
encoder/model.py - Audio preprocessing
encoder/audio.py
Speech synthesis
- Mel synthesis
synthesizer/inference.py - Text processing
synthesizer/utils/text.py - Tacotron model
synthesizer/models/tacotron.py
Waveform generation
- Waveform inference
vocoder/inference.py - WaveRNN model
vocoder/models/fatchord_version.py
Data preparation and training
- Encoder preprocessing
encoder/preprocess.py - Encoder training
encoder/train.py - Synthesizer preprocessing
synthesizer/preprocess.py - Synthesizer training
synthesizer/train.py - Vocoder preprocessing
vocoder_preprocess.py - Vocoder training
vocoder/train.py
Other components
- User
- Reference audio
- Text prompt
- Mel spectrogram
- Generated speech
Connections 20
- User → CLI workflow: runs CLI
demo_cli.py - User → Toolbox coordinator: opens toolbox
README.md - Toolbox coordinator → Toolbox UI: uses interface
toolbox/__init__.py - User → Reference audio: provides audio
- CLI workflow → Speaker embedding: requests embedding
demo_cli.py - Speaker embedding → Audio preprocessing: preprocesses audio
encoder/inference.py - Speaker embedding → Speaker encoder model: computes embedding
encoder/inference.py - Speaker embedding → Mel synthesis: passes embedding
demo_cli.py - User → Text prompt: provides text
- CLI workflow → Mel synthesis: requests spectrograms
demo_cli.py - Mel synthesis → Text processing: encodes text
synthesizer/inference.py - Mel synthesis → Tacotron model: runs model
synthesizer/inference.py - Mel synthesis → Mel spectrogram: produces spectrogram
synthesizer/inference.py - CLI workflow → Waveform inference: requests waveform
demo_cli.py - Waveform inference → WaveRNN model: generates waveform
vocoder/inference.py - Waveform inference → Generated speech: returns speech
vocoder/inference.py - Mel spectrogram → Waveform inference: provides spectrogram
- Toolbox coordinator → Speaker embedding: uses encoder
toolbox/__init__.py - Toolbox coordinator → Mel synthesis: uses synthesizer
toolbox/__init__.py - Toolbox coordinator → Waveform inference: uses vocoder
toolbox/__init__.py