# corentinj/real-time-voice-cloning architecture

> Architecture diagram of the GitHub repository corentinj/real-time-voice-cloning, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/corentinj/real-time-voice-cloning
- Repository: https://github.com/corentinj/real-time-voice-cloning

## Overview

Real-Time Voice Cloning implements the three-stage SV2TTS workflow: derive a speaker embedding from reference audio, synthesize a mel spectrogram from text and that embedding, then vocode it into speech. Users can run the workflow from a CLI or the toolbox; the README also documents preprocessing and training capabilities. The graph shows relationships supported by excerpts, the SOURCE INDEX, or README statements; unsampled subsystem internals are represented only where the tree supports a path.

Main entry points: [`synthesizer/inference.py`](https://github.com/corentinj/real-time-voice-cloning/blob/master/synthesizer/inference.py) (Mel synthesis), [`toolbox/__init__.py`](https://github.com/corentinj/real-time-voice-cloning/blob/master/toolbox/__init__.py) (Toolbox coordinator), [`encoder/inference.py`](https://github.com/corentinj/real-time-voice-cloning/blob/master/encoder/inference.py) (Speaker embedding), [`vocoder/inference.py`](https://github.com/corentinj/real-time-voice-cloning/blob/master/vocoder/inference.py) (Waveform inference)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_experience["User applications"]
  node_cli["CLI workflow<br/>[demo_cli.py]"]
  node_toolbox["Toolbox coordinator<br/>[__init__.py]"]
  node_toolbox_ui["Toolbox UI<br/>[ui.py]"]
end

subgraph group_voice["Speaker encoding"]
  node_encoder_inference["Speaker embedding<br/>[inference.py]"]
  node_encoder_model["Speaker encoder model<br/>[model.py]"]
  node_encoder_audio["Audio preprocessing<br/>[audio.py]"]
end

subgraph group_speech["Speech synthesis"]
  node_synth_inference["Mel synthesis<br/>[inference.py]"]
  node_text_processing["Text processing<br/>[text.py]"]
  node_tacotron["Tacotron model<br/>[tacotron.py]"]
end

subgraph group_audio["Waveform generation"]
  node_vocoder_inference["Waveform inference<br/>[inference.py]"]
  node_wavernn["WaveRNN model"]
end

subgraph group_preparation["Data preparation and training"]
  node_encoder_preprocess["Encoder preprocessing<br/>[preprocess.py]"]
  node_encoder_training["Encoder training<br/>[train.py]"]
  node_synth_preprocess["Synthesizer preprocessing<br/>[preprocess.py]"]
  node_synth_training["Synthesizer training<br/>[train.py]"]
  node_vocoder_preprocess["Vocoder preprocessing"]
  node_vocoder_training["Vocoder training<br/>[train.py]"]
end

node_user(("User"))
node_reference_audio["Reference audio"]
node_text_input["Text prompt"]
node_mel_audio["Mel spectrogram"]
node_generated_audio["Generated speech"]

node_user -->|"runs CLI"| node_cli
node_user -->|"opens toolbox"| node_toolbox
node_toolbox -->|"uses interface"| node_toolbox_ui
node_user -->|"provides audio"| node_reference_audio
node_cli -->|"requests embedding"| node_encoder_inference
node_encoder_inference -->|"preprocesses audio"| node_encoder_audio
node_encoder_inference -->|"computes embedding"| node_encoder_model
node_encoder_inference -->|"passes embedding"| node_synth_inference
node_user -->|"provides text"| node_text_input
node_cli -->|"requests spectrograms"| node_synth_inference
node_synth_inference -->|"encodes text"| node_text_processing
node_synth_inference -->|"runs model"| node_tacotron
node_synth_inference -->|"produces spectrogram"| node_mel_audio
node_cli -->|"requests waveform"| node_vocoder_inference
node_vocoder_inference -->|"generates waveform"| node_wavernn
node_vocoder_inference -->|"returns speech"| node_generated_audio
node_mel_audio -->|"provides spectrogram"| node_vocoder_inference
node_toolbox -->|"uses encoder"| node_encoder_inference
node_toolbox -->|"uses synthesizer"| node_synth_inference
node_toolbox -->|"uses vocoder"| node_vocoder_inference

click node_cli "https://github.com/corentinj/real-time-voice-cloning/blob/master/demo_cli.py"
click node_toolbox "https://github.com/corentinj/real-time-voice-cloning/blob/master/toolbox/__init__.py"
click node_toolbox_ui "https://github.com/corentinj/real-time-voice-cloning/blob/master/toolbox/ui.py"
click node_encoder_inference "https://github.com/corentinj/real-time-voice-cloning/blob/master/encoder/inference.py"
click node_encoder_model "https://github.com/corentinj/real-time-voice-cloning/blob/master/encoder/model.py"
click node_encoder_audio "https://github.com/corentinj/real-time-voice-cloning/blob/master/encoder/audio.py"
click node_synth_inference "https://github.com/corentinj/real-time-voice-cloning/blob/master/synthesizer/inference.py"
click node_text_processing "https://github.com/corentinj/real-time-voice-cloning/blob/master/synthesizer/utils/text.py"
click node_tacotron "https://github.com/corentinj/real-time-voice-cloning/blob/master/synthesizer/models/tacotron.py"
click node_vocoder_inference "https://github.com/corentinj/real-time-voice-cloning/blob/master/vocoder/inference.py"
click node_wavernn "https://github.com/corentinj/real-time-voice-cloning/blob/master/vocoder/models/fatchord_version.py"
click node_encoder_preprocess "https://github.com/corentinj/real-time-voice-cloning/blob/master/encoder/preprocess.py"
click node_encoder_training "https://github.com/corentinj/real-time-voice-cloning/blob/master/encoder/train.py"
click node_synth_preprocess "https://github.com/corentinj/real-time-voice-cloning/blob/master/synthesizer/preprocess.py"
click node_synth_training "https://github.com/corentinj/real-time-voice-cloning/blob/master/synthesizer/train.py"
click node_vocoder_preprocess "https://github.com/corentinj/real-time-voice-cloning/blob/master/vocoder_preprocess.py"
click node_vocoder_training "https://github.com/corentinj/real-time-voice-cloning/blob/master/vocoder/train.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_toolbox,node_toolbox_ui,node_user toneBlue
class node_encoder_inference,node_encoder_model,node_encoder_audio toneAmber
class node_synth_inference,node_text_processing,node_tacotron toneMint
class node_vocoder_inference,node_wavernn toneRose
class node_encoder_preprocess,node_encoder_training,node_synth_preprocess,node_synth_training,node_vocoder_preprocess,node_vocoder_training,node_reference_audio,node_text_input,node_mel_audio,node_generated_audio toneIndigo
```

## Components

### User applications

- [CLI workflow](https://github.com/corentinj/real-time-voice-cloning/blob/master/demo_cli.py): `demo_cli.py`
- [Toolbox coordinator](https://github.com/corentinj/real-time-voice-cloning/blob/master/toolbox/__init__.py): `toolbox/__init__.py`
- [Toolbox UI](https://github.com/corentinj/real-time-voice-cloning/blob/master/toolbox/ui.py): `toolbox/ui.py`

### Speaker encoding

- [Speaker embedding](https://github.com/corentinj/real-time-voice-cloning/blob/master/encoder/inference.py): `encoder/inference.py`
- [Speaker encoder model](https://github.com/corentinj/real-time-voice-cloning/blob/master/encoder/model.py): `encoder/model.py`
- [Audio preprocessing](https://github.com/corentinj/real-time-voice-cloning/blob/master/encoder/audio.py): `encoder/audio.py`

### Speech synthesis

- [Mel synthesis](https://github.com/corentinj/real-time-voice-cloning/blob/master/synthesizer/inference.py): `synthesizer/inference.py`
- [Text processing](https://github.com/corentinj/real-time-voice-cloning/blob/master/synthesizer/utils/text.py): `synthesizer/utils/text.py`
- [Tacotron model](https://github.com/corentinj/real-time-voice-cloning/blob/master/synthesizer/models/tacotron.py): `synthesizer/models/tacotron.py`

### Waveform generation

- [Waveform inference](https://github.com/corentinj/real-time-voice-cloning/blob/master/vocoder/inference.py): `vocoder/inference.py`
- [WaveRNN model](https://github.com/corentinj/real-time-voice-cloning/blob/master/vocoder/models/fatchord_version.py): `vocoder/models/fatchord_version.py`

### Data preparation and training

- [Encoder preprocessing](https://github.com/corentinj/real-time-voice-cloning/blob/master/encoder/preprocess.py): `encoder/preprocess.py`
- [Encoder training](https://github.com/corentinj/real-time-voice-cloning/blob/master/encoder/train.py): `encoder/train.py`
- [Synthesizer preprocessing](https://github.com/corentinj/real-time-voice-cloning/blob/master/synthesizer/preprocess.py): `synthesizer/preprocess.py`
- [Synthesizer training](https://github.com/corentinj/real-time-voice-cloning/blob/master/synthesizer/train.py): `synthesizer/train.py`
- [Vocoder preprocessing](https://github.com/corentinj/real-time-voice-cloning/blob/master/vocoder_preprocess.py): `vocoder_preprocess.py`
- [Vocoder training](https://github.com/corentinj/real-time-voice-cloning/blob/master/vocoder/train.py): `vocoder/train.py`

### Other components

- **User**
- **Reference audio**
- **Text prompt**
- **Mel spectrogram**
- **Generated speech**

## Connections

- User → CLI workflow: runs CLI (evidence: `demo_cli.py`)
- User → Toolbox coordinator: opens toolbox (evidence: `README.md`)
- Toolbox coordinator → Toolbox UI: uses interface (evidence: `toolbox/__init__.py`)
- User → Reference audio: provides audio
- CLI workflow → Speaker embedding: requests embedding (evidence: `demo_cli.py`)
- Speaker embedding → Audio preprocessing: preprocesses audio (evidence: `encoder/inference.py`)
- Speaker embedding → Speaker encoder model: computes embedding (evidence: `encoder/inference.py`)
- Speaker embedding → Mel synthesis: passes embedding (evidence: `demo_cli.py`)
- User → Text prompt: provides text
- CLI workflow → Mel synthesis: requests spectrograms (evidence: `demo_cli.py`)
- Mel synthesis → Text processing: encodes text (evidence: `synthesizer/inference.py`)
- Mel synthesis → Tacotron model: runs model (evidence: `synthesizer/inference.py`)
- Mel synthesis → Mel spectrogram: produces spectrogram (evidence: `synthesizer/inference.py`)
- CLI workflow → Waveform inference: requests waveform (evidence: `demo_cli.py`)
- Waveform inference → WaveRNN model: generates waveform (evidence: `vocoder/inference.py`)
- Waveform inference → Generated speech: returns speech (evidence: `vocoder/inference.py`)
- Mel spectrogram → Waveform inference: provides spectrogram
- Toolbox coordinator → Speaker embedding: uses encoder (evidence: `toolbox/__init__.py`)
- Toolbox coordinator → Mel synthesis: uses synthesizer (evidence: `toolbox/__init__.py`)
- Toolbox coordinator → Waveform inference: uses vocoder (evidence: `toolbox/__init__.py`)
