# babysor/mockingbird architecture

> Architecture diagram of the GitHub repository babysor/mockingbird, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/babysor/mockingbird
- Repository: https://github.com/babysor/mockingbird

## Overview

MockingBird is a voice-cloning and voice-conversion system with command-line, desktop-toolbox, and web interfaces. In the principal workflow, a user supplies speech and a target voice; the encoder derives a speaker representation, the synthesizer generates mel spectrograms from text and that representation, and a vocoder produces waveform audio. Voice conversion adds PPG extraction and PPG-to-mel conversion. Separate documented workflows preprocess datasets and train encoder, synthesizer, PPG-to-mel, and vocoder models. The graph includes these documented capabilities; several training and web relationships are inferred from README descriptions because their implementation was not sampled.

Main entry points: [`control/toolbox/ui.py`](https://github.com/babysor/mockingbird/blob/main/control/toolbox/ui.py) (Desktop toolbox), [`control/mkgui/app_vc.py`](https://github.com/babysor/mockingbird/blob/main/control/mkgui/app_vc.py) (Conversion app), [`models/synthesizer/preprocess.py`](https://github.com/babysor/mockingbird/blob/main/models/synthesizer/preprocess.py) (Dataset preprocessing), [`control/mkgui/app.py`](https://github.com/babysor/mockingbird/blob/main/control/mkgui/app.py) (Web synthesis app)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interfaces["User interfaces"]
  node_toolbox["Desktop toolbox<br/>[ui.py]"]
  node_webui["Web interface<br/>[streamlit_ui.py]"]
  node_webapp["Web synthesis app<br/>[app.py]"]
  node_vcapp["Conversion app<br/>[app_vc.py]"]
end

subgraph group_voice["Voice generation"]
  node_speaker["Speaker encoder<br/>[inference.py]"]
  node_synth["Text synthesizer<br/>[inference.py]"]
  node_hifigan["HiFi-GAN vocoder<br/>[inference.py]"]
  node_fregan["Fre-GAN vocoder<br/>[inference.py]"]
  node_wavernn["WaveRNN vocoder<br/>[inference.py]"]
end

subgraph group_conversion["Voice conversion"]
  node_ppg["PPG extractor"]
  node_ppgmel["PPG-to-mel decoder<br/>[rnn_decoder_mol.py]"]
end

subgraph group_training["Data and training"]
  node_preprocess["Dataset preprocessing<br/>[preprocess.py]"]
  node_encodertrain["Encoder training<br/>[train.py]"]
  node_synthtrain["Synthesizer training<br/>[train.py]"]
  node_ppgtrain["PPG-mel training<br/>[loss.py]"]
  node_vocodertrain["Vocoder training<br/>[train.py]"]
end

subgraph group_shared["Shared audio"]
  node_audio["Audio utilities<br/>[audio.py]"]
  node_result["Generated audio"]
end

node_user(("User"))

node_user -->|"submits speech"| node_toolbox
node_user -->|"submits request"| node_webui
node_webui -->|"routes request"| node_webapp
node_webapp -->|"loads model"| node_speaker
node_webapp -->|"loads model"| node_synth
node_webapp -->|"loads model"| node_hifigan
node_toolbox -->|"uses encoder"| node_speaker
node_toolbox -->|"uses synthesizer"| node_synth
node_toolbox -->|"uses vocoder"| node_hifigan
node_toolbox -->|"uses vocoder"| node_fregan
node_toolbox -->|"uses vocoder"| node_wavernn
node_speaker -->|"preprocesses speech"| node_audio
node_synth -->|"generates spectrogram"| node_result
node_hifigan -->|"generates waveform"| node_result
node_fregan -->|"generates waveform"| node_result
node_wavernn -->|"generates waveform"| node_result
node_user -->|"requests conversion"| node_vcapp
node_vcapp -.->|"extracts PPG"| node_ppg
node_vcapp -.->|"converts features"| node_ppgmel
node_vcapp -.->|"loads encoder"| node_speaker
node_vcapp -.->|"loads vocoder"| node_hifigan
node_ppgmel -.->|"produces mel"| node_result
node_preprocess -.->|"prepares data"| node_synthtrain
node_preprocess -.->|"prepares data"| node_encodertrain
node_preprocess -.->|"prepares data"| node_ppgtrain
node_preprocess -.->|"prepares data"| node_vocodertrain
node_user -->|"provides dataset"| node_preprocess

click node_toolbox "https://github.com/babysor/mockingbird/blob/main/control/toolbox/ui.py"
click node_webui "https://github.com/babysor/mockingbird/blob/main/control/mkgui/base/ui/streamlit_ui.py"
click node_webapp "https://github.com/babysor/mockingbird/blob/main/control/mkgui/app.py"
click node_speaker "https://github.com/babysor/mockingbird/blob/main/models/encoder/inference.py"
click node_synth "https://github.com/babysor/mockingbird/blob/main/models/synthesizer/inference.py"
click node_hifigan "https://github.com/babysor/mockingbird/blob/main/models/vocoder/hifigan/inference.py"
click node_fregan "https://github.com/babysor/mockingbird/blob/main/models/vocoder/fregan/inference.py"
click node_wavernn "https://github.com/babysor/mockingbird/blob/main/models/vocoder/wavernn/inference.py"
click node_ppg "https://github.com/babysor/mockingbird/tree/main/models/ppg_extractor"
click node_ppgmel "https://github.com/babysor/mockingbird/blob/main/models/ppg2mel/rnn_decoder_mol.py"
click node_vcapp "https://github.com/babysor/mockingbird/blob/main/control/mkgui/app_vc.py"
click node_preprocess "https://github.com/babysor/mockingbird/blob/main/models/synthesizer/preprocess.py"
click node_encodertrain "https://github.com/babysor/mockingbird/blob/main/models/encoder/train.py"
click node_synthtrain "https://github.com/babysor/mockingbird/blob/main/models/synthesizer/train.py"
click node_ppgtrain "https://github.com/babysor/mockingbird/blob/main/models/ppg2mel/train/loss.py"
click node_vocodertrain "https://github.com/babysor/mockingbird/blob/main/models/vocoder/hifigan/train.py"
click node_audio "https://github.com/babysor/mockingbird/blob/main/models/encoder/audio.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_toolbox,node_webui,node_webapp,node_vcapp,node_user toneBlue
class node_speaker,node_synth,node_hifigan,node_fregan,node_wavernn toneAmber
class node_ppg,node_ppgmel toneMint
class node_preprocess,node_encodertrain,node_synthtrain,node_ppgtrain,node_vocodertrain toneRose
class node_audio,node_result toneIndigo
```

## Components

### User interfaces

- [Desktop toolbox](https://github.com/babysor/mockingbird/blob/main/control/toolbox/ui.py): `control/toolbox/ui.py`
- [Web interface](https://github.com/babysor/mockingbird/blob/main/control/mkgui/base/ui/streamlit_ui.py): `control/mkgui/base/ui/streamlit_ui.py`
- [Web synthesis app](https://github.com/babysor/mockingbird/blob/main/control/mkgui/app.py): `control/mkgui/app.py`
- [Conversion app](https://github.com/babysor/mockingbird/blob/main/control/mkgui/app_vc.py): `control/mkgui/app_vc.py`

### Voice generation

- [Speaker encoder](https://github.com/babysor/mockingbird/blob/main/models/encoder/inference.py): `models/encoder/inference.py`
- [Text synthesizer](https://github.com/babysor/mockingbird/blob/main/models/synthesizer/inference.py): `models/synthesizer/inference.py`
- [HiFi-GAN vocoder](https://github.com/babysor/mockingbird/blob/main/models/vocoder/hifigan/inference.py): `models/vocoder/hifigan/inference.py`
- [Fre-GAN vocoder](https://github.com/babysor/mockingbird/blob/main/models/vocoder/fregan/inference.py): `models/vocoder/fregan/inference.py`
- [WaveRNN vocoder](https://github.com/babysor/mockingbird/blob/main/models/vocoder/wavernn/inference.py): `models/vocoder/wavernn/inference.py`

### Voice conversion

- [PPG extractor](https://github.com/babysor/mockingbird/tree/main/models/ppg_extractor): `models/ppg_extractor`
- [PPG-to-mel decoder](https://github.com/babysor/mockingbird/blob/main/models/ppg2mel/rnn_decoder_mol.py): `models/ppg2mel/rnn_decoder_mol.py`

### Data and training

- [Dataset preprocessing](https://github.com/babysor/mockingbird/blob/main/models/synthesizer/preprocess.py): `models/synthesizer/preprocess.py`
- [Encoder training](https://github.com/babysor/mockingbird/blob/main/models/encoder/train.py): `models/encoder/train.py`
- [Synthesizer training](https://github.com/babysor/mockingbird/blob/main/models/synthesizer/train.py): `models/synthesizer/train.py`
- [PPG-mel training](https://github.com/babysor/mockingbird/blob/main/models/ppg2mel/train/loss.py): `models/ppg2mel/train/loss.py`
- [Vocoder training](https://github.com/babysor/mockingbird/blob/main/models/vocoder/hifigan/train.py): `models/vocoder/hifigan/train.py`

### Shared audio

- [Audio utilities](https://github.com/babysor/mockingbird/blob/main/models/encoder/audio.py): `models/encoder/audio.py`
- **Generated audio**

### Other components

- **User**

## Connections

- User → Desktop toolbox: submits speech
- User → Web interface: submits request
- Web interface → Web synthesis app: routes request (evidence: `control/mkgui/base/ui/streamlit_ui.py`)
- Web synthesis app → Speaker encoder: loads model (evidence: `control/mkgui/app.py`)
- Web synthesis app → Text synthesizer: loads model (evidence: `control/mkgui/app.py`)
- Web synthesis app → HiFi-GAN vocoder: loads model (evidence: `control/mkgui/app.py`)
- Desktop toolbox → Speaker encoder: uses encoder (evidence: `control/toolbox/__init__.py`)
- Desktop toolbox → Text synthesizer: uses synthesizer (evidence: `control/toolbox/__init__.py`)
- Desktop toolbox → HiFi-GAN vocoder: uses vocoder (evidence: `control/toolbox/__init__.py`)
- Desktop toolbox → Fre-GAN vocoder: uses vocoder (evidence: `control/toolbox/__init__.py`)
- Desktop toolbox → WaveRNN vocoder: uses vocoder (evidence: `control/toolbox/__init__.py`)
- Speaker encoder → Audio utilities: preprocesses speech (evidence: `models/encoder/inference.py`)
- Text synthesizer → Generated audio: generates spectrogram
- HiFi-GAN vocoder → Generated audio: generates waveform (evidence: `models/vocoder/hifigan/inference.py`)
- Fre-GAN vocoder → Generated audio: generates waveform (evidence: `models/vocoder/fregan/inference.py`)
- WaveRNN vocoder → Generated audio: generates waveform (evidence: `models/vocoder/wavernn/inference.py`)
- User → Conversion app: requests conversion
- Conversion app → PPG extractor: extracts PPG (evidence: `control/mkgui/app_vc.py`)
- Conversion app → PPG-to-mel decoder: converts features
- Conversion app → Speaker encoder: loads encoder (evidence: `control/mkgui/app_vc.py`)
- Conversion app → HiFi-GAN vocoder: loads vocoder (evidence: `control/mkgui/app_vc.py`)
- PPG-to-mel decoder → Generated audio: produces mel
- Dataset preprocessing → Synthesizer training: prepares data
- Dataset preprocessing → Encoder training: prepares data
- Dataset preprocessing → PPG-mel training: prepares data
- Dataset preprocessing → Vocoder training: prepares data
- User → Dataset preprocessing: provides dataset
