# zhangyongmao/visinger2 architecture

> Architecture diagram of the GitHub repository zhangyongmao/visinger2, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/zhangyongmao/visinger2
- Repository: https://github.com/zhangyongmao/visinger2

## Overview

VISinger2 is a singing-voice synthesis system for OpenCpop-style phone, pitch, duration, and slur labels. Its documented workflow prepares acoustic features, trains a model, then converts labels into audio; the repository also provides a flow-prior variant. The map preserves both implementations and shared model utilities. Source excerpts are incomplete, so some documented workflow links rely on the README; unsampled implementation details and uncertain connections are omitted.

Main entry points: [`egs/visinger2/models.py`](https://github.com/zhangyongmao/visinger2/blob/master/egs/visinger2/models.py) (VISinger2 model), [`egs/visinger2/train.py`](https://github.com/zhangyongmao/visinger2/blob/master/egs/visinger2/train.py) (Base training), [`egs/visinger2_flow/train.py`](https://github.com/zhangyongmao/visinger2/blob/master/egs/visinger2_flow/train.py) (Flow training), [`egs/visinger2_flow/inference.py`](https://github.com/zhangyongmao/visinger2/blob/master/egs/visinger2_flow/inference.py) (Flow inference)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Data preparation"]
  node_corpus[("OpenCpop corpus")]
  node_preprocess["Feature extraction<br/>[preprocess.py]"]
  node_mel["Mel processing<br/>[mel_processing.py]"]
  node_symbols["Phone and pitch IDs<br/>[symbols.py]"]
  node_converter["Symbol conversion"]
  node_features[("Pitch and mel features")]
  node_dataset["Training datasets<br/>[dataset.py]"]
end

subgraph group_models["Synthesis models"]
  node_base_model["VISinger2 model<br/>[models.py]"]
  node_flow_model["Flow model<br/>[models.py]"]
  node_attention["Attention layers<br/>[attentions.py]"]
  node_model_modules["Model modules<br/>[modules.py]"]
  node_ddsp["DDSP synthesis<br/>[ddsp.py]"]
end

subgraph group_training["Training"]
  node_base_train["Base training<br/>[train.py]"]
  node_flow_train["Flow training<br/>[train.py]"]
  node_losses["Training losses<br/>[losses.py]"]
end

subgraph group_inference["Inference"]
  node_base_inference["Base inference<br/>[inference.py]"]
  node_flow_inference["Flow inference<br/>[inference.py]"]
  node_audio["Audio utilities<br/>[audio.py]"]
  node_output["Synthesized WAV"]
end

node_user(("Researcher"))

node_user -->|"provides data"| node_corpus
node_corpus -->|"extracts features"| node_preprocess
node_preprocess -->|"writes pitch and mel"| node_features
node_symbols -->|"defines symbol sets"| node_converter
node_user -->|"starts training"| node_base_train
node_features -.->|"loads training data"| node_dataset
node_base_train -.->|"constructs loaders"| node_dataset
node_base_train -.->|"trains model"| node_base_model
node_base_train -.->|"computes losses"| node_losses
node_base_train -.->|"uses mel functions"| node_mel
node_user -->|"starts training"| node_flow_train
node_flow_train -->|"constructs loaders"| node_dataset
node_flow_train -->|"imports model"| node_base_model
node_flow_train -->|"uses losses"| node_losses
node_flow_train -->|"uses mel functions"| node_mel
node_base_model -.->|"uses attention"| node_attention
node_base_model -.->|"uses modules"| node_model_modules
node_base_model -.->|"uses DDSP"| node_ddsp
node_user -->|"requests synthesis"| node_base_inference
node_base_inference -.->|"runs inference"| node_base_model
node_base_inference -.->|"writes audio"| node_output
node_user -->|"requests synthesis"| node_flow_inference
node_flow_inference -->|"loads and runs"| node_base_model
node_flow_inference -->|"maps labels"| node_converter
node_flow_inference -->|"writes WAV"| node_output

click node_corpus "https://github.com/zhangyongmao/visinger2/tree/master/data/opencpop"
click node_preprocess "https://github.com/zhangyongmao/visinger2/blob/master/preprocess/preprocess.py"
click node_mel "https://github.com/zhangyongmao/visinger2/blob/master/preprocess/mel_processing.py"
click node_symbols "https://github.com/zhangyongmao/visinger2/blob/master/text/npu/symbols.py"
click node_converter "https://github.com/zhangyongmao/visinger2/blob/master/text/npu/symbol_converter.py"
click node_features "https://github.com/zhangyongmao/visinger2/tree/master/data/opencpop"
click node_dataset "https://github.com/zhangyongmao/visinger2/blob/master/egs/visinger2/dataset.py"
click node_base_model "https://github.com/zhangyongmao/visinger2/blob/master/egs/visinger2/models.py"
click node_flow_model "https://github.com/zhangyongmao/visinger2/blob/master/egs/visinger2_flow/models.py"
click node_attention "https://github.com/zhangyongmao/visinger2/blob/master/modules/attentions.py"
click node_model_modules "https://github.com/zhangyongmao/visinger2/blob/master/modules/modules.py"
click node_ddsp "https://github.com/zhangyongmao/visinger2/blob/master/modules/ddsp.py"
click node_base_train "https://github.com/zhangyongmao/visinger2/blob/master/egs/visinger2/train.py"
click node_flow_train "https://github.com/zhangyongmao/visinger2/blob/master/egs/visinger2_flow/train.py"
click node_losses "https://github.com/zhangyongmao/visinger2/blob/master/modules/losses.py"
click node_base_inference "https://github.com/zhangyongmao/visinger2/blob/master/egs/visinger2/inference.py"
click node_flow_inference "https://github.com/zhangyongmao/visinger2/blob/master/egs/visinger2_flow/inference.py"
click node_audio "https://github.com/zhangyongmao/visinger2/blob/master/utils/audio.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_corpus,node_preprocess,node_mel,node_symbols,node_converter,node_features,node_dataset toneBlue
class node_base_model,node_flow_model,node_attention,node_model_modules,node_ddsp toneAmber
class node_base_train,node_flow_train,node_losses toneMint
class node_base_inference,node_flow_inference,node_audio,node_output toneRose
class node_user toneIndigo
```

## Components

### Data preparation

- [OpenCpop corpus](https://github.com/zhangyongmao/visinger2/tree/master/data/opencpop): `data/opencpop`
- [Feature extraction](https://github.com/zhangyongmao/visinger2/blob/master/preprocess/preprocess.py): `preprocess/preprocess.py`
- [Mel processing](https://github.com/zhangyongmao/visinger2/blob/master/preprocess/mel_processing.py): `preprocess/mel_processing.py`
- [Phone and pitch IDs](https://github.com/zhangyongmao/visinger2/blob/master/text/npu/symbols.py): `text/npu/symbols.py`
- [Symbol conversion](https://github.com/zhangyongmao/visinger2/blob/master/text/npu/symbol_converter.py): `text/npu/symbol_converter.py`
- [Pitch and mel features](https://github.com/zhangyongmao/visinger2/tree/master/data/opencpop): `data/opencpop`
- [Training datasets](https://github.com/zhangyongmao/visinger2/blob/master/egs/visinger2/dataset.py): `egs/visinger2/dataset.py`

### Synthesis models

- [VISinger2 model](https://github.com/zhangyongmao/visinger2/blob/master/egs/visinger2/models.py): `egs/visinger2/models.py`
- [Flow model](https://github.com/zhangyongmao/visinger2/blob/master/egs/visinger2_flow/models.py): `egs/visinger2_flow/models.py`
- [Attention layers](https://github.com/zhangyongmao/visinger2/blob/master/modules/attentions.py): `modules/attentions.py`
- [Model modules](https://github.com/zhangyongmao/visinger2/blob/master/modules/modules.py): `modules/modules.py`
- [DDSP synthesis](https://github.com/zhangyongmao/visinger2/blob/master/modules/ddsp.py): `modules/ddsp.py`

### Training

- [Base training](https://github.com/zhangyongmao/visinger2/blob/master/egs/visinger2/train.py): `egs/visinger2/train.py`
- [Flow training](https://github.com/zhangyongmao/visinger2/blob/master/egs/visinger2_flow/train.py): `egs/visinger2_flow/train.py`
- [Training losses](https://github.com/zhangyongmao/visinger2/blob/master/modules/losses.py): `modules/losses.py`

### Inference

- [Base inference](https://github.com/zhangyongmao/visinger2/blob/master/egs/visinger2/inference.py): `egs/visinger2/inference.py`
- [Flow inference](https://github.com/zhangyongmao/visinger2/blob/master/egs/visinger2_flow/inference.py): `egs/visinger2_flow/inference.py`
- [Audio utilities](https://github.com/zhangyongmao/visinger2/blob/master/utils/audio.py): `utils/audio.py`
- **Synthesized WAV**

### Other components

- **Researcher**

## Connections

- Researcher → OpenCpop corpus: provides data (evidence: `README.md`)
- OpenCpop corpus → Feature extraction: extracts features (evidence: `README.md`)
- Feature extraction → Pitch and mel features: writes pitch and mel (evidence: `README.md`)
- Phone and pitch IDs → Symbol conversion: defines symbol sets (evidence: `text/npu/symbol_converter.py`)
- Researcher → Base training: starts training (evidence: `README.md`)
- Pitch and mel features → Training datasets: loads training data
- Base training → Training datasets: constructs loaders
- Base training → VISinger2 model: trains model
- Base training → Training losses: computes losses
- Base training → Mel processing: uses mel functions
- Researcher → Flow training: starts training (evidence: `README.md`)
- Flow training → Training datasets: constructs loaders (evidence: `egs/visinger2_flow/train.py`)
- Flow training → VISinger2 model: imports model (evidence: `egs/visinger2_flow/train.py`)
- Flow training → Training losses: uses losses (evidence: `egs/visinger2_flow/train.py`)
- Flow training → Mel processing: uses mel functions (evidence: `egs/visinger2_flow/train.py`)
- VISinger2 model → Attention layers: uses attention
- VISinger2 model → Model modules: uses modules
- VISinger2 model → DDSP synthesis: uses DDSP
- Researcher → Base inference: requests synthesis (evidence: `README.md`)
- Base inference → VISinger2 model: runs inference
- Base inference → Synthesized WAV: writes audio
- Researcher → Flow inference: requests synthesis (evidence: `README.md`)
- Flow inference → VISinger2 model: loads and runs (evidence: `egs/visinger2_flow/inference.py`)
- Flow inference → Symbol conversion: maps labels (evidence: `egs/visinger2_flow/inference.py`)
- Flow inference → Synthesized WAV: writes WAV (evidence: `egs/visinger2_flow/inference.py`)
