# soul-ailab/soulx-singer architecture

> Architecture diagram of the GitHub repository soul-ailab/soulx-singer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/soul-ailab/soulx-singer
- Repository: https://github.com/soul-ailab/soulx-singer

## Overview

SoulX-Singer provides zero-shot singing synthesis with melody/F0 or score/MIDI control, plus a transcription-free singing voice conversion mode. Users can run inference through command-line entry points or the documented WebUIs. A separate preprocessing workflow prepares vocals and aligned lyric/note metadata, which can be reviewed in the MIDI editor before synthesis. The graph captures documented capabilities alongside sampled calls; internal model wiring and WebUI handlers are not fully shown in the excerpts, so uncertain links are omitted.

Main entry points: [`preprocess/pipeline.py`](https://github.com/soul-ailab/soulx-singer/blob/main/preprocess/pipeline.py) (Preprocess pipeline), [`soulxsinger/models/soulxsinger_svc.py`](https://github.com/soul-ailab/soulx-singer/blob/main/soulxsinger/models/soulxsinger_svc.py) (SoulX-Singer-SVC), [`cli/inference.py`](https://github.com/soul-ailab/soulx-singer/blob/main/cli/inference.py) (SVS inference), [`preprocess/tools/midi_editor/src/App.tsx`](https://github.com/soul-ailab/soulx-singer/blob/main/preprocess/tools/midi_editor/src/App.tsx) (MIDI editor)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interfaces["User interfaces"]
  node_svs_ui["SVS WebUI<br/>[webui.py]"]
  node_svc_ui["SVC WebUI<br/>[webui_svc.py]"]
  node_midi_editor["MIDI editor<br/>[App.tsx]"]
  node_midi_store[("MIDI state<br/>[useMidiStore.ts]")]
  node_midi_codec["MIDI import/export<br/>[midi.ts]"]
end

subgraph group_preparation["Audio preparation"]
  node_pipeline["Preprocess pipeline<br/>[pipeline.py]"]
  node_separator["Vocal separation<br/>[model.py]"]
  node_vocal_detect["Vocal segmentation<br/>[vocal_detection.py]"]
  node_f0["F0 extraction<br/>[f0_extraction.py]"]
  node_lyrics["Lyric transcription"]
  node_notes["Note transcription<br/>[model.py]"]
  node_metadata["Aligned metadata"]
  node_midi_parser["Metadata MIDI conversion<br/>[midi_parser.py]"]
end

subgraph group_synthesis["Singing synthesis"]
  node_svs_cli["SVS inference<br/>[inference.py]"]
  node_data_processor["Synthesis data prep<br/>[data_processor.py]"]
  node_svs_model["SoulX-Singer<br/>[soulxsinger.py]"]
end

subgraph group_conversion["Voice conversion"]
  node_svc_cli["SVC inference<br/>[inference_svc.py]"]
  node_svc_model["SoulX-Singer-SVC<br/>[soulxsinger_svc.py]"]
end

subgraph group_model["Model layers"]
  node_whisper["Content encoder<br/>[whisper_encoder.py]"]
  node_flow["Flow decoder<br/>[flow_matching.py]"]
  node_mel["Mel transform<br/>[mel_transform.py]"]
  node_vocoder["Vocoder<br/>[vocoder.py]"]
end

node_user(("User"))
node_input_audio["Audio input"]
node_generated["Generated audio"]

node_user -->|"requests synthesis"| node_svs_ui
node_user -->|"requests conversion"| node_svc_ui
node_user -->|"runs inference"| node_svs_cli
node_user -->|"runs inference"| node_svc_cli
node_user -->|"submits audio"| node_pipeline
node_input_audio -->|"audio input"| node_pipeline
node_user -->|"edits alignment"| node_midi_editor
node_midi_editor -->|"updates notes"| node_midi_store
node_midi_editor -->|"imports/exports"| node_midi_codec
node_pipeline -->|"separates vocals"| node_separator
node_pipeline -->|"extracts pitch"| node_f0
node_pipeline -->|"segments vocals"| node_vocal_detect
node_pipeline -->|"transcribes lyrics"| node_lyrics
node_pipeline -->|"transcribes notes"| node_notes
node_pipeline -->|"writes metadata"| node_metadata
node_metadata -->|"converts format"| node_midi_parser
node_midi_parser -->|"loads MIDI"| node_midi_editor
node_metadata -->|"conditions synthesis"| node_svs_cli
node_svs_cli -->|"prepares inputs"| node_data_processor
node_svs_cli -->|"runs inference"| node_svs_model
node_svs_model -->|"generates singing"| node_generated
node_input_audio -->|"source recording"| node_svc_cli
node_svc_cli -->|"runs inference"| node_svc_model
node_svc_model -->|"converts voice"| node_generated
node_svc_model -->|"encodes content"| node_whisper
node_svc_model -->|"decodes features"| node_flow
node_svc_model -->|"processes spectrum"| node_mel
node_svc_model -->|"synthesizes waveform"| node_vocoder
node_svs_ui -.->|"launches inference"| node_svs_cli
node_svc_ui -.->|"launches inference"| node_svc_cli

click node_svs_ui "https://github.com/soul-ailab/soulx-singer/blob/main/webui.py"
click node_svc_ui "https://github.com/soul-ailab/soulx-singer/blob/main/webui_svc.py"
click node_midi_editor "https://github.com/soul-ailab/soulx-singer/blob/main/preprocess/tools/midi_editor/src/App.tsx"
click node_midi_store "https://github.com/soul-ailab/soulx-singer/blob/main/preprocess/tools/midi_editor/src/store/useMidiStore.ts"
click node_midi_codec "https://github.com/soul-ailab/soulx-singer/blob/main/preprocess/tools/midi_editor/src/lib/midi.ts"
click node_pipeline "https://github.com/soul-ailab/soulx-singer/blob/main/preprocess/pipeline.py"
click node_separator "https://github.com/soul-ailab/soulx-singer/blob/main/preprocess/tools/vocal_separation/model.py"
click node_vocal_detect "https://github.com/soul-ailab/soulx-singer/blob/main/preprocess/tools/vocal_detection.py"
click node_f0 "https://github.com/soul-ailab/soulx-singer/blob/main/preprocess/tools/f0_extraction.py"
click node_lyrics "https://github.com/soul-ailab/soulx-singer/blob/main/preprocess/tools/lyric_transcription.py"
click node_notes "https://github.com/soul-ailab/soulx-singer/blob/main/preprocess/tools/note_transcription/model.py"
click node_midi_parser "https://github.com/soul-ailab/soulx-singer/blob/main/preprocess/tools/midi_parser.py"
click node_svs_cli "https://github.com/soul-ailab/soulx-singer/blob/main/cli/inference.py"
click node_data_processor "https://github.com/soul-ailab/soulx-singer/blob/main/soulxsinger/utils/data_processor.py"
click node_svs_model "https://github.com/soul-ailab/soulx-singer/blob/main/soulxsinger/models/soulxsinger.py"
click node_svc_cli "https://github.com/soul-ailab/soulx-singer/blob/main/cli/inference_svc.py"
click node_svc_model "https://github.com/soul-ailab/soulx-singer/blob/main/soulxsinger/models/soulxsinger_svc.py"
click node_whisper "https://github.com/soul-ailab/soulx-singer/blob/main/soulxsinger/models/modules/whisper_encoder.py"
click node_flow "https://github.com/soul-ailab/soulx-singer/blob/main/soulxsinger/models/modules/flow_matching.py"
click node_mel "https://github.com/soul-ailab/soulx-singer/blob/main/soulxsinger/models/modules/mel_transform.py"
click node_vocoder "https://github.com/soul-ailab/soulx-singer/blob/main/soulxsinger/models/modules/vocoder.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_svs_ui,node_svc_ui,node_midi_editor,node_midi_store,node_midi_codec,node_user toneBlue
class node_pipeline,node_separator,node_vocal_detect,node_f0,node_lyrics,node_notes,node_metadata,node_midi_parser toneAmber
class node_svs_cli,node_data_processor,node_svs_model toneMint
class node_svc_cli,node_svc_model toneRose
class node_whisper,node_flow,node_mel,node_vocoder,node_input_audio,node_generated toneIndigo
```

## Components

### User interfaces

- [SVS WebUI](https://github.com/soul-ailab/soulx-singer/blob/main/webui.py): `webui.py`
- [SVC WebUI](https://github.com/soul-ailab/soulx-singer/blob/main/webui_svc.py): `webui_svc.py`
- [MIDI editor](https://github.com/soul-ailab/soulx-singer/blob/main/preprocess/tools/midi_editor/src/App.tsx): `preprocess/tools/midi_editor/src/App.tsx`
- [MIDI state](https://github.com/soul-ailab/soulx-singer/blob/main/preprocess/tools/midi_editor/src/store/useMidiStore.ts): `preprocess/tools/midi_editor/src/store/useMidiStore.ts`
- [MIDI import/export](https://github.com/soul-ailab/soulx-singer/blob/main/preprocess/tools/midi_editor/src/lib/midi.ts): `preprocess/tools/midi_editor/src/lib/midi.ts`

### Audio preparation

- [Preprocess pipeline](https://github.com/soul-ailab/soulx-singer/blob/main/preprocess/pipeline.py): `preprocess/pipeline.py`
- [Vocal separation](https://github.com/soul-ailab/soulx-singer/blob/main/preprocess/tools/vocal_separation/model.py): `preprocess/tools/vocal_separation/model.py`
- [Vocal segmentation](https://github.com/soul-ailab/soulx-singer/blob/main/preprocess/tools/vocal_detection.py): `preprocess/tools/vocal_detection.py`
- [F0 extraction](https://github.com/soul-ailab/soulx-singer/blob/main/preprocess/tools/f0_extraction.py): `preprocess/tools/f0_extraction.py`
- [Lyric transcription](https://github.com/soul-ailab/soulx-singer/blob/main/preprocess/tools/lyric_transcription.py): `preprocess/tools/lyric_transcription.py`
- [Note transcription](https://github.com/soul-ailab/soulx-singer/blob/main/preprocess/tools/note_transcription/model.py): `preprocess/tools/note_transcription/model.py`
- **Aligned metadata**
- [Metadata MIDI conversion](https://github.com/soul-ailab/soulx-singer/blob/main/preprocess/tools/midi_parser.py): `preprocess/tools/midi_parser.py`

### Singing synthesis

- [SVS inference](https://github.com/soul-ailab/soulx-singer/blob/main/cli/inference.py): `cli/inference.py`
- [Synthesis data prep](https://github.com/soul-ailab/soulx-singer/blob/main/soulxsinger/utils/data_processor.py): `soulxsinger/utils/data_processor.py`
- [SoulX-Singer](https://github.com/soul-ailab/soulx-singer/blob/main/soulxsinger/models/soulxsinger.py): `soulxsinger/models/soulxsinger.py`

### Voice conversion

- [SVC inference](https://github.com/soul-ailab/soulx-singer/blob/main/cli/inference_svc.py): `cli/inference_svc.py`
- [SoulX-Singer-SVC](https://github.com/soul-ailab/soulx-singer/blob/main/soulxsinger/models/soulxsinger_svc.py): `soulxsinger/models/soulxsinger_svc.py`

### Model layers

- [Content encoder](https://github.com/soul-ailab/soulx-singer/blob/main/soulxsinger/models/modules/whisper_encoder.py): `soulxsinger/models/modules/whisper_encoder.py`
- [Flow decoder](https://github.com/soul-ailab/soulx-singer/blob/main/soulxsinger/models/modules/flow_matching.py): `soulxsinger/models/modules/flow_matching.py`
- [Mel transform](https://github.com/soul-ailab/soulx-singer/blob/main/soulxsinger/models/modules/mel_transform.py): `soulxsinger/models/modules/mel_transform.py`
- [Vocoder](https://github.com/soul-ailab/soulx-singer/blob/main/soulxsinger/models/modules/vocoder.py): `soulxsinger/models/modules/vocoder.py`

### Other components

- **User**
- **Audio input**
- **Generated audio**

## Connections

- User → SVS WebUI: requests synthesis
- User → SVC WebUI: requests conversion
- User → SVS inference: runs inference
- User → SVC inference: runs inference
- User → Preprocess pipeline: submits audio
- Audio input → Preprocess pipeline: audio input
- User → MIDI editor: edits alignment
- MIDI editor → MIDI state: updates notes
- MIDI editor → MIDI import/export: imports/exports
- Preprocess pipeline → Vocal separation: separates vocals
- Preprocess pipeline → F0 extraction: extracts pitch
- Preprocess pipeline → Vocal segmentation: segments vocals
- Preprocess pipeline → Lyric transcription: transcribes lyrics
- Preprocess pipeline → Note transcription: transcribes notes
- Preprocess pipeline → Aligned metadata: writes metadata
- Aligned metadata → Metadata MIDI conversion: converts format
- Metadata MIDI conversion → MIDI editor: loads MIDI
- Aligned metadata → SVS inference: conditions synthesis
- SVS inference → Synthesis data prep: prepares inputs
- SVS inference → SoulX-Singer: runs inference
- SoulX-Singer → Generated audio: generates singing
- Audio input → SVC inference: source recording
- SVC inference → SoulX-Singer-SVC: runs inference
- SoulX-Singer-SVC → Generated audio: converts voice
- SoulX-Singer-SVC → Content encoder: encodes content
- SoulX-Singer-SVC → Flow decoder: decodes features
- SoulX-Singer-SVC → Mel transform: processes spectrum
- SoulX-Singer-SVC → Vocoder: synthesizes waveform
- SVS WebUI → SVS inference: launches inference
- SVC WebUI → SVC inference: launches inference