How soul-ailab/soulx-singer works
SoulX-Singer provides zero-shot singing synthesis with melody/F0 or score/MIDI control, plus a transcription-free singing voice conversion mode. Users can run inference through command-line entry points or the documented WebUIs. A separate preprocessing workflow prepares vocals and aligned lyric/note metadata, which can be reviewed in the MIDI editor before synthesis. The graph captures documented capabilities alongside sampled calls; internal model wiring and WebUI handlers are not fully shown in the excerpts, so uncertain links are omitted.
Main entry points: pipeline.py Preprocess pipeline · soulxsinger_svc.py SoulX-Singer-SVC · inference.py SVS inference · App.tsx MIDI editor
Components 25
User interfaces
- SVS WebUI
webui.py - SVC WebUI
webui_svc.py - MIDI editor
preprocess/tools/midi_editor/src/App.tsx - MIDI state
preprocess/tools/midi_editor/src/store/useMidiStore.ts - MIDI import/export
preprocess/tools/midi_editor/src/lib/midi.ts
Audio preparation
- Preprocess pipeline
preprocess/pipeline.py - Vocal separation
preprocess/tools/vocal_separation/model.py - Vocal segmentation
preprocess/tools/vocal_detection.py - F0 extraction
preprocess/tools/f0_extraction.py - Lyric transcription
preprocess/tools/lyric_transcription.py - Note transcription
preprocess/tools/note_transcription/model.py - Aligned metadata
- Metadata MIDI conversion
preprocess/tools/midi_parser.py
Singing synthesis
- SVS inference
cli/inference.py - Synthesis data prep
soulxsinger/utils/data_processor.py - SoulX-Singer
soulxsinger/models/soulxsinger.py
Voice conversion
- SVC inference
cli/inference_svc.py - SoulX-Singer-SVC
soulxsinger/models/soulxsinger_svc.py
Model layers
- Content encoder
soulxsinger/models/modules/whisper_encoder.py - Flow decoder
soulxsinger/models/modules/flow_matching.py - Mel transform
soulxsinger/models/modules/mel_transform.py - Vocoder
soulxsinger/models/modules/vocoder.py
Other components
- User
- Audio input
- Generated audio
Connections 30
- User → SVS WebUI: requests synthesis
- User → SVC WebUI: requests conversion
- User → SVS inference: runs inference
- User → SVC inference: runs inference
- User → Preprocess pipeline: submits audio
- Audio input → Preprocess pipeline: audio input
- User → MIDI editor: edits alignment
- MIDI editor → MIDI state: updates notes
- MIDI editor → MIDI import/export: imports/exports
- Preprocess pipeline → Vocal separation: separates vocals
- Preprocess pipeline → F0 extraction: extracts pitch
- Preprocess pipeline → Vocal segmentation: segments vocals
- Preprocess pipeline → Lyric transcription: transcribes lyrics
- Preprocess pipeline → Note transcription: transcribes notes
- Preprocess pipeline → Aligned metadata: writes metadata
- Aligned metadata → Metadata MIDI conversion: converts format
- Metadata MIDI conversion → MIDI editor: loads MIDI
- Aligned metadata → SVS inference: conditions synthesis
- SVS inference → Synthesis data prep: prepares inputs
- SVS inference → SoulX-Singer: runs inference
- SoulX-Singer → Generated audio: generates singing
- Audio input → SVC inference: source recording
- SVC inference → SoulX-Singer-SVC: runs inference
- SoulX-Singer-SVC → Generated audio: converts voice
- SoulX-Singer-SVC → Content encoder: encodes content
- SoulX-Singer-SVC → Flow decoder: decodes features
- SoulX-Singer-SVC → Mel transform: processes spectrum
- SoulX-Singer-SVC → Vocoder: synthesizes waveform
- SVS WebUI → SVS inference: launches inference
- SVC WebUI → SVC inference: launches inference