# rickyl-2000/alignsts architecture

> Architecture diagram of the GitHub repository rickyl-2000/alignsts, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rickyl-2000/alignsts
- Repository: https://github.com/rickyl-2000/alignsts

## Overview

##### Project Type and Purpose
- AlignSTS is a research-grade speech-to-singing conversion system implemented in PyTorch.
- It provides both training and inference pipelines for an acoustic model (“AlignSTS”) and a neural vocoder (HiFi-GAN or Parallel WaveGAN).

##### File Structure
- **configs/**: YAML configuration files for different tasks (singing, speech2singing, TTS).
- **data_gen/**: Data preprocessing and binarization scripts for singing, speech2singing, and TTS.
- **modules/**: Core neural network components grouped by model type (diffusion, fastspeech, hifigan, parallel_wavegan, speech2singing).
- **vocoders/**: Wrapper classes for neural vocoders (HiFi-GAN, PWG).
- **tasks/**: Orchestrator scripts (`tasks/run.py`) and task definitions for training/inference of speech2singing and TTS.
- **utils/**: Common utilities (audio I/O, checkpointing, distributed training, plotting, text processing, etc.).
- **usr/**: User entry points for evaluation and custom scripts.

##### README for Insights
- Inference pipeline steps:
1. Prepare checkpoints (`checkpoints/alignsts`, `checkpoints/hifigan`).
2. Prepare dataset (binary stats + WAV files).
3. Run `tasks/run.py` with the proper config.
4. Model outputs mel-spectrograms → Vocoder generates waveforms.
- External dependencies: PyTorch, specific repos (NATSpeech, DiffSinger, ProDiff, SpeechSplit2) used as code references.

##### Main Components and Their Interactions
A. Configuration Manager
- Loads YAMLs from `configs/` → Provides hyperparameters to tasks and modules.
B. Data Preprocessing
- Scripts in `data_gen/` → Read raw audio + text → Extract features (e.g., mel, pitch) → Produce binary datasets.
C. Task Orchestrator
- Entry point: `tasks/run.py` → Reads config → Instantiates Trainer or Inference engine.
- Sub-tasks: `tasks/speech2singing/*`, `tasks/tts/*`.
D. Model Modules
- Acoustic model (AlignSTS): code in `modules/speech2singing/*`.
- Auxiliary models: diffusion (`modules/diffusion`), fastspeech (`modules/fastspeech`), transformer TTS.
- Neural vocoder: `modules/hifigan`, `modules/parallel_wavegan` & wrapper in `vocoders/`.
E. Utilities
- Logging, checkpointing (`utils/ckpt_utils.py`), distributed training (`utils/ddp_utils.py`), schedulers, plotting, text normalization, etc.
F. External Services (optional)
- ASR or alignment tools referenced in configs (MFA, asr.yaml).

Data Flow (Inference Example):
Raw Speech WAV  →  Data Preprocessing (feature extraction)
↓
Binary Dataset → Task Orchestrator (infer mode)
↓
AlignSTS Acoustic Model → Predicted Mel-Spectrogram
↓
Neural Vocoder (HiFi-GAN / PWG) → Output Singing WAV

##### Architectural Patterns & Design Principles
- **Modular Design**: Separate modules for each model type, enabling easy swapping (e.g., HiFi-GAN vs. PWG).
- **Config-Driven Pipeline**: YAML files centrally define hyperparameters, model choices, and dataset paths.
- **Task Abstraction**: `base_task.py` defines common training/inference behaviors; specialized in sub-packages.
- **Utility Layer**: A thin wrapper for I/O, logging, distributed training keeps core logic clean.

##### Final Checklist
- All major directories/components represented.
- Clear data flow from raw input → preprocessing → model → output.
- Configuration management shown centrally.
- External dependencies or services noted.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Configuration Manager
    subgraph "Configuration Manager"
        ConfigMgr["Configuration Manager"]:::config
    end
    click ConfigMgr "https://github.com/rickyl-2000/alignsts/tree/main/configs/"

    %% Data Layer
    subgraph "Data Layer"
        RawWAV[(Raw Speech WAV)]:::data
        DataPre["Data Preprocessing"]:::data
        BinaryDataset[(Binary Dataset)]:::data
    end
    click DataPre "https://github.com/rickyl-2000/alignsts/tree/main/data_gen/"

    %% Model Layer
    subgraph "Model Layer"
        subgraph "Acoustic Model"
            AcousticModel["AlignSTS Acoustic Model"]:::model
        end
        subgraph "Auxiliary Models"
            Diffusion["Diffusion Models"]:::model
            FastSpeech["FastSpeech/TTS Models"]:::model
        end
        subgraph "Neural Vocoder"
            VocoderWrapper["Vocoder Wrapper"]:::model
            HifiGANImpl["HiFi-GAN Impl"]:::model
            PWGImpl["Parallel WaveGAN Impl"]:::model
        end
    end
    click AcousticModel "https://github.com/rickyl-2000/alignsts/tree/main/modules/speech2singing/"
    click Diffusion "https://github.com/rickyl-2000/alignsts/tree/main/modules/diffusion/"
    click FastSpeech "https://github.com/rickyl-2000/alignsts/tree/main/modules/fastspeech/"
    click HifiGANImpl "https://github.com/rickyl-2000/alignsts/blob/main/modules/hifigan/hifigan.py"
    click PWGImpl "https://github.com/rickyl-2000/alignsts/tree/main/modules/parallel_wavegan/"
    click VocoderWrapper "https://github.com/rickyl-2000/alignsts/tree/main/vocoders/"

    %% Orchestration Layer
    subgraph "Orchestration Layer"
        subgraph "Task Orchestrator"
            TaskRun["tasks/run.py"]:::task
            BaseTask["tasks/base_task.py"]:::task
            Speech2STask["Speech2Singing Tasks"]:::task
            TTSTask["TTS Tasks"]:::task
        end
        ConfigMgr
        Utils["Utilities"]:::util
    end
    click TaskRun "https://github.com/rickyl-2000/alignsts/blob/main/tasks/run.py"
    click BaseTask "https://github.com/rickyl-2000/alignsts/blob/main/tasks/base_task.py"
    click Speech2STask "https://github.com/rickyl-2000/alignsts/tree/main/tasks/speech2singing/"
    click TTSTask "https://github.com/rickyl-2000/alignsts/tree/main/tasks/tts/"
    click Utils "https://github.com/rickyl-2000/alignsts/tree/main/utils/"

    %% User Scripts
    subgraph "User Entry Points"
        Eval["usr/eval.py"]:::util
        CommonUsr["usr_utils"]:::util
    end
    click Eval "https://github.com/rickyl-2000/alignsts/blob/main/usr/eval.py"
    click CommonUsr "https://github.com/rickyl-2000/alignsts/blob/main/usr/usr_utils/common.py"

    %% External Services
    ASR["ASR/MFA Tools"]:::external

    %% Data Flow
    RawWAV -->|"raw audio"| DataPre
    DataPre -->|"binary data"| BinaryDataset
    BinaryDataset -->|"dataset load"| TaskRun
    TaskRun -->|"infer model"| AcousticModel
    AcousticModel -->|"mel spectrogram"| PredMel[(Predicted Mel-Spectrogram)]:::model
    PredMel -->|"features"| VocoderWrapper
    VocoderWrapper -->|"generate waveform"| OutputWAV[(Output Singing WAV)]:::data
    VocoderWrapper --> HifiGANImpl
    VocoderWrapper --> PWGImpl

    %% Config & Utils connections
    ConfigMgr -->|"config"| DataPre
    ConfigMgr -->|"config"| TaskRun
    ConfigMgr -->|"config"| AcousticModel
    ConfigMgr -->|"config"| VocoderWrapper
    Utils -->|"utils"| DataPre
    Utils -->|"utils"| TaskRun
    Utils -->|"utils"| AcousticModel
    Utils -->|"utils"| VocoderWrapper

    %% External dependency
    ASR -.->|"optional tools"| DataPre

    classDef config fill:#E0C3FC,stroke:#A67DFF,color:#333;
    classDef data fill:#D0E8FF,stroke:#76C7FF,color:#333;
    classDef model fill:#DFFFD0,stroke:#88FF76,color:#333;
    classDef task fill:#FFE8C4,stroke:#FFC16D,color:#333;
    classDef util fill:#F0F0F0,stroke:#CCCCCC,color:#333;
    classDef external fill:#F5F5F5,stroke:#BBBBBB,color:#333;
```
