# rakuri255/ultrasinger architecture

> Architecture diagram of the GitHub repository rakuri255/ultrasinger, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rakuri255/ultrasinger
- Repository: https://github.com/rakuri255/ultrasinger

## Overview

##### Main Components Overview

a. Input Processing Layer
- Audio Input Handler (local files, YouTube)
- UltraStar File Input Handler (for re-pitching)

b. Core Processing Pipeline
- Audio Separation Module
- Speech Recognition Module
- Pitch Detection Module
- Hyphenation Module

c. Output Generation Layer
- UltraStar TXT Generator
- MIDI File Generator
- Sheet Music Generator
- Karaoke File Generator

##### Detailed Component Breakdown

**[Input Sources] → [Pre-processing] → [Core Processing] → [Output Generation]**
- ↓                   ↓                   ↓                    ↓
- Local Audio     - Audio Conversion   - Voice Separation   - UltraStar TXT
- YouTube Links   - Format Validation  - Whisper ASR       - MIDI Files
- UltraStar TXT  - Cache Management   - CREPE Pitcher     - Sheet Music
- Hyphenation        - Karaoke Files

##### Technology Stack Representation
Key technologies:
- FFmpeg for audio processing
- Whisper AI for speech recognition
- CREPE for pitch detection
- MuseScore for sheet music generation
- PyTorch/TensorFlow for AI models

##### Data Flow Visualization

Following flows:
**Input → Audio Processing → Voice Separation → Parallel Processing:**
- → Speech Recognition
- → Pitch Detection
- ↓
- Output Generation

##### Integration Points

External service connections:
- YouTube-DL integration
- MusicBrainz client
- MuseScore integration

##### Component Relationships

- Data transformation flows
- Optional processing paths

##### Configuration Elements

Configuration components:
- Model selection options
- Format version handling
- Device detection system
- Performance settings

##### Testing Framework

Test coverage:
- Unit test modules
- Integration test paths
- Performance test points

This design captures the complex audio processing pipeline while showing the modularity and extensibility of the system, making it easy to understand how new features or improvements can be integrated.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Styles and classes
    classDef input fill:#a8d1f0,stroke:#000
    classDef process fill:#90EE90,stroke:#000
    classDef ai fill:#DDA0DD,stroke:#000
    classDef external fill:#FFA500,stroke:#000
    classDef output fill:#87CEEB,stroke:#000
    classDef hardware fill:#FF6B6B,stroke:#000
    classDef config fill:#E6E6FA,stroke:#000

    %% Main Application
    MainApp["UltraSinger App"]:::config
    Settings["Settings"]:::config

    %% Input Layer
    subgraph InputLayer["Input Processing Layer"]
        AudioInput["Audio Input Handler"]:::input
        YoutubeHandler["YouTube Handler"]:::input
        AudioConversion["Audio Conversion"]:::process
        UltrastarInput["UltraStar Input"]:::input
    end

    %% Core Processing
    subgraph CoreProcessing["Core Processing Pipeline"]
        ProcessData["Process Data Controller"]:::process
        
        subgraph AudioProc["Audio Processing"]
            Separation["Audio Separation"]:::process
            VocalChunks["Vocal Chunks"]:::process
        end
        
        subgraph Recognition["Recognition & Analysis"]
            WhisperASR["Whisper ASR"]:::ai
            SpeechRecognition["Speech Recognition"]:::ai
            PitchDetection["CREPE Pitcher"]:::ai
            Hyphenation["Hyphenation"]:::process
        end
    end

    %% Hardware & External Services
    subgraph Infrastructure["Infrastructure"]
        DeviceDetection["Device Detection"]:::hardware
        MusicBrainz["MusicBrainz Client"]:::external
    end

    %% Output Generation
    subgraph OutputLayer["Output Generation Layer"]
        UltrastarWriter["UltraStar Writer"]:::output
        UltrastarTXT["UltraStar TXT"]:::output
        MidiCreator["MIDI Creator"]:::output
        SheetMusic["Sheet Music Generator"]:::output
    end

    %% Relationships
    MainApp --> Settings
    MainApp --> InputLayer
    AudioInput --> AudioConversion
    YoutubeHandler --> AudioConversion
    AudioConversion --> ProcessData
    UltrastarInput --> ProcessData
    
    ProcessData --> Separation
    Separation --> VocalChunks
    VocalChunks --> WhisperASR
    VocalChunks --> PitchDetection
    
    WhisperASR --> SpeechRecognition
    SpeechRecognition --> Hyphenation
    
    ProcessData --> OutputLayer
    
    DeviceDetection --> ProcessData
    MusicBrainz --> ProcessData

    %% Click events
    click MainApp "https://github.com/rakuri255/UltraSinger/blob/main/src/UltraSinger.py"
    click Settings "https://github.com/rakuri255/UltraSinger/blob/main/src/Settings.py"
    click YoutubeHandler "https://github.com/rakuri255/UltraSinger/blob/main/src/modules/Audio/youtube.py"
    click AudioConversion "https://github.com/rakuri255/UltraSinger/blob/main/src/modules/Audio/convert_audio.py"
    click Separation "https://github.com/rakuri255/UltraSinger/blob/main/src/modules/Audio/separation.py"
    click VocalChunks "https://github.com/rakuri255/UltraSinger/blob/main/src/modules/Audio/vocal_chunks.py"
    click WhisperASR "https://github.com/rakuri255/UltraSinger/blob/main/src/modules/Speech_Recognition/Whisper.py"
    click SpeechRecognition "https://github.com/rakuri255/UltraSinger/blob/main/src/modules/Speech_Recognition/speech_recognition.py"
    click PitchDetection "https://github.com/rakuri255/UltraSinger/blob/main/src/modules/Pitcher/pitcher.py"
    click Hyphenation "https://github.com/rakuri255/UltraSinger/blob/main/src/modules/Speech_Recognition/hyphenation.py"
    click UltrastarWriter "https://github.com/rakuri255/UltraSinger/blob/main/src/modules/Ultrastar/ultrastar_writer.py"
    click UltrastarTXT "https://github.com/rakuri255/UltraSinger/blob/main/src/modules/Ultrastar/ultrastar_txt.py"
    click MidiCreator "https://github.com/rakuri255/UltraSinger/blob/main/src/modules/Midi/midi_creator.py"
    click SheetMusic "https://github.com/rakuri255/UltraSinger/blob/main/src/modules/sheet.py"
    click DeviceDetection "https://github.com/rakuri255/UltraSinger/blob/main/src/modules/DeviceDetection/device_detection.py"
    click MusicBrainz "https://github.com/rakuri255/UltraSinger/blob/main/src/modules/musicbrainz_client.py"
    click ProcessData "https://github.com/rakuri255/UltraSinger/blob/main/src/modules/ProcessData.py"

    %% Legend
    subgraph Legend
        L1["Input Components"]:::input
        L2["Processing Components"]:::process
        L3["AI/ML Components"]:::ai
        L4["External Services"]:::external
        L5["Output Components"]:::output
        L6["Hardware Components"]:::hardware
        L7["Configuration"]:::config
    end
```
