How rakuri255/ultrasinger works
Main Components Overview
a. Input Processing Layer
• Audio Input Handler (local files, YouTube)
• UltraStar File Input Handler (for re-pitching)
b. Core Processing Pipeline
• Audio Separation Module
• Speech Recognition Module
• Pitch Detection Module
• Hyphenation Module
c. Output Generation Layer
• UltraStar TXT Generator
• MIDI File Generator
• Sheet Music Generator
• Karaoke File Generator
Detailed Component Breakdown
[Input Sources] → [Pre-processing] → [Core Processing] → [Output Generation]
• ↓ ↓ ↓ ↓
• Local Audio - Audio Conversion - Voice Separation - UltraStar TXT
• YouTube Links - Format Validation - Whisper ASR - MIDI Files
• UltraStar TXT - Cache Management - CREPE Pitcher - Sheet Music
• Hyphenation - Karaoke Files
Technology Stack Representation
Key technologies:
• FFmpeg for audio processing
• Whisper AI for speech recognition
• CREPE for pitch detection
• MuseScore for sheet music generation
• PyTorch/TensorFlow for AI models
Data Flow Visualization
Following flows:
Input → Audio Processing → Voice Separation → Parallel Processing:
• → Speech Recognition
• → Pitch Detection
• ↓
• Output Generation
Integration Points
External service connections:
• YouTube-DL integration
• MusicBrainz client
• MuseScore integration
Component Relationships
• Data transformation flows
• Optional processing paths
Configuration Elements
Configuration components:
• Model selection options
• Format version handling
• Device detection system
• Performance settings
Testing Framework
Test coverage:
• Unit test modules
• Integration test paths
• Performance test points
This design captures the complex audio processing pipeline while showing the modularity and extensibility of the system, making it easy to understand how new features or improvements can be integrated.