How eyov7/aud2stm2mdi works
Main Components
• User Interface (Gradio-based Web UI) – Represented in app.py. This is the entry point for users where they upload the audio file, select options such as stem type, and trigger processing.
• Audio Stem Separation Module – Implemented in demucs_handler.py. This component manages the core logic for separating audio stems using the Demucs deep learning model, inclusing tensor processing via PyTorch.
• MIDI Conversion Module – Implemented in basic_pitch_handler.py. This handles transcription of the separated audio to MIDI files using Spotify's Basic Pitch with optimizations.
• Validation and Error Handling – Found in validators.py, ensuring audio file integrity, format, size, and sample rate are in order before processing.
• Deployment and Infrastructure – Include a component that represents the Lightning.ai deployment environment, highlighting that the application is packaged to run either locally or on the cloud.
Relationships and Interactions
• The User Interface (app.py) accepts audio inputs and passes file data to both the stem separation and MIDI conversion modules.
• The Stem Separation Module (demucs_handler.py) processes the audio input to extract stems and then either directly returns output or optionally connects to the MIDI conversion step.
• The MIDI Conversion Module (basic_pitch_handler.py) receives processed audio for note detection and MIDI file generation.
• The Validation Module (validators.py) is invoked early in the process to ensure the input file meets all criteria; errors propagate back to the UI if validation fails.
• External libraries (PyTorch, etc.) support both the stem separation and MIDI conversion processes.
Architectural Patterns and Design Principles
• Separation of concerns – Each Python file is dedicated to a single responsibility (UI, stem separation, MIDI conversion, validation).
• Modularity – The clear division into distinct modules allows for easy updates and potential scalability (for example, adding new models or conversion features).
• Robust Error Handling – As highlighted in the README, error handling is a crucial aspect, so show relevant communication between modules and error reporting back to the UI.
• Scalability and Deployment – Illustrate the deployment environment (Lightning.ai) indicating how compute resources (CPU/GPU) interface with the application.