How eupston/deepbeat-beatbox2midi works
Project type and purpose
• Deepbeat is a desktop, cross-platform, full-stack (single-process) application written in Python that provides a GUI for converting raw beatboxing audio into MIDI drum sequences. It embeds a trained Convolutional Neural Network (CNN) model, audio I/O modules, MIDI generation and playback, and packaging scripts.
File structure
Top-level components:
• Deepbeat.py – application entry point that initializes the PyQt UI and ties together modules
• Mainwindow.ui / Mainwindow.py – PyQt5 GUI layout and controller
• utils/ – domain logic: audio preprocessing, onset detection, model inference, silence removal, MIDI output, metronome, playback
• Build_setup/ – OS-specific packaging configurations and scripts (pyinstaller spec for Windows, py2app setup for macOS)
• requirements.txt – Python dependencies (PyQt5, TensorFlow/Keras, numpy, mido, etc.)
README for additional insights
• Describes the audio recorder + trimmer, CNN-based conversion “Kick/Snare/Hihat,” four drum-kit playback options, looper, tempo slider, and metronome.
• Mentions external dependency: Panotti CNN training code.
• Packaging via pyinstaller/py2app.
Main components and their interactions
a. Components:
1. GUI Layer (PyQt5)
• Audio Recorder Widget
• Trimmer Widget
• Controls: “Convert to MIDI,” tempo slider, metronome on/off, kit selector, looper, export button
2. Audio Preprocessing
• remove_silence.py
• onset_offset.py
• audio2midi.py (coordinates slicing and feeding windows to model)
3. Classification Engine
• predict.py (loads resources/beatbox_model2.hdf5)
• runs CNN inference per audio frame or window
4. MIDI Generator & Exporter
• midi_player.py (real-time playback using chosen drum-kit samples)
• audio2midi.py (maps model outputs to MIDI note events)
5. Metronome
• metronome.py (click track generation based on tempo slider)
6. Resources
• resources/drumkits (WAV samples for Kick, Snare, Hi-Hat)
7. Packaging Layer
• Build_setup (pyinstaller/py2app configs)
b. Relationships:
• User clicks “Record” → GUI triggers audio recording (PyAudio or QtMultimedia)
• Recorded raw WAV → pass to remove_silence → onset detection → segmented
• Each segment piped to predict → CNN outputs class labels + timestamps
• Labels → audio2midi converts to a MIDI track in memory
• track → either sent to midi_player for immediate playback (samples from resources/drumkits) or written to .mid file for export
• Tempo slider and looper modules influence playback timing loops and metronome.py
c. Architectural patterns:
• MVC-like separation: PyQt UI (View+Controller) vs. utils modules (Model/Service)
• Pipeline pattern in audio processing: chained transforms
• Resource externalization for model and sample assets
d. Key technologies:
• PyQt5 – GUI
• TensorFlow/Keras – CNN inference
• mido or python-rtmidi – MIDI generation/playback
• pyinstaller / py2app – packaging
• WAV I/O via built-in wave or sounddevice/PyAudio