How shivamshinde123/instrument-based-music-recommendation works
Project type and purpose
• DeepMelody is a full-stack, data-driven ML application with two halves:
a. An offline pipeline that (a) ingest multitrack audio (Slakh2100), (b) preprocesses data, (c) trains a U-Net source-separation model, (d) extracts per-instrument embeddings, and (e) computes a similarity matrix.
b. An online Streamlit web app that serves personalized music recommendations based on those embeddings.
File structure
• Top level:
• Experimentation/: Jupyter notebooks for exploratory analysis.
• Metrics/: stored JSON metrics (validation_metrics.json).
• src/: production code, organized as sequential “step” scripts plus an app.
– step0_utility_functions.py: shared helpers
– step1_creating_csv.py: metadata consolidation
– step2_DatasetLoading.py: data preprocessing
– step3_calculate_similarity_scores.py: embedding & similarity matrix
– step4_ModelTraining.py: U-Net training
– prediction_funcs.py: inference utilities
– app.py: Streamlit front end
• Config & dependencies: params.yaml, requirements.txt.
• Documentation: README.md, project PDF, LICENSE.
README for additional insights
• Confirms a U-Net model for instrument isolation, creation of instrument embeddings, cosine-similarity–based recommendations, and a Streamlit UI.
a. Main components
1. Data Source
• Slakh2100 dataset (audio + metadata) on disk.
2. Offline Pipeline
• Data Consolidation (step1)
• Preprocessing & Loader (step2)
• Model Training (step4)
• Embedding Extraction & Similarity Computation (step3)
• Utility Functions (step0 + prediction_funcs)
3. Storage
• CSV metadata, processed data caches, trained model weights, embedding matrix, validation metrics.
4. Online Service (Streamlit App)
• UI, user input (instrument preferences), query to similarity engine, displays recommendations.
5. External Dependencies
• PyTorch, librosa, numpy, scikit-learn, pandas, Streamlit
b. Relationships & Interactions
• Data flows from Slakh2100 → step1 → step2 → step4 → step3 → storage.
• The trained U-Net model (step4 output) is loaded by prediction_funcs at inference time.
• Embedding matrix from step3 is loaded by the Streamlit app to compute top-N similar tracks per user query.
• Validation metrics saved in Metrics/ can be consumed by a monitoring or dashboard component.
c. Architectural patterns & design principles
• Pipeline pattern: clearly separated sequential ETL + training + embedding computation steps.
• Layered architecture: Data layer (raw & intermediate), Model layer (U-Net weights), Service layer (recommendation logic), Presentation layer (Streamlit UI).
• Single-process microservice for online recommendations (no distributed services yet).
d. Key technologies & frameworks
• Python 3.x orchestrates all steps.
• PyTorch for neural-network training.
• librosa for audio feature extraction.
• pandas/numpy for metadata and array manipulation.
• scikit-learn for similarity (cosine) calculations.
• Streamlit for web UI.
• YAML for parameter/config management.
Diagramming recommendations
• Use clear, concise labels for each component (e.g., “Step1: CSV Creation”, “Step4: U-Net Model Trainer”).
• Blue for data stores
• Green for processing/training steps
• Orange for the online service/UI
• Rectangles for computation modules
• Rounded rectangles for user interfaces
Detail—don’t omit any critical element
• Params.yaml and requirements.txt near the root as they drive reproducibility.
• Metrics/ as a feedback loop to the Experimentation component for continuous improvement.