How mtg/compiam works
1. What this repository is
compIAM is a Python library for computational analysis of Indian art music, with a strong emphasis on Carnatic and Hindustani MIR. It is organized as a package of reusable analysis modules rather than as a single application or service.
2. Top-level architecture
The package is split by musical dimension: melody, rhythm, structure, timbre, and separation. Each dimension contains one or more task-specific pipelines, and the root package exposes convenience entry points for listing tools/tasks and loading datasets, corpora, and pretrained models.
3. Core data boundary: audio in, musical representations out
Most subsystems accept audio files or derived time-series features and produce higher-level musical outputs such as pitch contours, tonic estimates, ragas, beat/meter annotations, segment boundaries, source-separated stems, or instrument stroke classes. This makes the architecture pipeline-oriented: raw audio is transformed through feature extraction, model inference, and postprocessing.
4. Melody subsystem
This is the largest domain area. It includes:
- Pitch extraction: FTANet Carnatic, FTAResNet Carnatic, and Melodia-based extraction
- Tonic identification: tonic estimation from multipitch/pitch information
- Raga recognition: DeepSRGM with attention and raga mapping tables
- Pattern search / sancara search: a separate branch with CQT, complex auto-correlation, self-similarity, segmentation, and visualization utilities
The melody stack depends on domain configuration files such as svara cent mappings and raga definitions.
5. Rhythm subsystem
Rhythm is split into:
- Meter / pulse tracking: akshara pulse tracker and a TCN-based Carnatic meter model with pre- and post-processing stages
- Rhythm transcription: mnemonic transcription resources, including bol mappings
This area is centered on feature extraction from audio and sequence modeling for rhythmic structure.
6. Separation subsystem
Separation contains two distinct approaches:
- Music source separation with a mixer-model implementation
- Singing voice extraction with the Cold Diff Sep pipeline, including U-Net, clustering, noise estimation, VAD, and signal-processing helpers
Separation acts as a preprocessing boundary for downstream melody or transcription tasks when isolated vocal content is needed.
7. Structure subsystem
Structure analysis is represented by dhrupad/bandish segmentation, with modules for audio processing, feature extraction, parameterization, and model utilities. This subsystem is more segment-boundary oriented than the melody/rhythm modules and likely consumes audio-derived features to produce structural labels or cut points.
8. Timbre subsystem
Timbre focuses on mridangam stroke classification. It includes feature extraction from percussion audio and a model for classifying stroke types. This is the most instrument-specific subsystem in the repo and is separate from the broader melody/rhythm pipelines.
9. Shared utilities and support code
Several cross-cutting components support the domain modules:
- compiam/utils/NMFtoolbox: signal-processing/NMF primitives
- augment: data augmentation helpers and template assets
- download.py, pitch.py, data.py, io.py, exceptions.py: shared plumbing for I/O, data access, and error handling
- conf/: static musical configuration data
These are the reusable building blocks that multiple subsystems rely on.
10. Data, metadata, and corpora integration
The repository includes local annotation examples, corpus integration via Dunya, and dataset-loader support via mirdata-style wrappers. The architecture therefore spans three data sources: bundled configs/resources, downloadable datasets, and external corpora services.
11. Visualization and user-facing inspection
Visualization is a separate package with audio, pitch, training, and peak-analysis plots, plus a waveform player UI bundle. This is not part of the model pipeline itself, but it is important for inspecting outputs, annotations, and training behavior.
12. Repository support layers
The tests/ tree mirrors the package structure and validates the domain subsystems independently. docs/ and notebooks provide usage examples and tutorials, while GitHub Actions handles CI and documentation builds. This indicates a library-first architecture with documentation and regression testing as first-class parts of the project.
13. How to draw the main diagram
A good architecture diagram would show:
- Raw audio / dataset / corpus inputs
- Branches into melody, rhythm, structure, timbre, and separation
- Shared utility/config/data layer underneath
- Visualization as an output/inspection layer
- Tests/docs/notebooks around the core package as supporting artifacts
The most important cross-module flow is from audio preprocessing and feature extraction into task-specific models, with some outputs feeding later analysis stages such as tonic estimation or raga recognition.
Main entry points: melody Melody · __init__.py compiam · rhythm Rhythm · separation Separation
Components 25
Core package
Root package entry points and shared interfaces.
- compiam · Top-level package exposing domain modules and convenience accessors.
compiam/__init__.py - Data access · Shared dataset, corpus, and model-loading plumbing.
compiam/data.py
Melody
Pitch, tonic, raga, and pattern analysis.
- Melody · Pitch-centric analysis pipelines: extraction, tonic, raga, and pattern search.
compiam/melody - Pitch extraction · FTANet, FTAResNet, and Melodia-based pitch tracking.
compiam/melody/pitch_extraction - Tonic ID · Tonic estimation from multipitch or pitch contours.
compiam/melody/tonic_identification - Raga recognition · DeepSRGM-based raga recognition with attention and label mapping.
compiam/melody/raga_recognition - Pattern search · CQT, autocorrelation, segmentation, self-similarity, and visualization utilities.
compiam/melody/pattern
Rhythm
Meter tracking and rhythmic transcription.
- Rhythm · Meter tracking and mnemonic rhythm transcription.
compiam/rhythm - Meter tracking · Akshara pulse tracking and TCN-based Carnatic meter modeling.
compiam/rhythm/meter - Transcription · Mnemonic transcription resources and bol mappings.
compiam/rhythm/transcription
Separation
Source separation and singing voice extraction.
- Separation · Music source separation and singing voice extraction pipelines.
compiam/separation - Source sep · Music source separation using mixer-model components.
compiam/separation/music_source_separation - Voice extract · Singing voice extraction with U-Net, clustering, noise estimation, and VAD.
compiam/separation/singing_voice_extraction
Structure
Segmentation and boundary detection.
- Structure · Segmentation pipeline for dhrupad/bandish structure analysis.
compiam/structure - Segmentation · Audio processing, feature extraction, parameters, and model utilities for structural cuts.
compiam/structure/segmentation
Timbre
Instrument-specific stroke classification.
- Timbre · Mridangam stroke classification pipeline.
compiam/timbre - Stroke class · Feature extraction and model for mridangam stroke classification.
compiam/timbre/stroke_classification
Support
Shared utilities, configs, data, visualization, and docs.
- I/O layer · Common I/O helpers used across analysis modules.
compiam/io.py - Utilities · Reusable signal-processing, pitch, download, and augmentation helpers.
compiam/utils - Configs · Static raga and svara configuration tables.
compiam/conf - Dunya · Integration layer for external music corpus access.
compiam/dunya - Annotations · Bundled example annotation files used by the library and docs.
compiam/annotations - Visualization · Plots and waveform-player UI for inspecting audio, pitch, peaks, and training.
compiam/visualisation - Docs · Tutorials and reference documentation for the package APIs.
docs - Tests · Regression tests mirroring the package subsystems.
tests
Connections 38
- compiam → Data access: exposes · Top-level package exposes data loading and model access helpers.
- compiam → Melody: contains · Root package includes the melody subsystem.
- compiam → Rhythm: contains · Root package includes the rhythm subsystem.
- compiam → Separation: contains · Root package includes the separation subsystem.
- compiam → Structure: contains · Root package includes the structure subsystem.
- compiam → Timbre: contains · Root package includes the timbre subsystem.
- Melody → Pitch extraction: routes to · Melody analysis begins with pitch extraction.
- Melody → Tonic ID: routes to · Pitch outputs feed tonic estimation.
- Melody → Raga recognition: routes to · Tonic and pitch features support raga recognition.
- Melody → Pattern search: routes to · Pattern search operates as a parallel melody branch.
- Pitch extraction → Configs: uses · Pitch pipelines depend on svara and raga configuration data.
- Raga recognition → Configs: uses · Raga mapping and definitions are configuration-driven.
- Rhythm → Meter tracking: contains · Rhythm includes meter and pulse tracking.
- Rhythm → Transcription: contains · Rhythm includes mnemonic transcription resources.
- Meter tracking → Utilities: uses · Meter models depend on shared signal-processing utilities.
- Separation → Source sep: contains · Separation includes music source separation.
- Separation → Voice extract: contains · Separation includes singing voice extraction.
- Voice extract → Utilities: uses · Voice extraction relies on shared audio helpers and signal processing.
- Structure → Segmentation: contains · Structure analysis centers on segmentation.
- Segmentation → Utilities: uses · Segmentation reuses shared audio and feature helpers.
- Timbre → Stroke class: contains · Timbre analysis centers on stroke classification.
- Stroke class → Utilities: uses · Stroke classification consumes shared feature utilities.
- Visualization → Melody: inspects · Visualization helps inspect melody outputs and pitch contours.
- Visualization → Rhythm: inspects · Visualization supports rhythm analysis review.
- Visualization → Separation: inspects · Visualization supports separation output inspection.
- Data access → Dunya: loads · Data access bridges to external corpus resources.
- Data access → Annotations: loads · Data access can use bundled annotations.
- Data access → Utilities: uses · Data and model loading use shared helpers.
- Docs → Melody: documents · Documentation includes melody usage and workflows.
- Docs → Rhythm: documents · Documentation includes rhythm usage and workflows.
- Docs → Separation: documents · Documentation includes separation usage and workflows.
- Docs → Structure: documents · Documentation includes structure analysis.
- Docs → Timbre: documents · Documentation includes timbre analysis.
- Tests → Melody: validates · Tests cover melody extractors and recognition components.
- Tests → Rhythm: validates · Tests cover rhythm models and pulse tracking.
- Tests → Separation: validates · Tests cover separation pipelines.
- Tests → Structure: validates · Tests cover structure segmentation.
- Tests → Timbre: validates · Tests cover stroke classification.