# mtg/compiam architecture

> Architecture diagram of the GitHub repository mtg/compiam, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mtg/compiam
- Repository: https://github.com/mtg/compiam

## Overview

1. **What this repository is**
   compIAM is a Python library for computational analysis of Indian art music, with a strong emphasis on Carnatic and Hindustani MIR. It is organized as a package of reusable analysis modules rather than as a single application or service.

2. **Top-level architecture**
   The package is split by musical dimension: **melody**, **rhythm**, **structure**, **timbre**, and **separation**. Each dimension contains one or more task-specific pipelines, and the root package exposes convenience entry points for listing tools/tasks and loading datasets, corpora, and pretrained models.

3. **Core data boundary: audio in, musical representations out**
   Most subsystems accept audio files or derived time-series features and produce higher-level musical outputs such as pitch contours, tonic estimates, ragas, beat/meter annotations, segment boundaries, source-separated stems, or instrument stroke classes. This makes the architecture pipeline-oriented: raw audio is transformed through feature extraction, model inference, and postprocessing.

4. **Melody subsystem**
   This is the largest domain area. It includes:
   - **Pitch extraction**: FTANet Carnatic, FTAResNet Carnatic, and Melodia-based extraction
   - **Tonic identification**: tonic estimation from multipitch/pitch information
   - **Raga recognition**: DeepSRGM with attention and raga mapping tables
   - **Pattern search / sancara search**: a separate branch with CQT, complex auto-correlation, self-similarity, segmentation, and visualization utilities  
   The melody stack depends on domain configuration files such as svara cent mappings and raga definitions.

5. **Rhythm subsystem**
   Rhythm is split into:
   - **Meter / pulse tracking**: akshara pulse tracker and a TCN-based Carnatic meter model with pre- and post-processing stages
   - **Rhythm transcription**: mnemonic transcription resources, including bol mappings  
   This area is centered on feature extraction from audio and sequence modeling for rhythmic structure.

6. **Separation subsystem**
   Separation contains two distinct approaches:
   - **Music source separation** with a mixer-model implementation
   - **Singing voice extraction** with the Cold Diff Sep pipeline, including U-Net, clustering, noise estimation, VAD, and signal-processing helpers  
   Separation acts as a preprocessing boundary for downstream melody or transcription tasks when isolated vocal content is needed.

7. **Structure subsystem**
   Structure analysis is represented by **dhrupad/bandish segmentation**, with modules for audio processing, feature extraction, parameterization, and model utilities. This subsystem is more segment-boundary oriented than the melody/rhythm modules and likely consumes audio-derived features to produce structural labels or cut points.

8. **Timbre subsystem**
   Timbre focuses on **mridangam stroke classification**. It includes feature extraction from percussion audio and a model for classifying stroke types. This is the most instrument-specific subsystem in the repo and is separate from the broader melody/rhythm pipelines.

9. **Shared utilities and support code**
   Several cross-cutting components support the domain modules:
   - **compiam/utils/NMFtoolbox**: signal-processing/NMF primitives
   - **augment**: data augmentation helpers and template assets
   - **download.py**, **pitch.py**, **data.py**, **io.py**, **exceptions.py**: shared plumbing for I/O, data access, and error handling
   - **conf/**: static musical configuration data
   These are the reusable building blocks that multiple subsystems rely on.

10. **Data, metadata, and corpora integration**
    The repository includes local annotation examples, corpus integration via **Dunya**, and dataset-loader support via **mirdata**-style wrappers. The architecture therefore spans three data sources: bundled configs/resources, downloadable datasets, and external corpora services.

11. **Visualization and user-facing inspection**
    Visualization is a separate package with audio, pitch, training, and peak-analysis plots, plus a waveform player UI bundle. This is not part of the model pipeline itself, but it is important for inspecting outputs, annotations, and training behavior.

12. **Repository support layers**
    The `tests/` tree mirrors the package structure and validates the domain subsystems independently. `docs/` and notebooks provide usage examples and tutorials, while GitHub Actions handles CI and documentation builds. This indicates a library-first architecture with documentation and regression testing as first-class parts of the project.

13. **How to draw the main diagram**
    A good architecture diagram would show:
    - **Raw audio / dataset / corpus inputs**
    - Branches into **melody**, **rhythm**, **structure**, **timbre**, and **separation**
    - Shared **utility/config/data** layer underneath
    - **Visualization** as an output/inspection layer
    - **Tests/docs/notebooks** around the core package as supporting artifacts  
    The most important cross-module flow is from audio preprocessing and feature extraction into task-specific models, with some outputs feeding later analysis stages such as tonic estimation or raga recognition.

Main entry points: [`compiam/melody`](https://github.com/mtg/compiam/tree/master/compiam/melody) (Melody), [`compiam/__init__.py`](https://github.com/mtg/compiam/blob/master/compiam/__init__.py) (compiam), [`compiam/rhythm`](https://github.com/mtg/compiam/tree/master/compiam/rhythm) (Rhythm), [`compiam/separation`](https://github.com/mtg/compiam/tree/master/compiam/separation) (Separation)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_core["Core package"]
  node_pkg["compiam<br/>library root<br/>[__init__.py]"]
  node_data_api["Data access<br/>loaders<br/>[data.py]"]
end

subgraph group_melody["Melody"]
  node_melody_pkg["Melody<br/>domain package"]
  node_pitch_ext["Pitch extraction<br/>pitch pipelines"]
  node_tonic["Tonic ID<br/>tonic estimation"]
  node_raga["Raga recognition<br/>classification"]
  node_pattern["Pattern search<br/>sancara search"]
end

subgraph group_rhythm["Rhythm"]
  node_rhythm_pkg["Rhythm<br/>domain package"]
  node_meter["Meter tracking<br/>pulse models"]
  node_transcription["Transcription<br/>rhythm labels"]
end

subgraph group_separation["Separation"]
  node_sep_pkg["Separation<br/>domain package"]
  node_source_sep["Source sep<br/>mixer model"]
  node_voice_ext["Voice extract<br/>cold diff sep"]
end

subgraph group_structure["Structure"]
  node_struct_pkg["Structure<br/>domain package"]
  node_segmentation["Segmentation<br/>boundary detection"]
end

subgraph group_timbre["Timbre"]
  node_timbre_pkg["Timbre<br/>domain package"]
  node_stroke["Stroke class<br/>classifier"]
end

subgraph group_support["Support"]
  node_io_api["I/O layer<br/>plumbing<br/>[io.py]"]
  node_utils["Utilities<br/>shared utils"]
  node_conf["Configs<br/>musical config"]
  node_dunya["Dunya<br/>corpus bridge"]
  node_ann["Annotations"]
  node_viz["Visualization<br/>inspection tools"]
  node_docs["Docs<br/>documentation"]
  node_tests["Tests<br/>validation"]
end

node_pkg -->|"exposes"| node_data_api
node_pkg -->|"contains"| node_melody_pkg
node_pkg -->|"contains"| node_rhythm_pkg
node_pkg -->|"contains"| node_sep_pkg
node_pkg -->|"contains"| node_struct_pkg
node_pkg -->|"contains"| node_timbre_pkg
node_melody_pkg -->|"routes to"| node_pitch_ext
node_melody_pkg -->|"routes to"| node_tonic
node_melody_pkg -->|"routes to"| node_raga
node_melody_pkg -->|"routes to"| node_pattern
node_pitch_ext -->|"uses"| node_conf
node_raga -->|"uses"| node_conf
node_rhythm_pkg -->|"contains"| node_meter
node_rhythm_pkg -->|"contains"| node_transcription
node_meter -->|"uses"| node_utils
node_sep_pkg -->|"contains"| node_source_sep
node_sep_pkg -->|"contains"| node_voice_ext
node_voice_ext -->|"uses"| node_utils
node_struct_pkg -->|"contains"| node_segmentation
node_segmentation -->|"uses"| node_utils
node_timbre_pkg -->|"contains"| node_stroke
node_stroke -->|"uses"| node_utils
node_viz -.->|"inspects"| node_melody_pkg
node_viz -.->|"inspects"| node_rhythm_pkg
node_viz -.->|"inspects"| node_sep_pkg
node_data_api -->|"loads"| node_dunya
node_data_api -->|"loads"| node_ann
node_data_api -->|"uses"| node_utils
node_docs -.->|"documents"| node_melody_pkg
node_docs -.->|"documents"| node_rhythm_pkg
node_docs -.->|"documents"| node_sep_pkg
node_docs -.->|"documents"| node_struct_pkg
node_docs -.->|"documents"| node_timbre_pkg
node_tests -.->|"validates"| node_melody_pkg
node_tests -.->|"validates"| node_rhythm_pkg
node_tests -.->|"validates"| node_sep_pkg
node_tests -.->|"validates"| node_struct_pkg
node_tests -.->|"validates"| node_timbre_pkg

click node_pkg "https://github.com/mtg/compiam/blob/master/compiam/__init__.py"
click node_data_api "https://github.com/mtg/compiam/blob/master/compiam/data.py"
click node_io_api "https://github.com/mtg/compiam/blob/master/compiam/io.py"
click node_utils "https://github.com/mtg/compiam/tree/master/compiam/utils"
click node_conf "https://github.com/mtg/compiam/tree/master/compiam/conf"
click node_dunya "https://github.com/mtg/compiam/tree/master/compiam/dunya"
click node_ann "https://github.com/mtg/compiam/tree/master/compiam/annotations"
click node_melody_pkg "https://github.com/mtg/compiam/tree/master/compiam/melody"
click node_pitch_ext "https://github.com/mtg/compiam/tree/master/compiam/melody/pitch_extraction"
click node_tonic "https://github.com/mtg/compiam/tree/master/compiam/melody/tonic_identification"
click node_raga "https://github.com/mtg/compiam/tree/master/compiam/melody/raga_recognition"
click node_pattern "https://github.com/mtg/compiam/tree/master/compiam/melody/pattern"
click node_rhythm_pkg "https://github.com/mtg/compiam/tree/master/compiam/rhythm"
click node_meter "https://github.com/mtg/compiam/tree/master/compiam/rhythm/meter"
click node_transcription "https://github.com/mtg/compiam/tree/master/compiam/rhythm/transcription"
click node_sep_pkg "https://github.com/mtg/compiam/tree/master/compiam/separation"
click node_source_sep "https://github.com/mtg/compiam/tree/master/compiam/separation/music_source_separation"
click node_voice_ext "https://github.com/mtg/compiam/tree/master/compiam/separation/singing_voice_extraction"
click node_struct_pkg "https://github.com/mtg/compiam/tree/master/compiam/structure"
click node_segmentation "https://github.com/mtg/compiam/tree/master/compiam/structure/segmentation"
click node_timbre_pkg "https://github.com/mtg/compiam/tree/master/compiam/timbre"
click node_stroke "https://github.com/mtg/compiam/tree/master/compiam/timbre/stroke_classification"
click node_viz "https://github.com/mtg/compiam/tree/master/compiam/visualisation"
click node_docs "https://github.com/mtg/compiam/tree/master/docs"
click node_tests "https://github.com/mtg/compiam/tree/master/tests"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_pkg,node_data_api,node_io_api,node_utils,node_conf,node_dunya,node_ann,node_viz,node_docs,node_tests toneBlue
class node_melody_pkg,node_pitch_ext,node_tonic,node_raga,node_pattern toneAmber
class node_rhythm_pkg,node_meter,node_transcription toneMint
class node_sep_pkg,node_source_sep,node_voice_ext toneRose
class node_struct_pkg,node_segmentation toneIndigo
class node_timbre_pkg,node_stroke toneTeal
```

## Components

### Core package

Root package entry points and shared interfaces.

- [compiam](https://github.com/mtg/compiam/blob/master/compiam/__init__.py): library root · Top-level package exposing domain modules and convenience accessors. · `compiam/__init__.py`
- [Data access](https://github.com/mtg/compiam/blob/master/compiam/data.py): loaders · Shared dataset, corpus, and model-loading plumbing. · `compiam/data.py`

### Melody

Pitch, tonic, raga, and pattern analysis.

- [Melody](https://github.com/mtg/compiam/tree/master/compiam/melody): domain package · Pitch-centric analysis pipelines: extraction, tonic, raga, and pattern search. · `compiam/melody`
- [Pitch extraction](https://github.com/mtg/compiam/tree/master/compiam/melody/pitch_extraction): pitch pipelines · FTANet, FTAResNet, and Melodia-based pitch tracking. · `compiam/melody/pitch_extraction`
- [Tonic ID](https://github.com/mtg/compiam/tree/master/compiam/melody/tonic_identification): tonic estimation · Tonic estimation from multipitch or pitch contours. · `compiam/melody/tonic_identification`
- [Raga recognition](https://github.com/mtg/compiam/tree/master/compiam/melody/raga_recognition): classification · DeepSRGM-based raga recognition with attention and label mapping. · `compiam/melody/raga_recognition`
- [Pattern search](https://github.com/mtg/compiam/tree/master/compiam/melody/pattern): sancara search · CQT, autocorrelation, segmentation, self-similarity, and visualization utilities. · `compiam/melody/pattern`

### Rhythm

Meter tracking and rhythmic transcription.

- [Rhythm](https://github.com/mtg/compiam/tree/master/compiam/rhythm): domain package · Meter tracking and mnemonic rhythm transcription. · `compiam/rhythm`
- [Meter tracking](https://github.com/mtg/compiam/tree/master/compiam/rhythm/meter): pulse models · Akshara pulse tracking and TCN-based Carnatic meter modeling. · `compiam/rhythm/meter`
- [Transcription](https://github.com/mtg/compiam/tree/master/compiam/rhythm/transcription): rhythm labels · Mnemonic transcription resources and bol mappings. · `compiam/rhythm/transcription`

### Separation

Source separation and singing voice extraction.

- [Separation](https://github.com/mtg/compiam/tree/master/compiam/separation): domain package · Music source separation and singing voice extraction pipelines. · `compiam/separation`
- [Source sep](https://github.com/mtg/compiam/tree/master/compiam/separation/music_source_separation): mixer model · Music source separation using mixer-model components. · `compiam/separation/music_source_separation`
- [Voice extract](https://github.com/mtg/compiam/tree/master/compiam/separation/singing_voice_extraction): cold diff sep · Singing voice extraction with U-Net, clustering, noise estimation, and VAD. · `compiam/separation/singing_voice_extraction`

### Structure

Segmentation and boundary detection.

- [Structure](https://github.com/mtg/compiam/tree/master/compiam/structure): domain package · Segmentation pipeline for dhrupad/bandish structure analysis. · `compiam/structure`
- [Segmentation](https://github.com/mtg/compiam/tree/master/compiam/structure/segmentation): boundary detection · Audio processing, feature extraction, parameters, and model utilities for structural cuts. · `compiam/structure/segmentation`

### Timbre

Instrument-specific stroke classification.

- [Timbre](https://github.com/mtg/compiam/tree/master/compiam/timbre): domain package · Mridangam stroke classification pipeline. · `compiam/timbre`
- [Stroke class](https://github.com/mtg/compiam/tree/master/compiam/timbre/stroke_classification): classifier · Feature extraction and model for mridangam stroke classification. · `compiam/timbre/stroke_classification`

### Support

Shared utilities, configs, data, visualization, and docs.

- [I/O layer](https://github.com/mtg/compiam/blob/master/compiam/io.py): plumbing · Common I/O helpers used across analysis modules. · `compiam/io.py`
- [Utilities](https://github.com/mtg/compiam/tree/master/compiam/utils): shared utils · Reusable signal-processing, pitch, download, and augmentation helpers. · `compiam/utils`
- [Configs](https://github.com/mtg/compiam/tree/master/compiam/conf): musical config · Static raga and svara configuration tables. · `compiam/conf`
- [Dunya](https://github.com/mtg/compiam/tree/master/compiam/dunya): corpus bridge · Integration layer for external music corpus access. · `compiam/dunya`
- [Annotations](https://github.com/mtg/compiam/tree/master/compiam/annotations): sample annotations · Bundled example annotation files used by the library and docs. · `compiam/annotations`
- [Visualization](https://github.com/mtg/compiam/tree/master/compiam/visualisation): inspection tools · Plots and waveform-player UI for inspecting audio, pitch, peaks, and training. · `compiam/visualisation`
- [Docs](https://github.com/mtg/compiam/tree/master/docs): documentation · Tutorials and reference documentation for the package APIs. · `docs`
- [Tests](https://github.com/mtg/compiam/tree/master/tests): validation · Regression tests mirroring the package subsystems. · `tests`

## Connections

- compiam → Data access: exposes · Top-level package exposes data loading and model access helpers.
- compiam → Melody: contains · Root package includes the melody subsystem.
- compiam → Rhythm: contains · Root package includes the rhythm subsystem.
- compiam → Separation: contains · Root package includes the separation subsystem.
- compiam → Structure: contains · Root package includes the structure subsystem.
- compiam → Timbre: contains · Root package includes the timbre subsystem.
- Melody → Pitch extraction: routes to · Melody analysis begins with pitch extraction.
- Melody → Tonic ID: routes to · Pitch outputs feed tonic estimation.
- Melody → Raga recognition: routes to · Tonic and pitch features support raga recognition.
- Melody → Pattern search: routes to · Pattern search operates as a parallel melody branch.
- Pitch extraction → Configs: uses · Pitch pipelines depend on svara and raga configuration data.
- Raga recognition → Configs: uses · Raga mapping and definitions are configuration-driven.
- Rhythm → Meter tracking: contains · Rhythm includes meter and pulse tracking.
- Rhythm → Transcription: contains · Rhythm includes mnemonic transcription resources.
- Meter tracking → Utilities: uses · Meter models depend on shared signal-processing utilities.
- Separation → Source sep: contains · Separation includes music source separation.
- Separation → Voice extract: contains · Separation includes singing voice extraction.
- Voice extract → Utilities: uses · Voice extraction relies on shared audio helpers and signal processing.
- Structure → Segmentation: contains · Structure analysis centers on segmentation.
- Segmentation → Utilities: uses · Segmentation reuses shared audio and feature helpers.
- Timbre → Stroke class: contains · Timbre analysis centers on stroke classification.
- Stroke class → Utilities: uses · Stroke classification consumes shared feature utilities.
- Visualization → Melody: inspects · Visualization helps inspect melody outputs and pitch contours.
- Visualization → Rhythm: inspects · Visualization supports rhythm analysis review.
- Visualization → Separation: inspects · Visualization supports separation output inspection.
- Data access → Dunya: loads · Data access bridges to external corpus resources.
- Data access → Annotations: loads · Data access can use bundled annotations.
- Data access → Utilities: uses · Data and model loading use shared helpers.
- Docs → Melody: documents · Documentation includes melody usage and workflows.
- Docs → Rhythm: documents · Documentation includes rhythm usage and workflows.
- Docs → Separation: documents · Documentation includes separation usage and workflows.
- Docs → Structure: documents · Documentation includes structure analysis.
- Docs → Timbre: documents · Documentation includes timbre analysis.
- Tests → Melody: validates · Tests cover melody extractors and recognition components.
- Tests → Rhythm: validates · Tests cover rhythm models and pulse tracking.
- Tests → Separation: validates · Tests cover separation pipelines.
- Tests → Structure: validates · Tests cover structure segmentation.
- Tests → Timbre: validates · Tests cover stroke classification.