How dap-lab/4way-tabla-transcription works
1. Repository purpose
- This repo is an offline ML pipeline for tabla stroke transcription: classifying strokes in audio into Damped, Resonant treble, Resonant bass, Resonant both.
- It is organized around two paper-specific codebases: ismir_related (ISMIR 2021) and tismir_related (TISMIR 2023).
2. Top-level split: training stack vs. inference/package-only stack
- ismir_related contains the full experimental workflow: data augmentation, training data creation, model training, evaluation, and transcription.
- tismir_related is a lighter deployment-oriented package with pretrained models, normalization stats, utilities, and transcription code, but no augmentation/training scripts.
3. Input data boundary
- The repo assumes a local audio dataset under ismir_related/dataset, plus curated song lists that define which recordings belong to each split or augmentation condition.
- Separate cv_folds files define reproducible cross-validation and test partitions, so data splitting is explicit and not inferred at runtime.
4. Augmentation subsystem
- ismir_related/code/data-augmentation provides standalone scripts for generating transformed audio variants and an accompanying templates.npy.
- The generated examples in audio-examples and the songlists show that augmentation is treated as a preprocessing layer before classification, producing multiple derived datasets from the same original recordings.
5. Training data preparation
- make_train_data.py and make_test_data.py are the bridge between raw/augmented audio and model-ready tensors or feature arrays.
- means_stds.npy stores normalization statistics used to standardize features consistently during training and inference.
6. Model training and evaluation
- train.py is the core training entry point.
- eval_cv.py and eval_test.py separate cross-validation assessment from held-out test evaluation, indicating a research workflow where results are reported on both internal folds and a fixed test set.
7. Inference / transcription path
- transcribe.py in both subprojects is the runtime entry point for classifying new audio.
- It loads pretrained checkpoints from saved_models, applies the same normalization/feature logic from utils.py, and outputs stroke labels over the input audio stream.
8. Model artifact layout
- The checkpoints are serialized as PyTorch .pt files under saved_models, so the architecture is built around persisted neural network weights rather than an online service.
- In ismir_related, models are organized by class family under 1-way/B, 1-way/D, 1-way/RB, 1-way/RT, each with multiple numbered checkpoints, suggesting ensemble or fold-specific models.
- tismir_related/saved_models similarly contains class-specific checkpoints for the later paper’s transcription setup.
9. Reproducibility and experiment tracking
- Fixed fold files, song lists, saved normalization stats, and stored checkpoints make the repo reproducible without regenerating the full training corpus each time.
- The repository therefore functions as an experiment artifact bundle as much as a codebase.
10. How the major pieces connect
- Raw tabla audio → optional augmentation → train/test data creation → model training → saved checkpoints + normalization stats → transcription/evaluation.
- The architecture is centered on this offline ML loop, with the TISMIR package representing the reusable inference slice of the larger ISMIR workflow.
Main entry points: train.py Training · make_train_data.py Data prep · means_stds.npy Mean/std stats · transcribe.py Transcribe
Components 16
Repository
Offline tabla stroke transcription artifacts and code
- Tabla audio · Local recordings used as the source corpus
ismir_related/dataset
ISMIR workflow
Full experimental pipeline with augmentation, training, and evaluation
- Song lists · Curated recording lists that define dataset conditions and splits
ismir_related/code/stroke-classification/songlists - CV folds · Fixed cross-validation and test partitions
ismir_related/code/stroke-classification/cv_folds - Augmentation · Standalone audio transformation generators for derived training variants
ismir_related/code/data-augmentation - Templates · Template array used by augmentation scripts
ismir_related/code/data-augmentation/templates.npy - Data prep · Builds model-ready train and test datasets from audio and metadata
ismir_related/code/stroke-classification/make_train_data.py - Mean/std stats · Feature normalization constants shared across training and inference
ismir_related/code/stroke-classification/means_stds.npy - Training · Core training entry point for stroke classifiers
ismir_related/code/stroke-classification/train.py - CV eval · Cross-validation scoring over the fixed folds
ismir_related/code/stroke-classification/eval_cv.py - Test eval · Held-out test-set evaluation
ismir_related/code/stroke-classification/eval_test.py - Transcribe · ISMIR runtime classifier for new audio
ismir_related/code/stroke-classification/transcribe.py - ISMIR models · Class-family checkpoints for the ISMIR workflow
ismir_related/code/stroke-classification/saved_models - ISMIR utils · Shared normalization and feature-processing helpers
ismir_related/code/stroke-classification/utils.py
TISMIR package
Deployment-oriented inference package with pretrained models
- Transcribe · TISMIR runtime classifier for new audio
tismir_related/transcribe.py - TISMIR models · Pretrained checkpoints bundled for deployment-oriented inference
tismir_related/saved_models - TISMIR utils · Shared normalization and feature-processing helpers
tismir_related/utils.py
Connections 20
- Tabla audio → Song lists: organized by · Curated lists select recordings and augmentation conditions
- Tabla audio → Augmentation: transformed by · Raw recordings are expanded into derived variants
- Templates → Augmentation: guides · Template data supports augmentation generation
- Song lists → CV folds: defines splits · Lists and fold files make partitioning reproducible
- Song lists → Data prep: inputs · Split manifests feed training data creation
- CV folds → Data prep: partitions · Fold definitions drive train/test assembly
- Augmentation → Data prep: feeds · Augmented examples are consumed during data preparation
- Data prep → Mean/std stats: scales with · Prepared features use shared mean/std statistics
- Data prep → Training: supplies · Model-ready tensors are used for training
- Mean/std stats → Training: applies · Training uses the same feature standardization
- Training → ISMIR models: writes · Training produces stored checkpoints
- Training → CV eval: assessed by · Cross-validation evaluates trained models
- Training → Test eval: assessed by · Held-out testing evaluates trained models
- ISMIR models → Transcribe: loaded by · Inference loads persisted checkpoints
- ISMIR utils → Transcribe: supports · Shared feature logic powers runtime classification
- Mean/std stats → Transcribe: applies · Inference uses the same normalization stats
- ISMIR models → Transcribe: loaded by · Deployment inference loads pretrained checkpoints
- TISMIR utils → Transcribe: supports · Shared feature logic powers runtime classification
- TISMIR models → Transcribe: loaded by · Transcription uses bundled pretrained weights
- Mean/std stats → Transcribe: applies · Deployment inference reuses shared feature scaling