How jlingohr/magenta-torch works
Project type + purpose (from README + tree)
This is an ML research/engineering codebase: a PyTorch implementation of MusicVAE using recurrent architectures (LSTM/GRU). It’s primarily a training + inference pipeline for symbolic music (MIDI), not a web service.
Top-level entrypoints / “system boundary”
• scripts/
• preprocess.py → CLI/runner for dataset creation
• train.py → CLI/runner for training
• evaluate.py → CLI/runner for evaluation (and possibly sampling quality metrics)
• conf.yml → central configuration for paths, hyperparameters, model settings
• requirements.txt → external runtime deps (PyTorch, YAML, MIDI libs, etc.)
These are the “outer interface” of the system.
Core library modules (src/) = internal components
• Data path
• src/preprocess.py → library preprocessing logic (invoked by scripts/preprocess.py)
• src/dataset.py → Dataset/Dataloader definitions and data access
• src/transform.py → feature transforms / normalization / encoding steps
• src/midi_functions.py → MIDI I/O + conversions (MIDI ↔ events/tokens/roll)
• Model path
• src/model.py → MusicVAE model assembly (encoder/decoder, latent sampling)
• src/layers.py → reusable NN building blocks (RNN cells, projections, etc.)
• src/loss.py → VAE losses (recon + KL, possibly annealing schedules)
• Training + lifecycle
• src/trainer.py → training loop orchestration (epochs, optimizer steps, logging hooks)
• src/checkpoint.py → save/load model/optimizer states
• src/sampler.py → latent sampling / sequence generation utilities
• src/reconstruction.py → encode-decode reconstruction flow (for eval/inspection)
Preprocess flow
• scripts/preprocess.py → src/preprocess.py → src/midi_functions.py + src/transform.py
• Output: processed dataset artifacts (files on disk)
Training flow
• scripts/train.py → src/trainer.py
• trainer.py uses:
• src/dataset.py (loads processed artifacts)
• src/model.py + src/layers.py
• src/loss.py
• src/checkpoint.py (writes checkpoints)
• Output: checkpoints + (optional) logs/metrics
Evaluation flow
• scripts/evaluate.py → likely src/reconstruction.py and/or src/sampler.py
• Reads: checkpoints + dataset
• Outputs: metrics + reconstructed/generated MIDI (via src/midi_functions.py)
• CLI Runners (scripts/*.py)
• Core Library (src/*)
• File-based Artifacts
• Raw MIDI dataset (input)
• Preprocessed dataset (intermediate)
• Checkpoints (model artifacts)
• Generated/Reconstructed MIDI (output)
• External Libraries
• PyTorch (training runtime)
• MIDI parsing/writing library (whatever requirements.txt implies)
• YAML loader for conf.yml
• Raw MIDI → Preprocess → Preprocessed dataset
• Preprocessed dataset → Training → Checkpoints
• Checkpoints + Dataset → Evaluate/Sample → MIDI outputs + metrics
This is where you “separate into as many components as possible” and show crisp dependencies.
Data & Representation
• midi_functions.py
• transform.py
• preprocess.py
• dataset.py
Model
• model.py
• Depends on: layers.py
• Used by: trainer.py, reconstruction.py, sampler.py
• loss.py
• Used by: trainer.py
Training & Lifecycle
• trainer.py
• Depends on: model.py, loss.py, dataset.py, checkpoint.py
• checkpoint.py
• Used by: trainer.py, evaluate.py, sampler.py, reconstruction.py
Inference utilities
• sampler.py
• Uses: model.py, checkpoint.py, outputs MIDI via midi_functions.py
• reconstruction.py
• Uses: model.py, dataset.py/data batches, outputs MIDI via midi_functions.py
Relationships/interactions to highlight (what matters architecturally here)
Because this is a VAE training pipeline, emphasize these interactions:
• Encoding/decoding loop: Dataset batch → Encoder (RNN) → latent z → Decoder (RNN) → reconstructed sequence
• Loss computation: recon loss + KL (and any schedules/annealing that loss.py likely supports)
• Checkpoint lifecycle: periodic save during training; load for evaluation/sampling
• Representation conversions:
• MIDI ↔ internal event/token representation (central role of midi_functions.py)
• Transform/encoding steps that must match between preprocess and inference
Clear labels
• “scripts/train.py (CLI entrypoint)”
• “src/trainer.py (training loop)”
• “Checkpoint files (model.pt / optimizer state)”
• “Preprocessed dataset (serialized tensors / indices)”
User/CLI Lane
• preprocess.py | train.py | evaluate.py (each reads conf.yml)
Core Modules Lane (src/)
• Preprocess/Transform/MIDI funcs
• Dataset
• Model/Layers/Loss
• Trainer/Checkpoint
• Sampler/Reconstruction
Artifacts Lane (Disk)
• Raw MIDI
• Preprocessed dataset
• Checkpoints
• Generated MIDI + metrics
Then connect:
• preprocess runner → preprocess modules → preprocessed dataset
• train runner → trainer → (dataset + model + loss) → checkpoints
• eval runner → (checkpoint + model + sampler/reconstruction + midi funcs) → generated MIDI + metrics
• exact dataset artifact formats (e.g., .npz, .pt, .pkl)
• exact MIDI library used (e.g., pretty_midi, mido)
• hyperparameters that influence architecture (latent size, RNN type, sequence length)
• whether evaluation computes quantitative metrics or only reconstruction outputs