# misya11p/amt-apc architecture

> Architecture diagram of the GitHub repository misya11p/amt-apc, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-11-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/misya11p/amt-apc
- Repository: https://github.com/misya11p/amt-apc

## Overview

##### Project type and purpose
- This is an open-source ML pipeline for automatic piano cover generation (AMT-APC).
- It comprises data ingestion & preparation, model fine-tuning/training, inference (cover generation), and evaluation.
- Key use cases: prepare a dataset from raw audio/MIDI, fine-tune a pre-trained AMT model, generate piano covers, and compute evaluation metrics.

##### File structure
Top-level directories:
- data     – scripts for downloading raw audio, syncing with MIDI, extracting features (style vectors), labeling, and assembling the PyTorch dataset.
- dataset  – metadata (src.json) output from data scripts.
- infer    – CLI entry point for inference (cover generation) using the fine-tuned model; supports WAV files or YouTube URLs and style options.
- train    – CLI for training/fine-tuning (`__main__.py`, trainer, loss).
- eval     – scripts implementing cover-based metrics (cover.py, distance.py, f1.py), plus integration with an external ChromaCoverId tool.
- models   – model definitions and specs (Transformer‐based AMT, model_spec2midi), plus directory for parameter files (`models/params/`).
- utils    – configuration loader, global info.
Other:
- config.json – default paths and hyperparameters.
- requirements.txt – Python dependencies.
- docs      – static HTML site for project page.

##### README for additional insights
- Describes two workflows: “Piano Cover Generation” (inference) and “Training & Evaluation.”
- Lists dependencies, command lines, external downloads (pre-trained models, yt-dlp, dataset).
- Mentions config.json for override, and evaluation metrics from an external repo.

A. External/Raw Data Sources
1. Online audio (YouTube via yt-dlp)
2. Raw WAV files
3. MIDI files (for alignment)
B. Data Pipeline
- download.py → raw audio
- sync.py → audio/MIDI alignment
- transcribe.py → generate initial AMT outputs
- sv/extract.py → style-vector extraction
- create_labels.py → generate target labels
- create_dataset.py → build PyTorch Dataset (`dataset/src.json`)
C. Model Artifacts
- Pre-trained AMT model (amt.pth) in models/params
- Fine-tuned APC model (apc.pth) in models/params
D. Training Pipeline
- train/__main__.py
- _trainer.py (training loop), _loss.py
- Reads Dataset and config.json (via utils/_config.py)
E. Inference (Cover Generation)
- infer/__main__.py
- Loads APC model + pretty-midi, torchaudio, style vectors
- Outputs a new MIDI or WAV cover
F. Evaluation Pipeline
- eval/cover.py, distance.py, f1.py
- Optional integration: eval/ChromaCoverId (external repo)
G. Utilities & Configuration
- utils/_config.py, utils/_info.py, utils/info.json
- config.json at root

##### Relationships & Data Flows
1. External Sources → Data Pipeline → Dataset
2. Dataset + Pre-trained AMT Model → Training Pipeline → Fine-tuned APC Model
3. Fine-tuned APC Model + Input (WAV/YouTube) → Inference → Output Cover
4. Dataset or Generated Covers → Evaluation Pipeline → Metrics

##### Architectural patterns & design principles
- CLI-driven workflows (separate entry points for data‐prep, training, inference, evaluation)
- Modular Python packages for each concern (data, train, infer, eval, models, utils)
- Configuration via JSON + command-line overrides
- Reuse of a pre-trained AMT model (transfer learning)
- Clear separation: data ingestion vs. model logic vs. inference vs. evaluation

##### Detailed capture of essential elements
- Every script group (data, train, infer, eval) should appear explicitly or as sub-steps.
- Configuration store (config.json + utils) should be shown as feeding parameters into both Data and Training.
- The intermediate dataset (dataset/src.json) is a key hand-off point.
- Pre-trained vs. fine-tuned models must be distinct.

## Diagram (Mermaid)

```mermaid
graph TB
    %% External Sources
    subgraph "External Sources" 
        direction TB
        YT((YouTube via yt-dlp)):::external
        WAV((Raw WAV Files)):::external
        MIDI((MIDI Files)):::external
    end

    %% Data Pipeline
    subgraph "Data Pipeline" 
        direction TB
        DL["download.py"]:::data
        SY["sync.py"]:::data
        TR["transcribe.py"]:::data
        SV["sv/extract.py"]:::data
        CL["create_labels.py"]:::data
        CD["create_dataset.py"]:::data
    end

    %% Dataset Metadata
    DS["dataset/src.json"]:::data

    %% Model Artifacts
    MA["Model Artifacts (params)"]:::artifact

    %% Training Pipeline
    subgraph "Training Pipeline" 
        direction TB
        TM["train/__main__.py"]:::train
        TRN["train/_trainer.py"]:::train
        LS["train/_loss.py"]:::train
        CFG["config.json"]:::config
        UCFG["utils/_config.py"]:::config
    end

    %% Inference
    subgraph "Inference (Cover Generation)" 
        direction TB
        INF["infer/__main__.py"]:::infer
    end

    %% Evaluation Pipeline
    subgraph "Evaluation Pipeline" 
        direction TB
        EC["eval/cover.py"]:::eval
        ED["eval/distance.py"]:::eval
        EF["eval/f1.py"]:::eval
        CCID[(ChromaCoverId)]:::external
    end

    %% Utilities & Global Info
    subgraph "Utilities & Configuration"
        direction TB
        UINFO["utils/_info.py"]:::config
        INFOJSON["utils/info.json"]:::config
        ROOTCFG["config.json"]:::config
    end

    %% Flows
    YT -->|audio| DL
    WAV -->|audio| DL
    MIDI -->|for alignment| SY
    DL --> SY
    SY --> TR
    TR --> SV
    SV --> CL
    CL --> CD
    CD -->|dataset| DS

    DS -->|input| TM
    MA -->|pre-trained amt.pth| TM
    CFG --> TM
    UCFG --> TM
    TM -->|apc.pth| MA

    MA --> INF
    INF -->|cover MIDI/WAV| EC
    EC -->|metrics| EF
    EC -->|features| ED
    CCID --> EC

    DS --> EC

    %% Click Events
    click DL "https://github.com/misya11p/amt-apc/blob/main/data/download.py"
    click SY "https://github.com/misya11p/amt-apc/blob/main/data/sync.py"
    click TR "https://github.com/misya11p/amt-apc/blob/main/data/transcribe.py"
    click SV "https://github.com/misya11p/amt-apc/blob/main/data/sv/extract.py"
    click CL "https://github.com/misya11p/amt-apc/blob/main/data/create_labels.py"
    click CD "https://github.com/misya11p/amt-apc/blob/main/data/create_dataset.py"
    click DS "https://github.com/misya11p/amt-apc/blob/main/dataset/src.json"
    click MA "https://github.com/misya11p/amt-apc/tree/main/models/params/"
    click TM "https://github.com/misya11p/amt-apc/blob/main/train/__main__.py"
    click TRN "https://github.com/misya11p/amt-apc/blob/main/train/_trainer.py"
    click LS "https://github.com/misya11p/amt-apc/blob/main/train/_loss.py"
    click CFG "https://github.com/misya11p/amt-apc/blob/main/config.json"
    click UCFG "https://github.com/misya11p/amt-apc/blob/main/utils/_config.py"
    click INF "https://github.com/misya11p/amt-apc/blob/main/infer/__main__.py"
    click EC "https://github.com/misya11p/amt-apc/blob/main/eval/cover.py"
    click ED "https://github.com/misya11p/amt-apc/blob/main/eval/distance.py"
    click EF "https://github.com/misya11p/amt-apc/blob/main/eval/f1.py"
    click UINFO "https://github.com/misya11p/amt-apc/blob/main/utils/_info.py"
    click INFOJSON "https://github.com/misya11p/amt-apc/blob/main/utils/info.json"
    click ROOTCFG "https://github.com/misya11p/amt-apc/blob/main/config.json"

    %% Styles
    classDef data fill:#d4fcbc,stroke:#333,stroke-width:1px
    classDef train fill:#cce5ff,stroke:#333,stroke-width:1px
    classDef infer fill:#ffe5b4,stroke:#333,stroke-width:1px
    classDef eval fill:#e5ccff,stroke:#333,stroke-width:1px
    classDef external fill:#eef,stroke:#555,stroke-width:1px,stroke-dasharray: 5 5
    classDef config fill:#f0f0f0,stroke:#333,stroke-width:1px
    classDef artifact fill:#fff2cc,stroke:#333,stroke-width:1px
```
