# lrenault/ddsp-piano architecture

> Architecture diagram of the GitHub repository lrenault/ddsp-piano, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/lrenault/ddsp-piano
- Repository: https://github.com/lrenault/ddsp-piano

## Overview

##### Project Type and Purpose
- DDSP-Piano is an open-source research tool (CLI-based Python package) for MIDI-to-audio performance synthesis. It leverages the DDSP TensorFlow library to build a differentiable piano model, train it on the MAESTRO dataset, evaluate it, and run inference (synthesis) from MIDI or CSV inputs.

##### File Structure
- Top-level scripts (CLI entry points):
- preprocess_maestro.py – convert MAESTRO MIDI+audio into TFRecords
- train_single_phase.py & train_ddsp_piano.sh – train the model
- evaluate_model.py – evaluate on the MAESTRO test set
- synthesize_midi_file.py & synthesize_from_csv.py – run inference
- ddsp_piano/ package:
- configs/ – `.gin` model configurations
- data_pipeline.py – reads TFRecords, batches data
- default_model.py – assembles the TF model from modules
- modules/ – core synthesis building blocks (oscillators, reverb, losses)
- utils/ – I/O helpers, MIDI encoders, logging summaries
- model_weights/ – pre‐trained checkpoints
- Training outputs: experiment directories with logs/, checkpoints, TensorBoard data

##### Extract Additional Insights from the README
Data flow → Encoder (MIDI features) → Core DDSP synthesis modules → Reverb → Audio
- Dependencies: TensorFlow 2.x, DDSP library, Gin-Config
- Major operations: preprocessing, single-phase training, multi-phase (legacy), batch & single inference, evaluation

##### Main Components & Their Interactions
a. Components
1. Data Sources
– Raw MAESTRO dataset (MIDI + WAV)
2. Preprocessing
– preprocess_maestro.py → TFRecords storage
3. Configuration Manager
– `.gin` files in configs/ determine model hyperparameters and architecture
4. Data Pipeline
– data_pipeline.py loads TFRecords, applies on-the-fly MIDI encoding / audio resampling
5. Model Builder
– default_model.py + modules/ assemble the DDSP graph (partials, oscillators, reverb)
6. Training Engine
– train_single_phase.py & train_ddsp_piano.sh
– Inputs: Data Pipeline + Model Builder + configs + training hyperparameters
– Outputs: Checkpoints → ddsp_piano/model_weights or `<experiment>/phase_X/`
– Logs: TensorBoard under `<experiment>/logs/`
7. Inference Engine
– synthesize_midi_file.py / synthesize_from_csv.py
– Inputs: MIDI file or CSV batch, config, checkpoint
– Processing: Load Model Builder + weights, run forward pass
– Outputs: WAV files (with optional dry vs. reverbed tracks)
8. Evaluation Engine
– evaluate_model.py
– Inputs: MAESTRO TFRecords, config, checkpoint
– Outputs: Objective metrics, optional audio samples
9. External Dependencies
– TensorFlow / DDSP library
– Gin-Config for hyperparameter injection
b. Relationships & Data Flows
c. Architectural Patterns & Principles
– Modular separation (data / config / model / utils)
– Pipeline pattern for data ingestion → training → inference
– Dependency injection via Gin-Config
– CLI orchestration for each major phase (preprocess, train, eval, synthesize)
d. Key Technologies & Frameworks
– Python 3
– TensorFlow 2.x & DDSP library
– Gin-Config for hyperparameters
– TFRecord format for fast I/O
– Bash scripts for full training orchestration
– CSV/MIDI input handling
– TensorBoard for logging and visualizations

##### Detail
- Config files as first-class components that feed into model construction.
- Two inference modes (single MIDI vs. batch CSV) converging on the same Inference Engine.
- Don’t omit the evaluation path (metrics + optional audio).

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Data Storage
    RawData["Raw MAESTRO Dataset\n(MIDI + WAV)"]:::data
    TFRecords["TFRecords Storage"]:::data
    Checkpoints["Model Checkpoints"]:::data
    Logs["TensorBoard Logs"]:::data
    WAV["WAV Outputs"]:::data
    Metrics["Metrics & Audio Samples"]:::data

    %% Configuration & External
    ConfigMgr["Configuration Manager\n(.gin files)"]:::config
    TFDS["TensorFlow & DDSP Library"]:::external
    Gin["Gin-Config"]:::external

    %% Compute Stages
    Preprocess["Preprocessing\n(preprocess_maestro.py)"]:::compute
    DataPipeline["Data Pipeline\n(data_pipeline.py)"]:::compute
    subgraph "Model Builder\n(default_model.py + modules)"
        DefaultModel["default_model.py"]:::compute
        Modules["Core Synthesis Modules"]:::compute
    end
    subgraph "Training Engine"
        TrainSingle["train_single_phase.py"]:::compute
        TrainFull["train_ddsp_piano.sh"]:::compute
    end
    subgraph "Inference Engine"
        InferenceMIDI["synthesize_midi_file.py"]:::compute
        InferenceCSV["synthesize_from_csv.py"]:::compute
    end
    Evaluate["Evaluation Engine\n(evaluate_model.py)"]:::compute

    %% Data & Control Flow
    RawData --> Preprocess --> TFRecords --> DataPipeline --> TrainSingle
    TFRecords --> DataPipeline --> TrainFull
    ConfigMgr --> DefaultModel
    Gin --> ConfigMgr
    TFDS --> DefaultModel
    DataPipeline --> TrainSingle
    DefaultModel --> TrainSingle
    DefaultModel --> TrainFull
    TrainSingle --> Checkpoints
    TrainFull --> Checkpoints
    TrainSingle --> Logs
    TrainFull --> Logs
    Checkpoints --> InferenceMIDI
    Checkpoints --> InferenceCSV
    ConfigMgr --> InferenceMIDI
    ConfigMgr --> InferenceCSV
    DefaultModel --> InferenceMIDI
    DefaultModel --> InferenceCSV
    InferenceMIDI --> WAV
    InferenceCSV --> WAV
    Checkpoints --> Evaluate
    ConfigMgr --> Evaluate
    DefaultModel --> Evaluate
    TFRecords --> Evaluate
    Evaluate --> Metrics

    %% Click Events
    click Preprocess "https://github.com/lrenault/ddsp-piano/blob/main/preprocess_maestro.py"
    click TrainSingle "https://github.com/lrenault/ddsp-piano/blob/main/train_single_phase.py"
    click TrainFull "https://github.com/lrenault/ddsp-piano/blob/main/train_ddsp_piano.sh"
    click Evaluate "https://github.com/lrenault/ddsp-piano/blob/main/evaluate_model.py"
    click InferenceMIDI "https://github.com/lrenault/ddsp-piano/blob/main/synthesize_midi_file.py"
    click InferenceCSV "https://github.com/lrenault/ddsp-piano/blob/main/synthesize_from_csv.py"
    click ConfigMgr "https://github.com/lrenault/ddsp-piano/tree/main/ddsp_piano/configs/"
    click DataPipeline "https://github.com/lrenault/ddsp-piano/blob/main/ddsp_piano/data_pipeline.py"
    click DefaultModel "https://github.com/lrenault/ddsp-piano/blob/main/ddsp_piano/default_model.py"
    click Modules "https://github.com/lrenault/ddsp-piano/tree/main/ddsp_piano/modules/"
    click Checkpoints "https://github.com/lrenault/ddsp-piano/tree/main/ddsp_piano/model_weights/"

    %% Styles
    classDef data fill:#add8e6,stroke:#1f78b4,color:#000;
    classDef compute fill:#90ee90,stroke:#2ca02c,color:#000;
    classDef config fill:#ffcc80,stroke:#ff7f0e,color:#000;
    classDef external fill:#ffcc80,stroke:#9467bd,color:#000;
```
