# sonyresearch/diffvox architecture

> Architecture diagram of the GitHub repository sonyresearch/diffvox, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sonyresearch/diffvox
- Repository: https://github.com/sonyresearch/diffvox

## Overview

##### Project type and purpose
- This is a research‐oriented, full‐pipeline Python application for training and analysing a differentiable vocal effects model (“DiffVox”).
- It covers data ingestion, model definition, training (parameter retrieval), preset collection, evaluation, PCA analysis, and visualization.

##### File structure
- Top level:
- cfg/ – configuration YAMLs (Hydra) and model definitions
- data/ – data‐loading logic (MedleyDB, internal datasets)
- loss/ – loss functions and evaluation script (loss_eval.py)
- modules/ – core differentiable DSP modules (functional, fx, real‐time, utilities)
- presets/ – precomputed parameter sets and PCA artifacts for two datasets
- scripts/ – auxiliary shell and Python scripts for dataset prep, parameter collection, PCA
- main.py – entry point for training/retrieval
- visualisation.ipynb – notebook for figures in the paper
- Build/config: requirements.txt, .gitignore, LICENSE, README.md

##### README for additional insights
- Uses Hydra for flexible configuration (cfg/config.yaml)
- Supports single‐track retrieval and batch MedleyDB processing (train_medley.sh)
- Parameter collection (get_params.py), evaluation (loss_eval.py), PCA transform (pca_trsfm.py)
- Generates presets in numpy/json format, then analyzes with notebooks or external tools

- Configuration Manager (Hydra, cfg/)
- Data Loader (data/)
- Model & DSP Chain (modules/functional.py, fx.py, rt.py)
- Loss & Evaluation (loss/, loss_eval.py)
- Training Orchestrator (main.py)
- Parameter Collector & Preset Manager (scripts/get_params.py, presets/)
- PCA Analyzer (scripts/pca_trsfm.py)
- Visualization Layer (visualisation.ipynb)
- External Dataset (MedleyDB, internal audio files)
- Storage/File System (logs, presets folder, output CSVs/NPYs)

1. Configuration Manager → Training Orchestrator
2. Training Orchestrator ↔ Data Loader (fetch audio)
3. Training Orchestrator → Model & DSP Chain → Loss & Evaluation → logs/checkpoints
4. Logs → Parameter Collector → presets/
5. Presets → PCA Analyzer → updated presets/
6. Loss/Evaluation → CSV outputs → Visualization Layer
7. External Dataset → Data Loader

c. Architectural patterns & design principles:
- Modular separation: data, model, loss, scripts
- Config-driven pipelines (Hydra) for reproducibility
- Batch vs single‐track modes via scripts vs main entry point
- Reuse of numpy artifacts for cross‐step handoff (raw_params.npy, feature_mask.npy, gaussian.npz)
- Optional GPU vs CPU evaluation (flag-driven)

d. Key technologies/frameworks:
- Python 3.x, numpy, scipy
- Hydra for configuration
- Numba for real‐time DSP modules
- YAML for configs, JSON for metadata
- Bash scripts for batch processing
- Jupyter for visualization
- Optional GPU (PyTorch or custom autodiff – inferred)

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Configuration & Data Ingestion"
        CFG["Configuration Manager"]:::blue
        DATA["Data Loader"]:::blue
        BM["Batch MedleyDB Processing"]:::blue
    end

    subgraph "Training & Model"
        ORCH["Training Orchestrator"]:::green
        subgraph "Model & DSP Chain"
            FUN["Functional Module"]:::green
            FX["FX Module"]:::green
            RT["Real-Time Module"]:::green
            UTIL["Utilities"]:::green
        end
        subgraph "Loss & Evaluation"
            LDR["Loss Functions"]:::green
            LEO["Evaluation Script"]:::green
        end
    end

    subgraph "Post-Processing & Analysis"
        subgraph "Parameter Collector & Presets"
            GP["Get Params Script"]:::orange
            PRE["Presets Storage"]:::orange
        end
        PCA["PCA Analyzer"]:::orange
        VIS["Visualization Layer"]:::orange
    end

    subgraph "External & Storage"
        EXT["External Dataset"]:::grey
        STOR["Storage/File System"]:::grey
    end

    CFG -->|"config YAML"| ORCH
    EXT -->|"audio files"| DATA
    DATA -->|"audio batch"| ORCH
    ORCH -->|"model input"| FUN
    FUN -->|"processed audio"| FX
    FX -->|"processed audio"| RT
    RT -->|"processed audio"| UTIL
    UTIL -->|"features"| LDR
    ORCH -->|"model predictions"| LEO
    LEO -->|"loss metrics"| STOR
    STOR -->|"logs/checkpoints"| PRE
    PRE -->|"presets (npy/json)"| STOR
    STOR -->|"presets"| PCA
    PCA -->|"updated presets"| STOR
    LEO -->|"CSV outputs"| VIS

    click CFG "https://github.com/sonyresearch/diffvox/tree/main/cfg/"
    click DATA "https://github.com/sonyresearch/diffvox/tree/main/data/"
    click BM "https://github.com/sonyresearch/diffvox/blob/main/scripts/train_medley.sh"
    click ORCH "https://github.com/sonyresearch/diffvox/blob/main/main.py"
    click FUN "https://github.com/sonyresearch/diffvox/blob/main/modules/functional.py"
    click FX "https://github.com/sonyresearch/diffvox/blob/main/modules/fx.py"
    click RT "https://github.com/sonyresearch/diffvox/blob/main/modules/rt.py"
    click UTIL "https://github.com/sonyresearch/diffvox/blob/main/modules/utils.py"
    click LDR "https://github.com/sonyresearch/diffvox/blob/main/loss/ldr.py"
    click LEO "https://github.com/sonyresearch/diffvox/blob/main/loss_eval.py"
    click GP "https://github.com/sonyresearch/diffvox/blob/main/scripts/get_params.py"
    click PRE "https://github.com/sonyresearch/diffvox/tree/main/presets/"
    click PCA "https://github.com/sonyresearch/diffvox/blob/main/scripts/pca_trsfm.py"
    click VIS "https://github.com/sonyresearch/diffvox/blob/main/visualisation.ipynb"

    classDef blue fill:#cce5ff,stroke:#333,stroke-width:1px;
    classDef green fill:#d4edda,stroke:#333,stroke-width:1px;
    classDef orange fill:#fff3cd,stroke:#333,stroke-width:1px;
    classDef grey fill:#e2e3e5,stroke:#333,stroke-width:1px;
```
