# nvlabs/nvae architecture

> Architecture diagram of the GitHub repository nvlabs/nvae, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nvlabs/nvae
- Repository: https://github.com/nvlabs/nvae

## Overview

##### Project Type and Purpose
- NVAE is a research-grade Python project implementing a deep hierarchical Variational Autoencoder (VAE) in PyTorch.
- It provides end-to-end functionality: dataset preparation (LMDB/TFRecord conversion), model definition, training (single-node & distributed), evaluation (log-likelihood, sampling), and FID computation.

##### High-Level Architecture Components
a. Data Ingestion & Preprocessing
- `scripts/` (Dockerfile, conversion & LMDB-creation scripts)
- `datasets.py`, `lmdb_datasets.py`, `distributions.py`
- External data sources (MNIST, CIFAR-10, CelebA, ImageNet TFRecords, LSUN)
- LMDB/TFRecord storage
b. Model Core
- `model.py` (hierarchical encoder/decoder layers)
- `neural_operations.py`, `neural_ar_operations.py` (building blocks: residual cells, attention, autoregressive ops)
- `thirdparty/` (custom ops: adamax optimizer, sync batchnorm, LSUN loader, swish activation)
c. Training Engine
- `train.py` (single-node or MPI-based distributed training orchestrator)
- `utils.py` (hyperparameter helpers, group scheduling)
- Checkpoint I/O (saving/loading to `root` directory)
- TensorBoard logging
d. Evaluation Suite
- `evaluate.py` (modes: sample, evaluate, evaluate_fid)
- `fid/` module (compute FID via Inception network)
- `scripts/precompute_fid_statistics.py`
e. Deployment & Environment
- `requirements.txt` (Python/PyTorch dependencies)
- `scripts/Dockerfile` (containerization)
- `LICENSE` and `README.md` (documentation)

##### Relationships & Data Flows
- Data flows from External Data Sources → Preprocessing Scripts → LMDB Storage.
- Training Engine reads from LMDB → builds Model Core → runs forward/backward passes on GPUs → writes Checkpoints + TensorBoard logs.
- Evaluation reads Checkpoints + LMDB → Model Core inference → outputs samples, log-likelihood metrics, FID scores.
- `thirdparty/` provides extensions to core PyTorch ops, invoked by both training and evaluation.
- Distributed training uses MPI (`mpirun`), coordinating across multiple nodes via `train.py`.

##### Architectural Patterns & Design Principles
- Modular Pipeline: clear separation of preprocessing, modeling, training, evaluation.
- Layered Architecture: top-level scripts orchestrate tasks; `model.py` and operation modules encapsulate core logic; `thirdparty/` isolates external dependencies.
- Distributed Execution: MPI-based multi-node training with single entrypoint.
- Data Abstraction: unified LMDB dataset interface for diverse image sources.

f. In a corner, illustrate “Containerization & Env” (`Dockerfile`, `requirements.txt`).

##### Essential Elements
- Versioning & configuration inputs (e.g., hyperparameters via CLI flags).
- MPI orchestration overlay for distributed training.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Data Layer
    subgraph "Data Layer" 
        direction TB
        A["Data Sources\n(MNIST, CIFAR-10,\nTFRecords, LSUN)"]:::data
        B["Preprocessing Scripts"]:::data
        C[(LMDB/TFRecord Storage)]:::data
    end

    %% Model & Operations
    subgraph "Model Core" 
        direction TB
        D["model.py"]:::model
        E["neural_operations.py"]:::model
        F["neural_ar_operations.py"]:::model
        subgraph "thirdparty/" 
            direction TB
            G1["adamax.py"]:::model
            G2["functions.py"]:::model
            G3["inplaced_sync_batchnorm.py"]:::model
            G4["lsun.py"]:::model
            G5["swish.py"]:::model
        end
    end

    %% Training Orchestration
    subgraph "Training Engine" 
        direction TB
        H["train.py"]:::train
        I["utils.py"]:::train
        J["GPU Cluster (MPI)"]:::infra
    end
    K[(Model Checkpoints)]:::infra
    L[(TensorBoard Logs)]:::infra

    %% Evaluation & Metrics
    subgraph "Evaluation Suite" 
        direction TB
        M["evaluate.py"]:::eval
        subgraph "fid/" 
            direction TB
            N1["inception.py"]:::eval
            N2["fid_score.py"]:::eval
            N3["__init__.py"]:::eval
        end
    end
    O["Samples"]:::eval
    P["Log-Likelihood"]:::eval
    Q["FID Score"]:::eval

    %% Deployment & Environment
    subgraph "Deployment & Env" 
        direction TB
        R["requirements.txt"]:::infra
        S["Dockerfile"]:::infra
        T["LICENSE"]:::infra
        U["README.md"]:::infra
    end

    %% Data Flow Arrows
    A -->|raw images| B
    B -->|batches| C
    C -->|training data| H
    C -->|eval data| M
    H -->|checkpoints.pt| K
    H -->|tensorboard logs| L
    K -->|load checkpoints| M
    M -->|samples| O
    M -->|metrics| P
    M -->|FID stats| Q

    %% Model Core reuse
    H -.->|uses| D
    H -.->|uses| E
    H -.->|uses| F
    H -.->|uses| G1
    H -.->|uses| G2
    H -.->|uses| G3
    H -.->|uses| G4
    H -.->|uses| G5
    M -.->|uses| D
    M -.->|uses| E
    M -.->|uses| F
    M -.->|uses| G1
    M -.->|uses| G2
    M -.->|uses| G3
    M -.->|uses| G4
    M -.->|uses| G5

    %% Containerization env arrow
    S -->|container| H
    S -->|container| M

    %% Click Events
    click B "https://github.com/nvlabs/nvae/tree/master/scripts/"
    click B "https://github.com/nvlabs/nvae/blob/master/datasets.py"
    click B "https://github.com/nvlabs/nvae/blob/master/lmdb_datasets.py"
    click B "https://github.com/nvlabs/nvae/blob/master/distributions.py"
    click G1 "https://github.com/nvlabs/nvae/blob/master/thirdparty/adamax.py"
    click G2 "https://github.com/nvlabs/nvae/blob/master/thirdparty/functions.py"
    click G3 "https://github.com/nvlabs/nvae/blob/master/thirdparty/inplaced_sync_batchnorm.py"
    click G4 "https://github.com/nvlabs/nvae/blob/master/thirdparty/lsun.py"
    click G5 "https://github.com/nvlabs/nvae/blob/master/thirdparty/swish.py"
    click D "https://github.com/nvlabs/nvae/blob/master/model.py"
    click E "https://github.com/nvlabs/nvae/blob/master/neural_operations.py"
    click F "https://github.com/nvlabs/nvae/blob/master/neural_ar_operations.py"
    click H "https://github.com/nvlabs/nvae/blob/master/train.py"
    click I "https://github.com/nvlabs/nvae/blob/master/utils.py"
    click M "https://github.com/nvlabs/nvae/blob/master/evaluate.py"
    click N1 "https://github.com/nvlabs/nvae/blob/master/fid/inception.py"
    click N2 "https://github.com/nvlabs/nvae/blob/master/fid/fid_score.py"
    click N3 "https://github.com/nvlabs/nvae/blob/master/fid/__init__.py"
    click R "https://github.com/nvlabs/nvae/blob/master/requirements.txt"
    click S "https://github.com/nvlabs/nvae/tree/master/scripts/Dockerfile"
    click T "https://github.com/nvlabs/nvae/tree/master/LICENSE"
    click U "https://github.com/nvlabs/nvae/blob/master/README.md"

    %% Styles
    classDef data fill:#cce5ff,stroke:#3399ff,color:#000;
    classDef model fill:#d5f5e3,stroke:#27ae60,color:#000;
    classDef train fill:#ffe5b4,stroke:#e67e22,color:#000;
    classDef eval fill:#f5e6ff,stroke:#8e44ad,color:#000;
    classDef infra fill:#e0e0e0,stroke:#7f8c8d,color:#000;
```
