# duongngockhanh/vae-colorization architecture

> Architecture diagram of the GitHub repository duongngockhanh/vae-colorization, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/duongngockhanh/vae-colorization
- Repository: https://github.com/duongngockhanh/vae-colorization

## Overview

##### Project type and purpose
- This is a research‐oriented deep learning pipeline for image colorization. It implements a Conditional Variational Autoencoder (CVAE) for training, and a Mixture Density Network (MDN) for inference, all in PyTorch.

##### File structure
- Top‐level scripts and modules:
- main.py – entry point for training and inference
- config.py – configuration (hyperparameters, paths)
- dataloader.py & dataset.sh – data‐preparation and loader utilities
- vae.py – VAE (encoder, decoder, conditional encoder) implementation
- mdn.py – MDN implementation used at inference time
- losses.py – loss functions for CVAE training
- Supporting directories:
- example/ – sample Jupyter notebook demonstrating end-to-end usage

##### README for additional insights
- Training stage: basic VAE + conditional encoder
- Inference stage: MDN takes VGG‐extracted features and generates Gaussian mixture parameters
- External dependencies: LFW dataset download, pre-trained VGG features, model weight downloads

##### Main components
a. Data Preparation
- LFW dataset (download via dataset.sh) → Preprocessing → dataloader.py
b. Model Training (CVAE)
- Encoder (vae.py) → latent distribution P
- Conditional Encoder (vae.py) → context features
- losses.py computes reconstruction + KL losses
- main.py orchestrates training loop, saves model_vae.pth
c. Model Inference (MDN stage)
- Input: Grayscale image G
- Pre-trained VGG feature extractor (external dependency) → feature vector
- mdn.py (Mixture Density Network) → mixture parameters of P
d. Configuration & Scripts
- config.py holds paths, hyperparameters
- main.py reads config, dispatches train vs. infer
- dataset.sh automates data download
e. External Services & Artifacts
- Google Drive links for pre-trained weights and dataset
- Pre-trained VGG network (Colorful Image Colorization)

##### Relationships and interactions
- Data flow is strictly linear: Dataset → Dataloader → CVAE training → model weights
During inference: Grayscale → VGG → MDN → CVAE Decoder → Output
- main.py sits at the center, invoking dataloader → model → loss modules
- config.py is imported by main.py and all submodules

##### Architectural patterns & design principles
- Modular separation: data I/O, model definitions, loss functions, configuration
- Pipeline architecture: distinct training and inference sub-pipelines
- Dependency injection via config.py
- Reusability: decoder block is reused in both training and inference

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Preparation Layer
    subgraph "Data Preparation" 
        direction TB
        Dataset["[(LFW Dataset)]"]:::data
        DatasetScript["dataset.sh"]:::data
        Dataloader["dataloader.py"]:::data
    end

    %% Training Pipeline Layer
    subgraph "Training Pipeline" 
        direction TB
        Main["main.py"]:::external
        Config["config.py"]:::external
        subgraph "CVAE Model" 
            direction TB
            Encoder["Encoder (vae.py)"]:::training
            CondEncoder["Conditional Encoder (vae.py)"]:::training
            Decoder["Decoder (vae.py)"]:::training
        end
        Losses["losses.py"]:::training
        ModelWeights["[(model_vae.pth)]"]:::data
    end

    %% Inference Pipeline Layer
    subgraph "Inference Pipeline"
        direction TB
        GrayInput["Grayscale G"]:::inference
        VGG["Pre-trained VGG"]:::external
        FeatureVec["Feature Vector"]:::inference
        MDN["MDN (mdn.py)"]:::inference
        GMMParams["Mixture Params"]:::inference
        SampleZ["Sample Latent z"]:::inference
        PredColor["Decoder (vae.py)"]:::inference
        Merge["Merge with Grayscale"]:::inference
        Output["[(data/output)]"]:::data
    end

    %% External Drivers
    GoogleDrive["Google Drive (Weights & Dataset)"]:::external
    Notebook["example/VAE_based_Image_Colorization.ipynb"]:::external
    Images["images/"]:::external

    %% Connections - Data Preparation to Training
    Dataset -->|download & preprocess| DatasetScript
    DatasetScript -->|prepare batches| Dataloader
    Dataloader -->|batches of C + G| Main
    Config -->|parameters| Main

    %% Training Flow
    Main -->|invoke| Encoder
    Main -->|invoke| CondEncoder
    Encoder -->|"latent P"| Decoder
    CondEncoder -->|"context features"| Decoder
    Decoder -->|"reconstructed C"| Losses
    Losses -->|compute losses| Main
    Main -->|save weights| ModelWeights

    %% Inference Flow
    GrayInput -->|input image| VGG
    VGG -->|features| FeatureVec
    FeatureVec -->|input| MDN
    MDN -->|mixture params| GMMParams
    GMMParams -->|sample z| SampleZ
    SampleZ -->|decode| PredColor
    PredColor -->|color field| Merge
    Merge -->|save image| Output

    %% External artifacts
    GoogleDrive -->|provide| Dataset
    GoogleDrive -->|provide| VGG
    Notebook -->|demo| Main
    Images -->|assets| Notebook

    %% Click Events
    click DatasetScript "https://github.com/duongngockhanh/vae-colorization/blob/main/dataset.sh"
    click Dataloader "https://github.com/duongngockhanh/vae-colorization/blob/main/dataloader.py"
    click Config "https://github.com/duongngockhanh/vae-colorization/blob/main/config.py"
    click Main "https://github.com/duongngockhanh/vae-colorization/blob/main/main.py"
    click Encoder "https://github.com/duongngockhanh/vae-colorization/blob/main/vae.py"
    click CondEncoder "https://github.com/duongngockhanh/vae-colorization/blob/main/vae.py"
    click Decoder "https://github.com/duongngockhanh/vae-colorization/blob/main/vae.py"
    click Losses "https://github.com/duongngockhanh/vae-colorization/blob/main/losses.py"
    click MDN "https://github.com/duongngockhanh/vae-colorization/blob/main/mdn.py"
    click Notebook "https://github.com/duongngockhanh/vae-colorization/blob/main/example/VAE_based_Image_Colorization.ipynb"
    click Images "https://github.com/duongngockhanh/vae-colorization/tree/main/images/"

    %% Styles
    classDef data fill:#f96,stroke:#333,stroke-width:1px
    classDef training fill:#8fd3e8,stroke:#333,stroke-width:1px
    classDef inference fill:#a8e6cf,stroke:#333,stroke-width:1px
    classDef external fill:#d3d3d3,stroke:#333,stroke-width:1px
```
