How duongngockhanh/vae-colorization works
Project type and purpose
• This is a research‐oriented deep learning pipeline for image colorization. It implements a Conditional Variational Autoencoder (CVAE) for training, and a Mixture Density Network (MDN) for inference, all in PyTorch.
File structure
• Top‐level scripts and modules:
• main.py – entry point for training and inference
• config.py – configuration (hyperparameters, paths)
• dataloader.py & dataset.sh – data‐preparation and loader utilities
• vae.py – VAE (encoder, decoder, conditional encoder) implementation
• mdn.py – MDN implementation used at inference time
• losses.py – loss functions for CVAE training
• Supporting directories:
• example/ – sample Jupyter notebook demonstrating end-to-end usage
README for additional insights
• Training stage: basic VAE + conditional encoder
• Inference stage: MDN takes VGG‐extracted features and generates Gaussian mixture parameters
• External dependencies: LFW dataset download, pre-trained VGG features, model weight downloads
Main components
a. Data Preparation
• LFW dataset (download via dataset.sh) → Preprocessing → dataloader.py
b. Model Training (CVAE)
• Encoder (vae.py) → latent distribution P
• Conditional Encoder (vae.py) → context features
• losses.py computes reconstruction + KL losses
• main.py orchestrates training loop, saves model_vae.pth
c. Model Inference (MDN stage)
• Input: Grayscale image G
• Pre-trained VGG feature extractor (external dependency) → feature vector
• mdn.py (Mixture Density Network) → mixture parameters of P
d. Configuration & Scripts
• config.py holds paths, hyperparameters
• main.py reads config, dispatches train vs. infer
• dataset.sh automates data download
e. External Services & Artifacts
• Google Drive links for pre-trained weights and dataset
• Pre-trained VGG network (Colorful Image Colorization)
Relationships and interactions
• Data flow is strictly linear: Dataset → Dataloader → CVAE training → model weights
During inference: Grayscale → VGG → MDN → CVAE Decoder → Output
• main.py sits at the center, invoking dataloader → model → loss modules
• config.py is imported by main.py and all submodules
Architectural patterns & design principles
• Modular separation: data I/O, model definitions, loss functions, configuration
• Pipeline architecture: distinct training and inference sub-pipelines
• Dependency injection via config.py
• Reusability: decoder block is reused in both training and inference