How xiaocangshenggr/ddgr works
Project Type & Purpose
• The project is an implementation of the continual learning framework using deep diffusion‐based generative replay (DDGR). It accompanies an ICML2023 paper and is designed to experiment with continual learning on complex datasets (e.g., CIFAR100, ImageNet, CORe50).
• The primary focus is on training and evaluating models using a diffusion-based generative replay method, with additional support for conventional fine-tuning (“Finetune”) experiments.
Analyzing the File Structure
• Top-Level Directories/Files:
• The main project directory (py-DDGR1.1) contains general scripts (main.py, generate_imagenet.py, run_cifar100.py) and a requirement file.
• The src directory is subdivided into several key components:
▪ data: Contains data loaders, preprocessing scripts, and dataset-specific scripts (e.g., for CIFAR100, ImageNet, and CORe50).
▪ framework: Implements the core continual learning pipeline (training routines, evaluation, inference, and learning rate grid search) that orchestrate experiments.
▪ methods: Contains code for the DDGR method (with a dedicated diffusion sub-module featuring the diffusion process, unet, losses, etc.) and Finetune method.
▪ models: Contains the architectural definitions (for example, models like alexnet_fc7out, net.py) that are used in experiments.
▪ utilities: Holds helper functions for general tasks.
▪ config.init: Acts as a central configuration file where key parameters are set.
Key Components
• Data Ingestion & Preprocessing:
• Relevant sub-components: data loaders (cifarCI_dataprep.py, imgfolder.py, etc.) and dataset management.
• Continual Learning Framework:
• A “Framework” component that encapsulates training, evaluation, and inference routines.
• Sub-components to mention: framework_train.py, eval.py, inference.py, lr_grid_train.py.
• To indicate that these modules orchestrate the overall experimental pipeline.
• Methods for Learning:
▪ DDGR Method: Highlight the diffusion sub-module that includes functionalities for generating images, loss calculation, training (train_DDGR.py), and other utilities (e.g., dist_util.py, gaussian_diffusion.py, unet.py).
▪ Finetune Method: Indicate its separate training routines (main_SGD.py, train_SGD.py).
• Model Definitions:
• Utilities and Configuration:
• “Configuration” component (config.init) as a central file feeding parameters to all other modules.
• External Dependencies & Execution Scripts:
• Mention that there are standalone scripts (e.g., main.py, run_cifar100.py) that serve as entry points for running experiments.
• External dependencies (PyTorch, CUDA, mpi4py, opencv, pillow, etc.) as part of the environment setup.
Relationships and Interactions
• Datasets (raw and preprocessed) flow into the “Data Module”.
• Preprocessed data is then passed to the “Framework” which controls training and evaluation.
• The “Framework” calls out to different “Methods” (either DDGR or Finetune) based on configuration, and these methods in turn use model definitions from the “Models” component.
• Utilities and configuration settings serve as cross-cutting resources available to all major components.
• Integration points where scripts like generate_imagenet.py and run_cifar100.py initiate specific workflows.
Architectural Patterns & Design Principles
• The overall design is modular: clear separation between data processing, experimental pipeline management, method implementation, and model definition.
• While not a classical MVC design, the layering of “data” → “framework” → “methods” allows easy extensibility and isolated testing.
• Experimental/research nature of the architecture where components can be interchanged or augmented depending on the continual learning scenario.