How juglab/hdn works
Main Components
• Data Sources and Ingestion: Represent the input image data (both natural and microscopy images). This corresponds to the examples folder where different datasets (e.g., BSD68 for natural images or Convallaria for microscopy images) are used.
• Noise Modeling Module: This component is responsible for creating noise models. Note the existence of notebooks “1-CreateNoiseModel.ipynb” in the Pixel_Noise as well as Structured_Noise examples and the lib directory files (such as gaussianMixtureNoiseModel.py, histNoiseModel.py, and stochastic.py) that implement noise modeling.
• Model Architecture and Definition: This is where the hierarchical VAE (diversity denoising networks) is built. The models folder contains key files (lvae.py and lvae_layers.py) that define the VAE-based architecture.
• Training Pipeline: Include both the standalone training script (training.py) and the corresponding notebook workflows (e.g., “1-Training.ipynb” in the respective subdirectories). This portion handles data pre-processing, noise model application, and optimization of the network parameters.
• Evaluation and Generation: Represent the evaluation phase shown by the “2-Evaluation.ipynb” (and “3-Evaluation.ipynb” for some pipelines) which uses the trained model to perform image restoration.
• Supporting Utility Functions: The lib folder (with files like utils.py, likelihoods.py, nn.py) supports core mathematical and neural network operations.
Relationships and Data Flow
● From the Noise Modeling Module, indicate a flow into the “Model Architecture” layer where the hierarchal VAE network is defined, drawing on the noise model parameters as additional inputs for training.
● Show a clear branch to the Evaluation Module where the trained model is used to predict or restore data.
Architectural Patterns and Design Principles
• Modular Design: Different aspects of the process (noise modeling, training, evaluation) are modularized in separate directories and scripts.
• Pipeline Workflow: The design follows a sequential pipeline—data ingestion, noise modeling, training, and evaluation—often implemented via Jupyter notebooks.
• Extensibility: Given that there are separate directories for pixel noise and structured noise, the architecture is extensible to different noise types with minimal changes.
• Reusability and Abstraction: Key shared functionalities (utilities and likelihood computations) are abstracted in the lib folder.
Relevant Technologies
• Deep Learning Framework: The project uses PyTorch (as indicated by the installation instructions involving cudatoolkit 10.1 and pytorch 1.7).
• Notebook Environment: Jupyter notebooks are heavily used to demonstrate end-to-end workflows (training, evaluation, noise model creation).
• Python Environment Management: The use of a conda environment (HDN.yml) for dependency management.
• Model-specific components such as VAEs and potentially custom layers (from lvae_layers.py).
Instructing the Principal Software Engineer
• Data Input (Raw Images)
• Noise Modeling (using both GaussianMixture and Histogram-based methods)
• Hierarchical DivNoising Network (the VAE architecture from the “models” directory)
• Training Pipeline (including feedback loops from evaluation)
• Evaluation/Prediction (different notebooks for pixel vs. structured noise)
• Clear labels to each node/component.