# mingyuliu1/liednet architecture

> Architecture diagram of the GitHub repository mingyuliu1/liednet, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mingyuliu1/liednet
- Repository: https://github.com/mingyuliu1/liednet

## Overview

##### Project type and purpose
- This is a research‐oriented deep-learning framework (full-stack ML pipeline) for low-light image enhancement and deblurring.
- It provides both training and inference entry points (train_llie.py, train_lolblur.py, inference_llie.py, inference_lol_blur.py) and evaluation scripts (calculate_pair.py, calculate_unpair.py).
- The core functionality lives in the `basicsr` Python package, extended here with the LIEDNet architecture (VSSM, LFM, DGDFFN).

##### File structure
- Top-level scripts & YAML configs
- train_llie.py, train_lolblur.py (training)
- inference_llie.py, inference_lol_blur.py (inference)
- calculate_pair.py, calculate_unpair.py (evaluation)
- options/*.yml (dataset/training configurations)
- Core library (`basicsr/`)
- archs/      – network definitions and utilities (csm_triton, csms6s, vmamba, zerodce, vgg, discriminator)
- data/       – dataset classes, samplers, degradations, transforms, dataloader
- losses/     – loss functions and helpers
- metrics/    – PSNR/SSIM, LPIPS, unpaired metrics
- models/     – base_model.py, lr_scheduler.py, liednet_model.py
- ops/        – custom CUDA kernels for performance (deform conv, fused act, upfirdn2d)
- transform/  – RGB data transformations
- utils/      – distributed training, downloading, file I/O, logging, options parsing, registry, etc.
- warmup_scheduler/ – warm-up LR scheduling scripts
- Setup & dependencies
- basicsr/setup.py for `pip install -e .`
- requirements.txt (PyTorch, CUDA≥11.6, other Python libs)
- kernels/ for compiling VSSM-specific CUDA kernels

##### Readme insights
- Confirms end-to-end LIEDNet framework with three modules: Visual State Space Module (VSSM), Local Feature Module (LFM), Dual Gated-Dconv Feedforward Network (DGDFFN).
- Training recipes for multiple datasets (LOL-Blur, LOL-v1, v2, FiveK).
- Inference and evaluation workflows.

a. Main components
1. CLI Entrypoints
- Training (train_llie.py, train_lolblur.py)
- Inference (inference_llie.py, inference_lol_blur.py)
- Evaluation (calculate_pair.py, calculate_unpair.py)
2. Configuration Manager
- Parses YAML options (options/*.yml via basicsr/utils/options.py)
3. Data Pipeline
- Dataset loader (LOL-Blur, LLIE datasets)
- Data transforms, degradations, batching (data/, transform/)
4. Core Model
- LIEDNet assembled from archs/ modules (VSSM, LFM, DGDFFN)
- BaseModel wrapper (models/base_model.py)
5. Custom Operations
- CUDA kernels (ops/dcn, ops/fused_act, ops/upfirdn2d)
6. Training Engine
- Optimizer + learning-rate scheduler (models/lr_scheduler.py, warmup_scheduler/)
- Loss computation (losses/)
- Backprop and parameter updates
7. Metrics & Evaluation
- Paired metrics (PSNR, SSIM, LPIPS)
- Unpaired metrics (MUSIQ, NRQM, NIQE)
8. Utilities
- Logging, checkpointing (utils/logger.py, utils/misc.py)
- Distributed training helpers (utils/dist_util.py)
- Download & registry (utils/download_util.py, utils/registry.py)
9. External Resources
- Disk I/O (datasets, pretrained models, output images)
- GPU devices (CUDA runtime)

b. Relationships and interactions
- User invokes a CLI script → Configuration Manager loads options → Data Pipeline yields batches → Core Model (with Custom Ops) processes inputs → Training Engine applies Loss → Optimizer updates Model → Metrics computed → Results and logs saved to disk → (for evaluation) metrics scripts read outputs and ground truth.
- Inference follows a similar flow without gradient/backprop.

c. Architectural patterns and design principles
- Modular separation: each concern (data, model, loss, metrics, ops, utils) in its own package.
- Plugin/registry pattern: options and models registered dynamically.
- Extensibility: new architectures or datasets can be dropped into `archs/` or `data/`.
- High-performance custom CUDA kernels co-located in `ops/`.

d. Key technologies
- PyTorch (tensor operations, autograd, modules)
- CUDA/C++ (custom kernels)
- YAML for configuration
- Python packaging (`setup.py`) for easy development install
- Python logging, distributed training utilities

## Diagram (Mermaid)

```mermaid
graph TB

    subgraph "CLI Entrypoints"
        TRAIN_LLIE["train_llie.py"] 
        TRAIN_LOL["train_lolblur.py"]
        INF_LLIE["inference_llie.py"]
        INF_LOL["inference_llol_blur.py"]
        EVAL_PAIR["calculate_pair.py"]
        EVAL_UNPAIR["calculate_unpair.py"]
    end

    subgraph "Configuration Manager"
        CONFIG["options Parser\n(basicsr/utils/options.py)"]
    end

    subgraph "Data Pipeline"
        DATASET1["lol_image_dataset.py"]
        DATASET2["paired_image_dataset.py"]
        SAMPLER["data_sampler.py"]
        DATALOADER["prefetch_dataloader.py"]
        DEGRAD["degradations.py"]
        TRANSFORMS["transforms.py"]
        RGB1["data_RGB.py"]
        RGB2["dataset_RGB.py"]
    end

    subgraph "Core Model (LIEDNet)"
        BASE_MODEL["base_model.py"]
        LIED_MODEL["liednet_model.py"]
        ARCH_UTIL["arch_util.py"]
        LIED_ARCH["lied_arch.py"]
        VSSM["csm_triton.py"]
        CSMS6S["csms6s.py"]
        VMAMBA["vmamba.py"]
        VMAMBA_CHECKS["vmamba_checks.py"]
        ZERODCE["zerodce_arch.py"]
        VGG["vgg_arch.py"]
        DISCR["discriminator_arch.py"]
    end

    subgraph "Custom Operations (CUDA)"
        DCN["deform_conv.py"]
        FUSED["fused_act.py"]
        UPFIRD["upfirdn2d.py"]
    end

    subgraph "Training Engine"
        LR_SCHED["lr_scheduler.py"]
        WARMUP_RUN["run.py"]
        WARMUP_SCHED["scheduler.py"]
        LOSSES["losses.py"]
        LOSS_UTIL["loss_util.py"]
    end

    subgraph "Metrics & Evaluation"
        PSNR_SSIM["psnr_ssim.py"]
        METRIC_UTIL["metric_util.py"]
    end

    subgraph "Utilities"
        DIST["dist_util.py"]
        LOGGER["logger.py"]
        FILE_CLIENT["file_client.py"]
        IMG_UTIL["img_util.py"]
        LMDB_UTIL["lmdb_util.py"]
        DOWNLOAD["download_util.py"]
        REGISTRY["registry.py"]
        MISC["misc.py"]
        MATLAB["matlab_functions.py"]
        REAL["realesrgan_utils.py"]
    end

    CYL_STORAGE["Disk I/O"]:::cylinder
    GPU["GPU/CUDA"]:::cylinder

    %% Flows
    TRAIN_LLIE --> CONFIG
    TRAIN_LOL --> CONFIG
    INF_LLIE --> CONFIG
    INF_LOL --> CONFIG
    EVAL_PAIR --> CONFIG
    EVAL_UNPAIR --> CONFIG

    CONFIG --> DATASET1
    CONFIG --> DATASET2
    CONFIG --> BASE_MODEL
    CONFIG --> LR_SCHED

    DATASET1 --> DATALOADER
    DATASET2 --> DATALOADER
    DATALOADER --> BASE_MODEL

    BASE_MODEL --> LIED_MODEL
    LIED_MODEL --> LIED_ARCH
    LIED_MODEL --> ARCH_UTIL
    LIED_ARCH --> VSSM
    LIED_ARCH --> CSMS6S
    LIED_ARCH --> VMAMBA
    LIED_ARCH --> VMAMBA_CHECKS
    LIED_ARCH --> ZERODCE
    LIED_ARCH --> VGG
    LIED_ARCH --> DISCR

    LIED_MODEL --> DCN
    LIED_MODEL --> FUSED
    LIED_MODEL --> UPFIRD

    BASE_MODEL --> LOSSES
    LOSSES --> LOSS_UTIL
    LOSSES --> LR_SCHED

    LR_SCHED --> WARMUP_RUN
    WARMUP_RUN --> WARMUP_SCHED

    BASE_MODEL --> PSNR_SSIM
    BASE_MODEL --> METRIC_UTIL

    BASE_MODEL --> CYL_STORAGE
    PSNR_SSIM --> CYL_STORAGE
    METRIC_UTIL --> CYL_STORAGE

    LIED_MODEL --> GPU
    DCN --> GPU
    FUSED --> GPU
    UPFIRD --> GPU
    WARMUP_RUN --> GPU

    %% Utilities usage
    DIST --> TRAIN_LLIE
    DIST --> BASE_MODEL
    LOGGER --> TRAIN_LLIE
    LOGGER --> BASE_MODEL
    FILE_CLIENT --> DATASET1
    IMG_UTIL --> DATASET1
    LMDB_UTIL --> DATASET1
    DOWNLOAD --> BASE_MODEL
    REGISTRY --> BASE_MODEL
    MISC --> BASE_MODEL
    MATLAB --> METRIC_UTIL
    REAL --> BASE_MODEL

    %% Click Events
    click TRAIN_LLIE "https://github.com/mingyuliu1/liednet/blob/main/train_llie.py"
    click TRAIN_LOL "https://github.com/mingyuliu1/liednet/blob/main/train_lolblur.py"
    click INF_LLIE "https://github.com/mingyuliu1/liednet/blob/main/inference_llie.py"
    click INF_LOL "https://github.com/mingyuliu1/liednet/blob/main/inference_llol_blur.py"
    click EVAL_PAIR "https://github.com/mingyuliu1/liednet/blob/main/calculate_pair.py"
    click EVAL_UNPAIR "https://github.com/mingyuliu1/liednet/blob/main/calculate_unpair.py"
    click CONFIG "https://github.com/mingyuliu1/liednet/blob/main/basicsr/utils/options.py"
    click DATASET1 "https://github.com/mingyuliu1/liednet/blob/main/basicsr/data/lol_image_dataset.py"
    click DATASET2 "https://github.com/mingyuliu1/liednet/blob/main/basicsr/data/paired_image_dataset.py"
    click SAMPLER "https://github.com/mingyuliu1/liednet/blob/main/basicsr/data/data_sampler.py"
    click DATALOADER "https://github.com/mingyuliu1/liednet/blob/main/basicsr/data/prefetch_dataloader.py"
    click DEGRAD "https://github.com/mingyuliu1/liednet/blob/main/basicsr/data/degradations.py"
    click TRANSFORMS "https://github.com/mingyuliu1/liednet/blob/main/basicsr/data/transforms.py"
    click RGB1 "https://github.com/mingyuliu1/liednet/blob/main/basicsr/transform/data_RGB.py"
    click RGB2 "https://github.com/mingyuliu1/liednet/blob/main/basicsr/transform/dataset_RGB.py"
    click BASE_MODEL "https://github.com/mingyuliu1/liednet/blob/main/basicsr/models/base_model.py"
    click LIED_MODEL "https://github.com/mingyuliu1/liednet/blob/main/basicsr/models/liednet_model.py"
    click ARCH_UTIL "https://github.com/mingyuliu1/liednet/blob/main/basicsr/archs/arch_util.py"
    click LIED_ARCH "https://github.com/mingyuliu1/liednet/blob/main/basicsr/archs/lied_arch.py"
    click VSSM "https://github.com/mingyuliu1/liednet/blob/main/basicsr/archs/csm_triton.py"
    click CSMS6S "https://github.com/mingyuliu1/liednet/blob/main/basicsr/archs/csms6s.py"
    click VMAMBA "https://github.com/mingyuliu1/liednet/blob/main/basicsr/archs/vmamba.py"
    click VMAMBA_CHECKS "https://github.com/mingyuliu1/liednet/blob/main/basicsr/archs/vmamba_checks.py"
    click ZERODCE "https://github.com/mingyuliu1/liednet/blob/main/basicsr/archs/zerodce_arch.py"
    click VGG "https://github.com/mingyuliu1/liednet/blob/main/basicsr/archs/vgg_arch.py"
    click DISCR "https://github.com/mingyuliu1/liednet/blob/main/basicsr/archs/discriminator_arch.py"
    click DCN "https://github.com/mingyuliu1/liednet/blob/main/basicsr/ops/dcn/deform_conv.py"
    click FUSED "https://github.com/mingyuliu1/liednet/blob/main/basicsr/ops/fused_act/fused_act.py"
    click UPFIRD "https://github.com/mingyuliu1/liednet/blob/main/basicsr/ops/upfirdn2d/upfirdn2d.py"
    click LR_SCHED "https://github.com/mingyuliu1/liednet/blob/main/basicsr/models/lr_scheduler.py"
    click WARMUP_RUN "https://github.com/mingyuliu1/liednet/blob/main/basicsr/warmup_scheduler/run.py"
    click WARMUP_SCHED "https://github.com/mingyuliu1/liednet/blob/main/basicsr/warmup_scheduler/scheduler.py"
    click LOSSES "https://github.com/mingyuliu1/liednet/blob/main/basicsr/losses/losses.py"
    click LOSS_UTIL "https://github.com/mingyuliu1/liednet/blob/main/basicsr/losses/loss_util.py"
    click PSNR_SSIM "https://github.com/mingyuliu1/liednet/blob/main/basicsr/metrics/psnr_ssim.py"
    click METRIC_UTIL "https://github.com/mingyuliu1/liednet/blob/main/basicsr/metrics/metric_util.py"
    click DIST "https://github.com/mingyuliu1/liednet/blob/main/basicsr/utils/dist_util.py"
    click LOGGER "https://github.com/mingyuliu1/liednet/blob/main/basicsr/utils/logger.py"
    click FILE_CLIENT "https://github.com/mingyuliu1/liednet/blob/main/basicsr/utils/file_client.py"
    click IMG_UTIL "https://github.com/mingyuliu1/liednet/blob/main/basicsr/utils/img_util.py"
    click LMDB_UTIL "https://github.com/mingyuliu1/liednet/blob/main/basicsr/utils/lmdb_util.py"
    click DOWNLOAD "https://github.com/mingyuliu1/liednet/blob/main/basicsr/utils/download_util.py"
    click REGISTRY "https://github.com/mingyuliu1/liednet/blob/main/basicsr/utils/registry.py"
    click MISC "https://github.com/mingyuliu1/liednet/blob/main/basicsr/utils/misc.py"
    click MATLAB "https://github.com/mingyuliu1/liednet/blob/main/basicsr/utils/matlab_functions.py"
    click REAL "https://github.com/mingyuliu1/liednet/blob/main/basicsr/utils/realesrgan_utils.py"

    %% Styles
    classDef cli fill:#ffe680,stroke:#333
    classDef data fill:#bbdefb,stroke:#333
    classDef model fill:#c8e6c9,stroke:#333
    classDef ops fill:#a5d6a7,stroke:#333
    classDef train fill:#ffe0b2,stroke:#333
    classDef utils fill:#e0e0e0,stroke:#333
    classDef cylinder fill:#ffecb3,stroke:#333

    class TRAIN_LLIE,TRAIN_LOL,INF_LLIE,INF_LOL,EVAL_PAIR,EVAL_UNPAIR cli
    class DATASET1,DATASET2,SAMPLER,DATALOADER,DEGRAD,TRANSFORMS,RGB1,RGB2,PSNR_SSIM,METRIC_UTIL data
    class BASE_MODEL,LIED_MODEL,ARCH_UTIL,LIED_ARCH,VSSM,CSMS6S,VMAMBA,VMAMBA_CHECKS,ZERODCE,VGG,DISCR model
    class DCN,FUSED,UPFIRD ops
    class LR_SCHED,WARMUP_RUN,WARMUP_SCHED,LOSSES,LOSS_UTIL train
    class CONFIG,DIST,LOGGER,FILE_CLIENT,IMG_UTIL,LMDB_UTIL,DOWNLOAD,REGISTRY,MISC,MATLAB,REAL utils
    class CYL_STORAGE,GPU cylinder
```
