# lintaopeng/u-shape_transformer_for_underwater_image_enhancement architecture

> Architecture diagram of the GitHub repository lintaopeng/u-shape_transformer_for_underwater_image_enhancement, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/lintaopeng/u-shape_transformer_for_underwater_image_enhancement
- Repository: https://github.com/lintaopeng/u-shape_transformer_for_underwater_image_enhancement

## Overview

##### High‐level file‐structure analysis
- Top‐level:
– `train.ipynb`, `test.ipynb`: Jupyter controllers for training and inference
– `net/`: model definitions and building blocks
– `loss/`: custom loss modules (LAB, LCH)
– `pytorch-ssim-loss/`, `pytorch_ssim/`: third‐party SSIM loss packages
– `test/`: sample input images and ground truth for quick testing
– `requirements.txt`, README.md, LICENSE, scripts to install dependencies
- Model subpackages in `net/`:
– CMSFFT.py (channel‐wise multi‐scale feature fusion transformer)
– SGFMT.py (spatial‐wise global feature modeling transformer)
– Transformer.py, PositionalEncoding.py, block.py, IntmdSequential.py, utils.py

##### README insights
- Describes dataset (LSUI) and data splits, training regime (L2 then L1 loss), environmental requirements (Python, PyTorch, CUDA, OpenCV, Jupyter).
- Mentions external dataset/download sources (BaiduYun, Google Drive).

##### Main components
a. Data Layer
- **LSUI Dataset** (downloaded externally) → local `data/` and `test/` folders
- DataLoader & Preprocessing (utility/data.py)
b. Model Layer
- Submodules: CMSFFT, SGFMT, standard Transformer blocks, PositionalEncoding
c. Loss Layer
d. Training Controller
- **train.ipynb** orchestrates DataLoader → Model → Loss → Optimizer → checkpoint saving (`saved_models/`)
e. Inference Controller
- **test.ipynb** loads checkpoint → DataLoader (test/input) → Model → saves enhanced images (test/output) → PSNR computation
f. Utilities & Visualization
g. Infrastructure & Dependencies
- Python 3.7+, PyTorch ≥1.7, CUDA, OpenCV, Jupyter Notebook, pip (requirements.txt)

##### Relationships and data‐flow
- In **Testing**: Input images → DataLoader → Model inference → enhanced outputs → PSNR evaluation → write to disk

##### Architectural patterns and design principles
- **Modular design**: separate directories for model, loss, utilities
- **Notebook orchestration**: Jupyter notebooks driving end‐to‐end workflows rather than monolithic scripts

##### Essential elements
- Clear, concise labels for every component and module
- Notes on external dependencies (dataset download sources, pip install)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data & Preprocessing
    subgraph "Data & Preprocessing"
        direction TB
        Dataset["LSUI Dataset"]:::data
        SampleTest["Sample Test Data\n(test/input, test/GT)"]:::data
        DataLoader["DataLoader & Preprocessing\n(utility/data.py)"]:::data
    end

    %% Model Architecture
    subgraph "Model Architecture"
        direction TB
        UShape["U-shape Transformer\n(net/Ushape_Trans.py)"]:::model
        CMSFFT["CMSFFT Module\n(net/CMSFFT.py)"]:::model
        SGFMT["SGFMT Module\n(net/SGFMT.py)"]:::model
        TransformerBlock["Transformer Blocks\n(net/Transformer.py)"]:::model
        PosEnc["Positional Encoding\n(net/PositionalEncoding.py)"]:::model
        IntmdSeq["Intermediate Sequential\n(net/IntmdSequential.py)"]:::model
        BlockDef["Block Definitions\n(net/block.py)"]:::model
        NetUtils["Model Utilities\n(net/utils.py)"]:::model
    end

    %% Loss & Metrics
    subgraph "Loss & Metrics"
        direction TB
        LABLoss["LAB Color-space Loss\n(loss/LAB.py)"]:::loss
        LCHLoss["LCH Color-space Loss\n(loss/LCH.py)"]:::loss
        SSIM1["SSIM Loss Package\n(max_ssim.py)"]:::loss
        SSIM2["SSIM Loss Package\n(pytorch_ssim/)"]:::loss
    end

    %% Controllers
    subgraph "Controllers"
        direction TB
        Train["Training Notebook\n(train.ipynb)"]:::controller
        Inference["Inference Notebook\n(test.ipynb)"]:::controller
    end

    %% Utilities & Visualization
    subgraph "Utilities & Visualization"
        direction TB
        Plots["Plotting Module\n(utility/plots.py)"]:::util
        PtColor["Color-space Conversions\n(utility/ptcolor.py)"]:::util
        PtUtils["Metrics & Misc Utils\n(utility/ptutils.py)"]:::util
        Qnt["Quantization & Lab Softmax\n(utility/Qnt.py,\nutility/Relu_Softmax_LAB.py)"]:::util
        Splines["Splines Preprocessing\n(utility/splines.py)"]:::util
    end

    %% Infrastructure & Dependencies
    subgraph "Infrastructure & Diagrams"
        direction TB
        Requirements["requirements.txt"]:::infra
        Readme["README.md"]:::infra
        Figs["Diagrams & Figures\n(figs/)"]:::infra
        Images["Result Snapshots\n(images/)"]:::infra
    end

    %% Training Flow
    Dataset -->|"download/split"| DataLoader
    DataLoader -->|"batch of RGB images"| UShape
    UShape -->|"feature maps"| CMSFFT
    UShape -->|"feature maps"| SGFMT
    CMSFFT --> TransformerBlock
    SGFMT --> TransformerBlock
    TransformerBlock --> IntmdSeq
    IntmdSeq --> PosEnc
    PosEnc --> BlockDef
    BlockDef --> NetUtils
    NetUtils -->|"model output"| UShape
    UShape -->|"predictions"| LossFlow
    subgraph LossFlow["Loss Computation"]
        direction TB
        UShapeOut["Predictions"]:::loss
        GT["Ground Truth"]:::data
        UShapeOut --> LABLoss
        UShapeOut --> LCHLoss
        UShapeOut --> SSIM1
        UShapeOut --> SSIM2
        GT --> LABLoss
        GT --> LCHLoss
        GT --> SSIM1
        GT --> SSIM2
    end
    LossFlow -->|"loss value"| Optimizer["Optimizer"]:::model
    Optimizer -->|"updated weights"| Checkpoint["Checkpoint\n(saved_models/)"]:::model
    Checkpoint --> Train

    %% Inference Flow
    SampleTest -->|"test batch"| DataLoader
    DataLoader --> UShape
    UShape -->|"enhanced images"| Enhanced["Enhanced Outputs"]:::data
    Enhanced --> PSNR["PSNR Evaluation"]:::util
    PSNR -->|"metrics"| Disk["save to disk"]:::data
    Disk --> Inference

    %% Utilities invocations
    Train --> Plots
    Train --> PtColor
    Train --> PtUtils
    Train --> Qnt
    Train --> Splines

    Inference --> PtColor
    Inference --> PtUtils

    %% Infrastructure links
    Train --> Requirements
    Train --> Readme
    Readme --> Figs
    Readme --> Images

    %% Click Events
    click SampleTest "https://github.com/lintaopeng/u-shape_transformer_for_underwater_image_enhancement/tree/main/test/input"
    click SampleTest "https://github.com/lintaopeng/u-shape_transformer_for_underwater_image_enhancement/tree/main/test/GT"
    click DataLoader "https://github.com/lintaopeng/u-shape_transformer_for_underwater_image_enhancement/blob/main/utility/data.py"
    click UShape "https://github.com/lintaopeng/u-shape_transformer_for_underwater_image_enhancement/blob/main/net/Ushape_Trans.py"
    click CMSFFT "https://github.com/lintaopeng/u-shape_transformer_for_underwater_image_enhancement/blob/main/net/CMSFFT.py"
    click SGFMT "https://github.com/lintaopeng/u-shape_transformer_for_underwater_image_enhancement/blob/main/net/SGFMT.py"
    click TransformerBlock "https://github.com/lintaopeng/u-shape_transformer_for_underwater_image_enhancement/blob/main/net/Transformer.py"
    click PosEnc "https://github.com/lintaopeng/u-shape_transformer_for_underwater_image_enhancement/blob/main/net/PositionalEncoding.py"
    click IntmdSeq "https://github.com/lintaopeng/u-shape_transformer_for_underwater_image_enhancement/blob/main/net/IntmdSequential.py"
    click BlockDef "https://github.com/lintaopeng/u-shape_transformer_for_underwater_image_enhancement/blob/main/net/block.py"
    click NetUtils "https://github.com/lintaopeng/u-shape_transformer_for_underwater_image_enhancement/blob/main/net/utils.py"
    click LABLoss "https://github.com/lintaopeng/u-shape_transformer_for_underwater_image_enhancement/blob/main/loss/LAB.py"
    click LCHLoss "https://github.com/lintaopeng/u-shape_transformer_for_underwater_image_enhancement/blob/main/loss/LCH.py"
    click SSIM1 "https://github.com/lintaopeng/u-shape_transformer_for_underwater_image_enhancement/blob/main/pytorch-ssim-loss/max_ssim.py"
    click SSIM2 "https://github.com/lintaopeng/u-shape_transformer_for_underwater_image_enhancement/blob/main/pytorch_ssim/__init__.py"
    click Train "https://github.com/lintaopeng/u-shape_transformer_for_underwater_image_enhancement/blob/main/train.ipynb"
    click Inference "https://github.com/lintaopeng/u-shape_transformer_for_underwater_image_enhancement/blob/main/test.ipynb"
    click Plots "https://github.com/lintaopeng/u-shape_transformer_for_underwater_image_enhancement/blob/main/utility/plots.py"
    click PtColor "https://github.com/lintaopeng/u-shape_transformer_for_underwater_image_enhancement/blob/main/utility/ptcolor.py"
    click PtUtils "https://github.com/lintaopeng/u-shape_transformer_for_underwater_image_enhancement/blob/main/utility/ptutils.py"
    click Qnt "https://github.com/lintaopeng/u-shape_transformer_for_underwater_image_enhancement/blob/main/utility/Qnt.py"
    click Qnt "https://github.com/lintaopeng/u-shape_transformer_for_underwater_image_enhancement/blob/main/utility/Relu_Softmax_LAB.py"
    click Splines "https://github.com/lintaopeng/u-shape_transformer_for_underwater_image_enhancement/blob/main/utility/splines.py"
    click Requirements "https://github.com/lintaopeng/u-shape_transformer_for_underwater_image_enhancement/blob/main/requirements.txt"
    click Readme "https://github.com/lintaopeng/u-shape_transformer_for_underwater_image_enhancement/blob/main/README.md"
    click Figs "https://github.com/lintaopeng/u-shape_transformer_for_underwater_image_enhancement/tree/main/figs/"
    click Images "https://github.com/lintaopeng/u-shape_transformer_for_underwater_image_enhancement/tree/main/images/"

    %% Styles
    classDef data fill:#cce5ff,stroke:#034f84,color:#034f84;
    classDef model fill:#d4edda,stroke:#155724,color:#155724;
    classDef loss fill:#ffe5b4,stroke:#ff8c00,color:#ff8c00;
    classDef controller fill:#fff3cd,stroke:#856404,color:#856404,shape:hexagon;
    classDef util fill:#e2e3e5,stroke:#6c757d,color:#6c757d,shape:parallelogram;
    classDef infra fill:#f0e6f6,stroke:#6f42c1,color:#6f42c1;
```
