# mingcv/dsit architecture

> Architecture diagram of the GitHub repository mingcv/dsit, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mingcv/dsit
- Repository: https://github.com/mingcv/dsit

## Overview

##### Project Type and Purpose
- DSIT is a research-oriented deep-learning pipeline for Single Image Reflection Separation, based on an Interactive Dual-Stream Transformer.
- It provides data preparation, model architectures, training, evaluation, and testing scripts—no frontend/UI; it’s purely a Python-based ML framework.

##### File Structure
- Top-level scripts:
- `train_sirs_setting1.py` / `train_sirs_setting2.py` – entry points for training under two configurations.
- `eval_sirs.py` – evaluation entry point.
- `test_sirs.py` – testing/inference entry point.
- Core modules:
- `data/` – dataset classes (`sirs_dataset.py`, `image_folder.py`), text files listing images, and transform definitions.
- `models/` – model definitions:
– `arch/` holds specific architectures (`dsit.py`, `swin_det.py`).
– `base_model.py`, `networks.py`, `vgg.py`, plus concrete wrappers like `dsit_model_sirs_lrm.py`.
– `losses.py` – loss functions.
- `engine.py` – training/evaluation loop implementation.
- `options.py` – command-line argument parsing.
- `tools/` – utilities for checkpoint saving (`saver.py`), miscellaneous utilities (`mutils.py`).
- `util/` – logging, HTML report generation (`html.py`, `index.py`), scheduler factories, visualization helpers.
- Config & dependencies:
- `requirements.txt`
- `assets/` for logos and figures.

##### README
- Confirms environment (Python 3.9 + PyTorch + dependencies).
- Summarizes datasets (VOC, real-world paired sets, SIR²).
- Illustrates high-level network architecture (dual-stream transformer).
- Shows usage patterns (train/eval/test CLI commands).
- Points out pretrained backbone (Swin Transformer).

##### Main Components and Relationships
A. CLI Layer
– `train_sirs_*.py`, `eval_sirs.py`, `test_sirs.py`
– Invokes `options.py` → parses arguments → passes config to Engine.
B. Data Layer
– `data/` module: Dataset classes, transforms
– Preprocessing pipeline: transforms → DataLoader
C. Model Layer
– `models/arch/`: DSIT core, Swin backbone
– `base_model.py`, `networks.py`, `vgg.py`
– `dsit_model_sirs_lrm.py` binds arch + loss + optional nature branch
D. Training Engine
– `engine.py`: orchestrates forward, backward, optimizer step
– interacts with Model, DataLoader, Loss, Scheduler
E. Loss & Metrics
– `models/losses.py`: reconstruction, perceptual (VGG) losses
F. Utilities
– `tools/saver.py`: checkpoint I/O
– `util/visualizer.py`, `util/html.py`, `util/index.py`: logging & report generation
– `util/schedulers.py`: LR schedulers
G. Output
– Saved model weights (`weights/`)
– HTML reports, tensorboard logs (if used)
– Evaluation metrics and visual comparisons

##### Architectural Patterns & Design Principles
- Modular separation:
- Data ↔ Model ↔ Engine with clear interfaces.
- CLI-driven configuration via `options.py`.
- Extensibility: easily swap architectures in `models/arch/` or datasets in `data/`.
- Reuse of standard libraries (PyTorch, torchvision, timm for Swin).

a. Layout
– Left: Data Layer → DataLoader → Engine.
- calls Model (→ Forward)
- calls Loss → Backprop → Optimizer/Scheduler
– Right: Utilities & Output:
- checkpoint saver, visualizer, HTML report generator
- final weights & evaluation results
CLI ➔ Engine
Data ➔ Engine
Engine ➔ Model
Model & Ground-Truth ➔ Loss ➔ Engine
Engine ➔ Saver/Visualizer ➔ Output
- Blue for core components (Engine, Model, Data)
- Green for utilities (Saver, Visualizer)
- Orange for entry scripts (CLI)
- Datasets on disk
- Pretrained weights
d. Labels
– Annotate key classes/functions (e.g., `DSITModel`, `SIRSDataSet`, `train_one_epoch`).

##### Final Checklist
- [ ] Annotations for key technologies: PyTorch, timm (Swin), torchvision transforms
- [ ] Clear labeling of entry scripts and their parameters

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% CLI Layer
    subgraph "CLI Layer"
        direction TB
        Train1["train_sirs_setting1.py"]:::cli
        Train2["train_sirs_setting2.py"]:::cli
        Eval["eval_sirs.py"]:::cli
        Test["test_sirs.py"]:::cli
        Options["options.py"]:::cli
    end

    %% Data Layer
    subgraph "Data Layer"
        direction TB
        DataInit["data/__init__.py"]:::data
        SIRSDS["sirs_dataset.py"]:::data
        ImgFld["image_folder.py"]:::data
        TorchData["torchdata.py"]:::data
        Transforms["transforms.py"]:::data
        VOCtxt["VOC2012_224_train_png.txt"]:::data
        RealTxt["real_test.txt"]:::data
    end

    %% Model Layer
    subgraph "Model Layer"
        direction TB
        ArchDSIT["dsit.py"]:::model
        ArchSwin["swin_det.py"]:::model
        BaseModel["base_model.py"]:::model
        Networks["networks.py"]:::model
        VGG["vgg.py"]:::model
        DSITModel["dsit_model_sirs_lrm.py"]:::model
    end

    %% Engine and Loss
    Engine["engine.py"]:::engine
    Losses["losses.py"]:::model

    %% Utilities - Checkpoint & Misc
    subgraph "Utilities - Checkpoint & Misc"
        direction TB
        Saver["saver.py"]:::util
        Mutils["mutils.py"]:::util
    end

    %% Utilities - Visualization & Reporting
    subgraph "Utilities - Visualization & Reporting"
        direction TB
        HTML["html.py"]:::util
        Index["index.py"]:::util
        Visualizer["visualizer.py"]:::util
        Schedulers["schedulers.py"]:::util
        NetUtils["net_utils.py"]:::util
        UtilMod["util.py"]:::util
    end

    %% Dependencies and Assets
    Requirements["requirements.txt"]:::external
    Assets["assets/"]:::external

    %% Output
    subgraph "Output"
        direction TB
        Weights["Saved Model Weights"]:::output
        Reports["HTML Reports & Logs"]:::output
    end

    %% Connections
    Train1 --> Options
    Train2 --> Options
    Eval --> Options
    Test --> Options
    Options --> Engine

    DataInit --> Engine
    SIRSDS --> Engine
    ImgFld --> Engine
    TorchData --> Engine
    Transforms --> Engine
    VOCtxt --> Engine
    RealTxt --> Engine

    Engine --> ModelCall["Model Inference & Training"]:::engine
    ModelCall --> ArchDSIT
    ModelCall --> ArchSwin
    ModelCall --> BaseModel
    ModelCall --> Networks
    ModelCall --> VGG
    ModelCall --> DSITModel

    ModelCall --> Losses
    Losses --> Engine

    Engine --> Saver
    Engine --> Mutils

    Engine --> HTML
    Engine --> Index
    Engine --> Visualizer
    Engine --> Schedulers
    Engine --> NetUtils
    Engine --> UtilMod

    Saver --> Weights
    Visualizer --> Reports
    HTML --> Reports
    Index --> Reports

    Requirements -.-> Engine
    Assets -.-> Reports

    %% Click Events
    click Train1 "https://github.com/mingcv/dsit/blob/main/train_sirs_setting1.py"
    click Train2 "https://github.com/mingcv/dsit/blob/main/train_sirs_setting2.py"
    click Eval "https://github.com/mingcv/dsit/blob/main/eval_sirs.py"
    click Test "https://github.com/mingcv/dsit/blob/main/test_sirs.py"
    click Options "https://github.com/mingcv/dsit/blob/main/options.py"
    click DataInit "https://github.com/mingcv/dsit/blob/main/data/__init__.py"
    click SIRSDS "https://github.com/mingcv/dsit/blob/main/data/sirs_dataset.py"
    click ImgFld "https://github.com/mingcv/dsit/blob/main/data/image_folder.py"
    click TorchData "https://github.com/mingcv/dsit/blob/main/data/torchdata.py"
    click Transforms "https://github.com/mingcv/dsit/blob/main/data/transforms.py"
    click VOCtxt "https://github.com/mingcv/dsit/blob/main/data/VOC2012_224_train_png.txt"
    click RealTxt "https://github.com/mingcv/dsit/blob/main/data/real_test.txt"
    click ArchDSIT "https://github.com/mingcv/dsit/blob/main/models/arch/dsit.py"
    click ArchSwin "https://github.com/mingcv/dsit/blob/main/models/arch/swin_det.py"
    click BaseModel "https://github.com/mingcv/dsit/blob/main/models/base_model.py"
    click Networks "https://github.com/mingcv/dsit/blob/main/models/networks.py"
    click VGG "https://github.com/mingcv/dsit/blob/main/models/vgg.py"
    click DSITModel "https://github.com/mingcv/dsit/blob/main/models/dsit_model_sirs_lrm.py"
    click Engine "https://github.com/mingcv/dsit/blob/main/engine.py"
    click Losses "https://github.com/mingcv/dsit/blob/main/models/losses.py"
    click Saver "https://github.com/mingcv/dsit/blob/main/tools/saver.py"
    click Mutils "https://github.com/mingcv/dsit/blob/main/tools/mutils.py"
    click HTML "https://github.com/mingcv/dsit/blob/main/util/html.py"
    click Index "https://github.com/mingcv/dsit/blob/main/util/index.py"
    click Visualizer "https://github.com/mingcv/dsit/blob/main/util/visualizer.py"
    click Schedulers "https://github.com/mingcv/dsit/blob/main/util/schedulers.py"
    click NetUtils "https://github.com/mingcv/dsit/blob/main/util/net_utils.py"
    click UtilMod "https://github.com/mingcv/dsit/blob/main/util/util.py"
    click Requirements "https://github.com/mingcv/dsit/blob/main/requirements.txt"
    click Assets "https://github.com/mingcv/dsit/tree/main/assets/"

    %% Styles
    classDef cli fill:#f9d5e5,stroke:#333,stroke-width:1px;
    classDef data fill:#bde0fe,stroke:#333,stroke-width:1px;
    classDef model fill:#a0c4ff,stroke:#333,stroke-width:1px;
    classDef engine fill:#ffadad,stroke:#333,stroke-width:1px;
    classDef util fill:#caffbf,stroke:#333,stroke-width:1px;
    classDef external fill:#e0e0e0,stroke:#333,stroke-width:1px;
    classDef output fill:#ffd6a5,stroke:#333,stroke-width:1px;
```
