# vinthony/deep-blind-watermark-removal architecture

> Architecture diagram of the GitHub repository vinthony/deep-blind-watermark-removal, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/vinthony/deep-blind-watermark-removal
- Repository: https://github.com/vinthony/deep-blind-watermark-removal

## Overview

##### Project Type and Purpose
- This is a research‐oriented deep learning pipeline for blind single‐image visible watermark removal, implementing a two‐stage neural network (SplitNet + RefineNet) as described in the AAAI 2021 paper.
- The goal is to detect, remove, and recover watermarks in images via end‐to‐end training.

##### File Structure Analysis
- Top‐level scripts:
- main.py / options.py: entry point for training and testing; CLI parsing.
- test.py, examples/*.sh: evaluation and inference wrappers.
- requirements.txt: Python dependencies (PyTorch, torchvision, etc.).
- scripts/
- datasets/: dataset‐specific loader classes (BIH, COCO).
- machines/: “Machine” classes orchestrate training and evaluation loops (BasicMachine, S2AM, VX).
- models/: core network definitions (backbone_unet, rasc (SplitNet), sa_resunet (RefineNet), discriminator, etc.).
- utils/: logging, transformation pipelines, loss functions, metrics, parallelization helpers.
- Other artifacts: watermark_synthesis.ipynb for data synthesis, pre‐trained model download links in README.

##### Key README Insights
- Two‐stage framework:
a. SplitNet: multi‐task network for detection, removal, recovery.
b. RefineNet: refines and smooths predictions from SplitNet.
- Datasets hosted externally (Hugging Face).
- Pre‐trained weights and online Colab demo available.

##### Main Components
a. Data Layer
– External Dataset Repositories (Hugging Face)
– scripts/datasets DataLoader classes
– Preprocessing & Augmentation (scripts/utils/transforms.py, imutils.py)
b. Training Orchestration
– CLI Interface (main.py + options.py)
– Machine classes (scripts/machines/*Machine.py) managing loops, checkpointing, logging
c. Model Layer
– SplitNet (scripts/models/rasc.py + backbone_unet + blocks)
– RefineNet (scripts/models/sa_resunet.py + related blocks)
– Auxiliary nets (discriminator.py for adversarial loss, vgg.py for perceptual loss)
d. Utility Services
– Logging & Monitoring (scripts/utils/logger.py)
– Parallel & GPU management (scripts/utils/parallel.py)
– Evaluation & Metrics (scripts/utils/evaluation.py)
– Checkpoint & Model Initialization (scripts/utils/model_init.py)
e. Output Layer
– Inference (test.py, examples/test.sh)
– Saved checkpoints & result images
f. External Services
– Google Colab demo (online inference)
– Storage for pre‐trained models (Google Drive links)

##### Relationships and Data Flow
- Data ingestion flows from External Datasets → scripts/datasets loaders → preprocessing utils → batch tensors.
- Batches feed into SplitNet; its outputs (mask, coarse background) feed into RefineNet.
- Loss components (reconstruction, adversarial, perceptual) computed via utils and backpropagated.
- Machine class orchestrates optimizer steps, checkpointing, and logging.
- At inference, test.py invokes the same pipeline without backward passes, outputs final images.

##### Architectural Patterns and Principles
- Modular separation: data, models, training loops, and utilities are clearly decoupled.
- Two‐stage cascade pattern: processing pipeline with intermediate outputs.
- Use of strategy pattern in Machine classes to support multiple model variants (BasicMachine, S2AM, VX).
- Reusability: utils and dataset loaders are generic enough to plug in different datasets or models.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Data Layer
    subgraph "Data Layer" 
        direction TB
        ExtData["External Datasets<br/>(Hugging Face)"]:::external
        BIH["BIH Loader"]:::data
        COCO["COCO Loader"]:::data
        Transforms["Preprocessing & Augmentation<br/>(transforms.py)"]:::data
        Imutils["Image Utils<br/>(imutils.py)"]:::data
    end

    %% Training Orchestration
    subgraph "Training Orchestration" 
        direction TB
        CLI["CLI Interface<br/>(main.py + options.py)"]:::orchestration
        Machines["Machine Classes"]:::orchestration
        BM["BasicMachine"]:::orchestration
        S2AM["S2AM"]:::orchestration
        VX["VX"]:::orchestration
    end

    %% Model Layer
    subgraph "Model Layer"
        direction TB
        SplitNet["SplitNet<br/>(rasc.py + backbone_unet.py + blocks.py)"]:::model
        RefineNet["RefineNet<br/>(sa_resunet.py + blocks.py)"]:::model
        Discriminator["Adversarial Discriminator<br/>(discriminator.py)"]:::model
        VGG["Perceptual VGG<br/>(vgg.py)"]:::model
    end

    %% Utility Services
    subgraph "Utility Services"
        direction TB
        Logger["Logging & Monitoring<br/>(logger.py)"]:::utils
        Parallel["Parallel & GPU Management<br/>(parallel.py)"]:::utils
        Eval["Evaluation & Metrics<br/>(evaluation.py)"]:::utils
        ModelInit["Checkpoint & Model Init<br/>(model_init.py)"]:::utils
    end

    %% Output Layer
    subgraph "Output Layer"
        direction TB
        TestPy["Inference Wrapper<br/>(test.py)"]:::output
        TestSh["Shell Script<br/>(examples/test.sh)"]:::output
        Checkpoints["Saved Checkpoints & Results"]:::output
    end

    %% External Services
    subgraph "External Services"
        direction TB
        Colab["Google Colab Demo"]:::external
        Pretrained["Pre-trained Models Storage"]:::external
        Docs["Documentation & Links<br/>(README.md)"]:::external
    end

    %% Data Flow
    ExtData -->|download| BIH
    ExtData -->|download| COCO
    BIH -->|raw images| Transforms
    COCO -->|raw images| Transforms
    Transforms -->|augmented tensors| SplitNet
    Imutils -->|helper ops| Transforms

    %% Two-Stage Pipeline
    SplitNet -->|mask, coarse bg| RefineNet
    RefineNet -->|refined images| Checkpoints

    %% Training Loop
    CLI -->|start training| Machines
    Machines -->|load data| BIH
    Machines -->|load data| COCO
    Machines -->|forward/backward| SplitNet
    Machines -->|forward/backward| RefineNet
    Machines -->|compute loss| Discriminator
    Machines -->|compute loss| VGG
    Machines -->|optimize| SplitNet
    Machines -->|optimize| RefineNet
    Machines -->|checkpoint| ModelInit
    Machines -->|log metrics| Logger
    Machines -->|parallelize| Parallel
    Machines -->|evaluate| Eval
    Machines -->|loop| SplitNet

    %% Inference Flow
    TestPy -->|loads model| Checkpoints
    TestPy -->|runs pipeline| SplitNet
    TestSh -->|executes TestPy| TestPy

    %% External Integrations
    Colab -->|runs pipeline| TestPy
    Pretrained -->|provides weights| ModelInit
    Docs -->|links to| ExtData
    Docs -->|links to| Pretrained

    %% Click Events
    click BIH "https://github.com/vinthony/deep-blind-watermark-removal/blob/main/scripts/datasets/BIH.py"
    click COCO "https://github.com/vinthony/deep-blind-watermark-removal/blob/main/scripts/datasets/COCO.py"
    click Transforms "https://github.com/vinthony/deep-blind-watermark-removal/blob/main/scripts/utils/transforms.py"
    click Imutils "https://github.com/vinthony/deep-blind-watermark-removal/blob/main/scripts/utils/imutils.py"
    click CLI "https://github.com/vinthony/deep-blind-watermark-removal/blob/main/main.py"
    click CLI "https://github.com/vinthony/deep-blind-watermark-removal/blob/main/options.py"
    click BM "https://github.com/vinthony/deep-blind-watermark-removal/blob/main/scripts/machines/BasicMachine.py"
    click S2AM "https://github.com/vinthony/deep-blind-watermark-removal/blob/main/scripts/machines/S2AM.py"
    click VX "https://github.com/vinthony/deep-blind-watermark-removal/blob/main/scripts/machines/VX.py"
    click SplitNet "https://github.com/vinthony/deep-blind-watermark-removal/blob/main/scripts/models/rasc.py"
    click SplitNet "https://github.com/vinthony/deep-blind-watermark-removal/blob/main/scripts/models/backbone_unet.py"
    click SplitNet "https://github.com/vinthony/deep-blind-watermark-removal/blob/main/scripts/models/blocks.py"
    click RefineNet "https://github.com/vinthony/deep-blind-watermark-removal/blob/main/scripts/models/sa_resunet.py"
    click RefineNet "https://github.com/vinthony/deep-blind-watermark-removal/blob/main/scripts/models/blocks.py"
    click Discriminator "https://github.com/vinthony/deep-blind-watermark-removal/blob/main/scripts/models/discriminator.py"
    click VGG "https://github.com/vinthony/deep-blind-watermark-removal/blob/main/scripts/models/vgg.py"
    click Logger "https://github.com/vinthony/deep-blind-watermark-removal/blob/main/scripts/utils/logger.py"
    click Parallel "https://github.com/vinthony/deep-blind-watermark-removal/blob/main/scripts/utils/parallel.py"
    click Eval "https://github.com/vinthony/deep-blind-watermark-removal/blob/main/scripts/utils/evaluation.py"
    click ModelInit "https://github.com/vinthony/deep-blind-watermark-removal/blob/main/scripts/utils/model_init.py"
    click TestPy "https://github.com/vinthony/deep-blind-watermark-removal/blob/main/test.py"
    click TestSh "https://github.com/vinthony/deep-blind-watermark-removal/blob/main/examples/test.sh"
    click Docs "https://github.com/vinthony/deep-blind-watermark-removal/blob/main/README.md"

    %% Styles
    classDef data fill:#CCE5FF,stroke:#004085,color:#004085
    classDef model fill:#D4EDDA,stroke:#155724,color:#155724
    classDef orchestration fill:#FFF3CD,stroke:#856404,color:#856404
    classDef utils fill:#E2E3E5,stroke:#818182,color:#818182
    classDef output fill:#F8D7DA,stroke:#721C24,color:#721C24
    classDef external fill:#D1ECF1,stroke:#0C5460,color:#0C5460
```
