How vinthony/deep-blind-watermark-removal works
Project Type and Purpose
• This is a research‐oriented deep learning pipeline for blind single‐image visible watermark removal, implementing a two‐stage neural network (SplitNet + RefineNet) as described in the AAAI 2021 paper.
• The goal is to detect, remove, and recover watermarks in images via end‐to‐end training.
File Structure Analysis
• Top‐level scripts:
• main.py / options.py: entry point for training and testing; CLI parsing.
• test.py, examples/*.sh: evaluation and inference wrappers.
• requirements.txt: Python dependencies (PyTorch, torchvision, etc.).
• scripts/
• datasets/: dataset‐specific loader classes (BIH, COCO).
• machines/: “Machine” classes orchestrate training and evaluation loops (BasicMachine, S2AM, VX).
• models/: core network definitions (backbone_unet, rasc (SplitNet), sa_resunet (RefineNet), discriminator, etc.).
• utils/: logging, transformation pipelines, loss functions, metrics, parallelization helpers.
• Other artifacts: watermark_synthesis.ipynb for data synthesis, pre‐trained model download links in README.
Key README Insights
• Two‐stage framework:
a. SplitNet: multi‐task network for detection, removal, recovery.
b. RefineNet: refines and smooths predictions from SplitNet.
• Datasets hosted externally (Hugging Face).
• Pre‐trained weights and online Colab demo available.
Main Components
a. Data Layer
– External Dataset Repositories (Hugging Face)
– scripts/datasets DataLoader classes
– Preprocessing & Augmentation (scripts/utils/transforms.py, imutils.py)
b. Training Orchestration
– CLI Interface (main.py + options.py)
– Machine classes (scripts/machines/*Machine.py) managing loops, checkpointing, logging
c. Model Layer
– SplitNet (scripts/models/rasc.py + backbone_unet + blocks)
– RefineNet (scripts/models/sa_resunet.py + related blocks)
– Auxiliary nets (discriminator.py for adversarial loss, vgg.py for perceptual loss)
d. Utility Services
– Logging & Monitoring (scripts/utils/logger.py)
– Parallel & GPU management (scripts/utils/parallel.py)
– Evaluation & Metrics (scripts/utils/evaluation.py)
– Checkpoint & Model Initialization (scripts/utils/model_init.py)
e. Output Layer
– Inference (test.py, examples/test.sh)
– Saved checkpoints & result images
f. External Services
– Google Colab demo (online inference)
– Storage for pre‐trained models (Google Drive links)
Relationships and Data Flow
• Data ingestion flows from External Datasets → scripts/datasets loaders → preprocessing utils → batch tensors.
• Batches feed into SplitNet; its outputs (mask, coarse background) feed into RefineNet.
• Loss components (reconstruction, adversarial, perceptual) computed via utils and backpropagated.
• Machine class orchestrates optimizer steps, checkpointing, and logging.
• At inference, test.py invokes the same pipeline without backward passes, outputs final images.
Architectural Patterns and Principles
• Modular separation: data, models, training loops, and utilities are clearly decoupled.
• Two‐stage cascade pattern: processing pipeline with intermediate outputs.
• Use of strategy pattern in Machine classes to support multiple model variants (BasicMachine, S2AM, VX).
• Reusability: utils and dataset loaders are generic enough to plug in different datasets or models.