How yuhangzhoucjy/wformer works
Project type and purpose
• Project type: Deep‑learning research codebase with training and inference pipelines.
• Purpose: Embed and extract watermarks in images robustly under various noise/distortion layers using a transformer‑based architecture and adversarial training (generator vs. discriminator).
File structure
• Top‑level scripts:
• train.py / train_settings.json → orchestrates training.
• test.py, test_noise.py / test_settings.json → run inference/robustness evaluation.
• Core module: transformer_watermarking/
• encoder.py, decoder.py, encoder_decoder.py → watermark embedding & extraction logic.
• Discriminator.py → adversarial discriminator network.
• Network.py → overall model wrapper.
• Noise.py & noise_layers/ → various simulated attacks (JPEG, crop, brightness, etc.).
• blocks/ → building blocks (ConvNet, ExpandNet, SENet) used in encoder/decoder.
• Utilities: utils/ (Dataloader, settings loaders, save_images).
• Results folder: stores trained model checkpoints and output images.
README insights
• Confirms transformer‑based watermarking approach.
• Emphasis on robustness against multiple noise/distortion pipelines.
Main components
a. Data Ingestion & Preprocessing
– Raw images in → Dataloader (WFormer/utils/Dataloader.py).
– Settings loaded (train/test JSON).
b. Watermark Embedding (Generator)
– Encoder module: takes cover image + secret bits → embedded image.
– Fusion block (transformer layers + Conv/SENet/ExpandNet blocks).
c. Noise Simulation Pipeline
– Composed of multiple noise_layers (JPEG, crop, Gaussian, etc.).
– Applied to embedded image to simulate attacks.
d. Watermark Extraction (Decoder)
– Decoder module: recovers secret bits from attacked image.
e. Discriminator (Adversarial)
– Takes embedded image (or attacked image?) → distinguishes real vs. watermarked.
– Provides adversarial loss to generator.
f. Training Orchestration (train.py)
– Iterates: encode→noise→decode & discriminate → compute losses → backprop.
– Uses settings from train_settings.json.
g. Inference/Evaluation (test.py, test_noise.py)
– Runs encode + noise layers + decode.
– Saves images & metrics via utils/save_images.
h. Storage
– Model checkpoints (.pth in results/.../models).
– Output images in results/.../images.
Relationships & data flow
• Data loader → Generator (encoder) → Embedded image.
• Embedded image → Noise pipeline → Noised image.
• Noised image → Decoder → Recovered watermark.
• Embedded image (or noised) → Discriminator → Real vs. fake score.
• Losses from decoder & discriminator → back to Generator.
• Scripts (train.py/test.py) orchestrate flows and leverage utils.
Architectural patterns & design principles
• Modular separation: encoder, decoder, discriminator, noise layers.
• Plugin‑style noise modules for extensibility.
• Adversarial training loop (GAN‑like).
• Config‑driven via JSON settings.
• Clear separation of training vs. inference scripts.