How nkolkin13/strotss works
Project type and purpose
• This is an open-source research tool for performing neural style transfer (“Style Transfer by Relaxed Optimal Transport and Self-Similarity”).
• It implements the STROTSS algorithm on the command line using Python and PyTorch, with optional spatial guidance masks.
File structure
Top-level files (all Python scripts) with no subdirectories:
• styleTransfer.py – entry point; parses arguments and orchestrates the pipeline
• contextual_loss.py – computes contextual loss between feature maps
• pyr_lap.py – builds a Laplacian image pyramid for multi‐scale processing
• stylize_objectives.py – defines style and content objectives per scale
• vgg_pt.py – wraps a pretrained VGG network for feature extraction
• st_helper.py – helper routines (e.g., mask alignment, image I/O)
• utils.py – generic utilities (e.g., image normalization, logging)
• contextual_loss.py, pyr_lap.py, stylize_objectives.py, vgg_pt.py are the core algorithmic modules.
README for additional insights
• Dependencies: Python3, PyTorch, NumPy, ImageIO.
• Usage patterns: unconstrained vs. spatially guided style transfer.
• Mentions VGG feature normalization, multi‐scale loop, content vs. style weight, guidance masks.
a. Main components
1. Command-Line Interface (styleTransfer.py)
2. Preprocessing Module (st_helper.py, utils.py)
– Load content/style images; normalize with ImageNet statistics; load guidance masks
3. Feature Extraction (vgg_pt.py)
– Forward content and style images (and masks) through selected VGG layers
4. Loss Computation
– Contextual Loss (contextual_loss.py)
– Style / Content Objectives (stylize_objectives.py)
5. Multi‐Scale Optimization Controller (styleTransfer.py with pyr_lap.py)
– Build image pyramids; iterate scales; perform gradient‐based optimization (PyTorch autograd / LBFGS)
6. Output Renderer (utils.py, imageio)
– Upsample and save stylized image per scale; final output ‘output.png’
7. External Libraries
– PyTorch, NumPy, ImageIO
b. Relationships and interactions
– The CLI parses input paths and options → calls Preprocessing → passes data to Feature Extraction → feeds feature maps to Loss Computation → Loss values drive gradient updates in the Optimization Controller → produces intermediate and final images via the Output Renderer.
– ply_lap (pyramidal decomposition) sits between Loss Computation and Optimization Controller.
– Utility modules support all stages (I/O, normalization, mask handling).
c. Architectural patterns and principles
– Pipeline architecture: a linear flow of data from image load to final save.
– Modularity: clearly separated concerns (I/O, feature extraction, loss, optimization).
d. Key technologies
– PyTorch (autograd, neural nets, optimizers)
– ImageIO (reading/writing images)
– NumPy (data handling)
– Pretrained VGG (feature backbone)