# nkolkin13/strotss architecture

> Architecture diagram of the GitHub repository nkolkin13/strotss, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nkolkin13/strotss
- Repository: https://github.com/nkolkin13/strotss

## Overview

##### Project type and purpose
- This is an open-source research tool for performing neural style transfer (“Style Transfer by Relaxed Optimal Transport and Self-Similarity”).
- It implements the STROTSS algorithm on the command line using Python and PyTorch, with optional spatial guidance masks.

##### File structure
Top-level files (all Python scripts) with no subdirectories:
- styleTransfer.py         – entry point; parses arguments and orchestrates the pipeline
- contextual_loss.py       – computes contextual loss between feature maps
- pyr_lap.py               – builds a Laplacian image pyramid for multi‐scale processing
- stylize_objectives.py    – defines style and content objectives per scale
- vgg_pt.py                – wraps a pretrained VGG network for feature extraction
- st_helper.py             – helper routines (e.g., mask alignment, image I/O)
- utils.py                 – generic utilities (e.g., image normalization, logging)
- contextual_loss.py, pyr_lap.py, stylize_objectives.py, vgg_pt.py are the core algorithmic modules.

##### README for additional insights
- Dependencies: Python3, PyTorch, NumPy, ImageIO.
- Usage patterns: unconstrained vs. spatially guided style transfer.
- Mentions VGG feature normalization, multi‐scale loop, content vs. style weight, guidance masks.

a. Main components
1. Command-Line Interface (styleTransfer.py)
2. Preprocessing Module (st_helper.py, utils.py)
– Load content/style images; normalize with ImageNet statistics; load guidance masks
3. Feature Extraction (vgg_pt.py)
– Forward content and style images (and masks) through selected VGG layers
4. Loss Computation
– Contextual Loss (contextual_loss.py)
– Style / Content Objectives (stylize_objectives.py)
5. Multi‐Scale Optimization Controller (styleTransfer.py with pyr_lap.py)
– Build image pyramids; iterate scales; perform gradient‐based optimization (PyTorch autograd / LBFGS)
6. Output Renderer (utils.py, imageio)
– Upsample and save stylized image per scale; final output ‘output.png’
7. External Libraries
– PyTorch, NumPy, ImageIO

b. Relationships and interactions
– The CLI parses input paths and options → calls Preprocessing → passes data to Feature Extraction → feeds feature maps to Loss Computation → Loss values drive gradient updates in the Optimization Controller → produces intermediate and final images via the Output Renderer.
– ply_lap (pyramidal decomposition) sits between Loss Computation and Optimization Controller.
– Utility modules support all stages (I/O, normalization, mask handling).

c. Architectural patterns and principles
– Pipeline architecture: a linear flow of data from image load to final save.
– Modularity: clearly separated concerns (I/O, feature extraction, loss, optimization).

d. Key technologies
– PyTorch (autograd, neural nets, optimizers)
– ImageIO (reading/writing images)
– NumPy (data handling)
– Pretrained VGG (feature backbone)

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Subgraphs
    subgraph "Core Algorithm Modules"
        direction TB
        CLI["CLI\n(styleTransfer.py)"]:::core
        Pre["Preprocessing\n(st_helper.py, utils.py)"]:::util
        FE["Feature Extraction\n(vgg_pt.py)"]:::core
        subgraph "Loss Computation"
            direction TB
            LC1["Contextual Loss\n(contextual_loss.py)"]:::core
            LC2["Style & Content Objectives\n(stylize_objectives.py)"]:::core
        end
        Opt["Optimizer Loop\n(styleTransfer.py)"]:::core
        Pyr["Pyramidal Decomposition\n(pyr_lap.py)"]:::core
        Renderer["Output Renderer\n(utils.py, ImageIO)"]:::util
    end

    External["External Libraries\n(PyTorch, NumPy, ImageIO)"]:::ext

    %% Data Flow
    CLI -->|"parse args"| Pre
    Pre -->|"load & normalize images\nload masks (optional)"| FE
    FE -->|"extract features"| LC1
    FE -->|"extract features"| LC2
    LC1 -->|"compute loss"| Opt
    LC2 -->|"compute loss"| Opt
    Opt -->|"build pyramid"| Pyr
    Pyr -->|"up/downsample images"| Opt
    Opt -->|"generate stylized output"| Renderer

    %% External libs integration
    External --> FE
    External --> LC1
    External --> LC2
    External --> Opt
    External --> Renderer

    %% Click Events
    click CLI "https://github.com/nkolkin13/strotss/blob/master/styleTransfer.py"
    click Pre "https://github.com/nkolkin13/strotss/blob/master/st_helper.py"
    click Pre "https://github.com/nkolkin13/strotss/blob/master/utils.py"
    click FE "https://github.com/nkolkin13/strotss/blob/master/vgg_pt.py"
    click LC1 "https://github.com/nkolkin13/strotss/blob/master/contextual_loss.py"
    click LC2 "https://github.com/nkolkin13/strotss/blob/master/stylize_objectives.py"
    click Opt "https://github.com/nkolkin13/strotss/blob/master/styleTransfer.py"
    click Pyr "https://github.com/nkolkin13/strotss/blob/master/pyr_lap.py"
    click Renderer "https://github.com/nkolkin13/strotss/blob/master/utils.py"

    %% Styles
    classDef core fill:#D0E6FF,stroke:#0366d6,color:#0366d6,stroke-width:1px;
    classDef util fill:#E6FFE6,stroke:#28a745,color:#28a745,stroke-width:1px;
    classDef ext fill:#FFF5E6,stroke:#d9822b,color:#d9822b,stroke-width:1px;
```
