# josh00-lu/csbnet architecture

> Architecture diagram of the GitHub repository josh00-lu/csbnet, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/josh00-lu/csbnet
- Repository: https://github.com/josh00-lu/csbnet

## Overview

##### Project Type and Purpose
- This is a research‐oriented deep learning project—specifically, a PyTorch implementation of an arbitrary video style‐transfer network (CSBNet).
- It’s not a web or full‐stack service but a data‐processing/model pipeline: you feed it content images or videos plus style images, it extracts features, runs them through three custom network modules (Crystallization, Separation, Blending), and outputs stylized images/videos.

##### File‐Structure Analysis
Top‐level scripts:
- train.py       ← orchestrates the training loop (data loading, forward/backward pass, loss computation, checkpointing)
- test_image.py  ← runs inference on images
- test_video.py  ← runs inference on videos
Core modules:
- net.py         ← defines the CSBNet architecture (Cr, Sp, Bd modules, encoder/decoder)
- sampler.py     ← handles video‐frame sampling or batch creation
- function.py    ← helper routines: I/O, pre/post‐processing, saving outputs
Models directory:
- models/vgg_normalised.pth   ← pretrained VGG encoder weights
- (optional) csbnet pretrained .pth for quick testing

##### Key Architectural Patterns
- Pipeline pattern: sequential data flow from input → encoder → Cr → Sp → Bd → decoder → output
- Modular NN design: three specialized “blocks” for crystallization, separation, and blending wrapped in one top‐level CSBNet class
- Training vs. inference modes: shared model graph but different entry points and I/O scripts

##### Components
A. Data Ingestion
- Content Loader: reads images or videos (via OpenCV, imageio‐ffmpeg)
- Frame Sampler: splits videos into frames or batches of images
- Style Loader: reads one or multiple style images
B. Feature Extraction
- VGG Encoder (pretrained weights)
C. CSBNet Core Modules (inside net.py)
- Crystallization Module (Cr)
- Separation Module (Sp)
- Blending Module (Bd)
D. Decoder / Reconstruction
- Upsampling blocks to form final stylized frames
E. Loss Computation (during training)
- Content loss, style loss, component enhancement losses
F. Script Orchestrators
- train.py
- test_image.py / test_video.py
G. Persistence & Configuration
- Model checkpoint saving/loading (models/)
- Command‐line args (KC, KS, batch_size, GPU IDs, etc.)
H. External Dependencies
- PyTorch, torchvision, NumPy, SciPy, OpenCV, imageio‐ffmpeg

##### Relationships & Data Flow
1. Start at train.py or test_*.py →
2. Invoke Loader → Sampler
3. Feed batch into VGG Encoder
4. Pass features to Cr → output crystal nuclei
5. Feed nuclei to Sp → separate content/style features
6. Feed separated features into Bd → blended features
7. Decoder reconstructs stylized images/frames
8. (Training only) compute losses → backprop → update CSBNet parameters
9. Write outputs to disk and (if training) save checkpoints

##### Capturing Essential Elements
- That both image and video inference share the same backbone, differing only at the input loader stage.
- Multi‐GPU or batch_size config flowing into train.py.
- Every module from net.py is represented with its role named exactly as in code (Cr, Sp, Bd).

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Script Orchestrators"
        Train["train.py"]:::orchestrator
        TestImg["test_image.py"]:::orchestrator
        TestVid["test_video.py"]:::orchestrator
    end

    Config["Config Params"]:::config

    Train --> Func
    TestImg --> Func
    TestVid --> Func

    Func["Data/Style Loader"]:::io
    Sam["Frame Sampler"]:::io
    Func -->|"raw images/videos"| Sam

    Sam -->|"frames/batches"| VGG
    VGG["VGG Encoder"]:::model

    subgraph "CSBNet Model"
        Cr["Crystallization Module"]:::model
        Sp["Separation Module"]:::model
        Bd["Blending Module"]:::model
        Dec["Decoder/Upsampling"]:::model
    end

    VGG -->|"feature maps"| Cr
    Cr -->|"crystal nuclei"| Sp
    Sp -->|"separated features"| Bd
    Bd -->|"blended features"| Dec
    Dec -->|"stylized frames"| Output["Output"]:::io

    Dec --> Loss
    Loss["Loss Computation"]:::training
    Loss --> Optimizer
    Optimizer["Optimizer & Checkpoint"]:::training

    Config -.-> Train
    Config -.-> TestImg
    Config -.-> TestVid

    click Train "https://github.com/josh00-lu/csbnet/blob/main/train.py"
    click TestImg "https://github.com/josh00-lu/csbnet/blob/main/test_image.py"
    click TestVid "https://github.com/josh00-lu/csbnet/blob/main/test_video.py"
    click Func "https://github.com/josh00-lu/csbnet/blob/main/function.py"
    click Sam "https://github.com/josh00-lu/csbnet/blob/main/sampler.py"
    click VGG "https://github.com/josh00-lu/csbnet/blob/main/net.py"
    click Cr "https://github.com/josh00-lu/csbnet/blob/main/net.py"
    click Sp "https://github.com/josh00-lu/csbnet/blob/main/net.py"
    click Bd "https://github.com/josh00-lu/csbnet/blob/main/net.py"
    click Dec "https://github.com/josh00-lu/csbnet/blob/main/net.py"
    click Optimizer "https://github.com/josh00-lu/csbnet/tree/main/models/"

    classDef orchestrator fill:#cccccc,stroke:#888888
    classDef io fill:#ADD8E6,stroke:#0000CC
    classDef model fill:#9f6,stroke:#067D05
    classDef training fill:#f96,stroke:#D35400
    classDef config fill:none,stroke:#000000,stroke-dasharray:5 5
```
