# numz/comfyui-seedvr2_videoupscaler architecture

> Architecture diagram of the GitHub repository numz/comfyui-seedvr2_videoupscaler, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/numz/comfyui-seedvr2_videoupscaler
- Repository: https://github.com/numz/comfyui-seedvr2_videoupscaler

## Overview

##### Project Type and Purpose
- This is an open-source ComfyUI custom-node plugin and standalone CLI tool for high-quality video/image upscaling using the SeedVR2 diffusion models.
- It integrates directly into ComfyUI as a “SeedVR2 Video Upscaler” node, and can also run independently via `inference_cli.py` with multi-GPU support.

##### File Structure Overview
- Top level
- configs_3b, configs_7b: YAML presets for 3B/7B models
- inference_cli.py: CLI entry point for standalone mode
- neg_emb.pt / pos_emb.pt: precomputed embeddings
- requirements.txt, LICENSE, README.md, docs/
- src/
- common/: shared utilities (config loading, caching, diffusion samplers/schedules, distributed helpers, logging, seeding)
- core/: orchestration of generation and inference (model_manager, generation.py, infer.py)
- data/: image/video transforms pipeline
- interfaces/: ComfyUI node wrapper (`comfyui_node.py`)
- models/: model definitions for DIT and video-VAE modules (v1, v2, variants)
- optimization/: BlockSwap, memory manager, performance tweaks

##### README Insights
- Describes two modes: ComfyUI node vs. standalone CLI
- Highlights requirements (PyTorch, optional flash_attn/triton), multi-GPU batching, memory optimizations (BlockSwap, preserve_vram)
- Details node parameters and CLI flags

a. Main Components
1. ComfyUI Frontend (Node Menu)
2. CLI Frontend (`inference_cli.py`)
3. Configuration Loader (src/common/config.py + YAML)
4. Model Manager (src/core/model_manager.py)
5. Diffusion Engine
- Samplers (Euler, etc.)
- Schedules / Timesteps
- Core model definitions (src/models/…)
6. Data Pipeline (src/data/image/transforms + video frame loader)
7. Optimization Layer (BlockSwap, Memory Manager)
8. Distributed Runner (multi-GPU orchestration)
9. Logging & Debug (src/common/logger.py)
10. Output Writer (video/png export)
b. Relationships & Data Flow
- Frontend (UI or CLI) → reads user parameters → passes to Config Loader
- Config Loader → yields model settings + sampler/timestep settings → Model Manager
- Model Manager → loads weights into PyTorch modules → hands off to Diffusion Engine
- Data Pipeline → batches frames, applies transforms → feeds Diffusion Engine
- Diffusion Engine ↔ Optimization Layer (BlockSwap, VRAM management)
- Diffusion Engine → generates upscaled frames → Output Writer
- Distributed Runner orchestrates parallel model/data shards across GPUs
- Logger collects metrics/stats, Debug hooks can be enabled
c. Architectural Patterns & Principles
- Plugin architecture (ComfyUI interface wrapping core inference)
- Config-driven design (YAML + code defaults)
- Modular diffusion pipeline (sampler/schedule abstraction)
- Separation of concerns: data, models, optimization, orchestration, interface
- Extensibility points: new samplers, model variants, transforms
d. Key Technologies & Frameworks
- Python 3.12, PyTorch 2.x
- ComfyUI Node API
- flash_attn / triton (optional performance libs)
- Multi-GPU using torch.distributed / custom `common/distributed`
- YAML for configs, safetensors weights

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Frontends"
        UI["ComfyUI Frontend Node"]:::frontend
        CLI["CLI Frontend"]:::frontend
    end

    subgraph "Configuration"
        YAML["YAML Presets"]:::core
        Config["Config Loader"]:::core
        DiffConfig["Diffusion Config"]:::core
    end

    subgraph "Model & Diffusion"
        MM["Model Manager"]:::core
        subgraph "Diffusion Engine"
            Samplers["Samplers"]:::core
            Schedules["Schedules"]:::core
            Timesteps["Timesteps"]:::core
            UtilsDiff["Diffusion Utils"]:::core
        end
    end

    subgraph "Core Inference"
        Generation["Generation Orchestrator"]:::core
        Infer["Inference Runner"]:::core
    end

    subgraph "Data Pipeline"
        Transforms["Image/Video Transforms"]:::core
    end

    subgraph "Optimization & Distributed"
        BS["BlockSwap"]:::opt
        Mem["Memory Manager"]:::opt
        Perf["Performance Tweaks"]:::opt
        Compat["Compatibility Fixes"]:::opt
        Dist["Distributed Runner"]:::opt
    end

    subgraph "Model Definitions"
        DIT1["DIT v1"]:::core
        DIT2["DIT v2"]:::core
        VAE3["Video VAE v3"]:::core
        AltVAE["Alternate VAE"]:::core
    end

    subgraph "Utilities"
        Logger["Logging & Debug"]:::util
        Cache["Cache & Helpers"]:::util
        Color["Color Fix & Downloads"]:::util
    end

    UI -->|"params"| Config
    CLI -->|"params"| Config
    YAML -->|"load presets"| Config
    Config -->|"diff config"| DiffConfig
    Config -->|"model settings"| MM
    MM -->|"load weights"| DIT1
    MM --> DIT2
    MM --> VAE3
    MM --> AltVAE
    MM -->|"init engine"| Samplers
    MM --> Schedules
    MM --> Timesteps
    MM --> UtilsDiff
    Transforms -->|"batches frames"| Generation
    Samplers -->|"sample"| Generation
    Schedules -->|"schedule"| Generation
    Timesteps -->|"timesteps"| Generation
    UtilsDiff -->|"utils"| Generation
    Generation -->|"generate frames"| Infer
    Generation --> BS
    Generation --> Mem
    Generation --> Perf
    Generation --> Compat
    Dist -->|"orchestrate GPUs"| Generation
    Dist --> Transforms
    Generation --> Logger
    Infer --> Logger
    Cache --> Logger
    Color --> Logger

    click UI "https://github.com/numz/comfyui-seedvr2_videoupscaler/blob/main/src/interfaces/comfyui_node.py"
    click CLI "https://github.com/numz/comfyui-seedvr2_videoupscaler/blob/main/inference_cli.py"
    click Config "https://github.com/numz/comfyui-seedvr2_videoupscaler/blob/main/src/common/config.py"
    click YAML "https://github.com/numz/comfyui-seedvr2_videoupscaler/blob/main/configs_3b/main.yaml"
    click YAML "https://github.com/numz/comfyui-seedvr2_videoupscaler/blob/main/configs_7b/main.yaml"
    click DiffConfig "https://github.com/numz/comfyui-seedvr2_videoupscaler/blob/main/src/common/diffusion/config.py"
    click MM "https://github.com/numz/comfyui-seedvr2_videoupscaler/blob/main/src/core/model_manager.py"
    click Samplers "https://github.com/numz/comfyui-seedvr2_videoupscaler/tree/main/src/common/diffusion/samplers"
    click Schedules "https://github.com/numz/comfyui-seedvr2_videoupscaler/tree/main/src/common/diffusion/schedules"
    click Timesteps "https://github.com/numz/comfyui-seedvr2_videoupscaler/tree/main/src/common/diffusion/timesteps"
    click UtilsDiff "https://github.com/numz/comfyui-seedvr2_videoupscaler/blob/main/src/common/diffusion/utils.py"
    click Generation "https://github.com/numz/comfyui-seedvr2_videoupscaler/blob/main/src/core/generation.py"
    click Infer "https://github.com/numz/comfyui-seedvr2_videoupscaler/blob/main/src/core/infer.py"
    click Transforms "https://github.com/numz/comfyui-seedvr2_videoupscaler/tree/main/src/data/image/transforms"
    click BS "https://github.com/numz/comfyui-seedvr2_videoupscaler/blob/main/src/optimization/blockswap.py"
    click Mem "https://github.com/numz/comfyui-seedvr2_videoupscaler/blob/main/src/optimization/memory_manager.py"
    click Perf "https://github.com/numz/comfyui-seedvr2_videoupscaler/blob/main/src/optimization/performance.py"
    click Compat "https://github.com/numz/comfyui-seedvr2_videoupscaler/blob/main/src/optimization/compatibility.py"
    click Dist "https://github.com/numz/comfyui-seedvr2_videoupscaler/tree/main/src/common/distributed"
    click Logger "https://github.com/numz/comfyui-seedvr2_videoupscaler/blob/main/src/common/logger.py"
    click Cache "https://github.com/numz/comfyui-seedvr2_videoupscaler/blob/main/src/common/cache.py"
    click Color "https://github.com/numz/comfyui-seedvr2_videoupscaler/blob/main/src/utils/color_fix.py"
    click Color "https://github.com/numz/comfyui-seedvr2_videoupscaler/blob/main/src/utils/downloads.py"

    classDef frontend fill:#AED6F1,stroke:#21618C,color:#1B4F72;
    classDef core fill:#ABEBC6,stroke:#1D8348,color:#145A32;
    classDef opt fill:#F9E79F,stroke:#B7950B,color:#7D6608;
    classDef util fill:#D5D8DC,stroke:#626567,color:#17202A;
```
