# lihaoyun6/comfyui-seedvr2_videoupscaler architecture

> Architecture diagram of the GitHub repository lihaoyun6/comfyui-seedvr2_videoupscaler, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-05-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/lihaoyun6/comfyui-seedvr2_videoupscaler
- Repository: https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler

## Overview

1. **What this repository is**
   This is a ComfyUI custom node package for SeedVR2, with a second execution path for standalone CLI inference. Both modes ultimately run the same internal upscale pipeline; the difference is only the frontend and how inputs/outputs are wired.

2. **Main entry points**
   The two user-facing entry points are `src/interfaces/comfyui_node.py` for ComfyUI integration and `inference_cli.py` for command-line batch processing. The README also ships example ComfyUI workflows for image and video upscaling, which show the intended graph-level integration.

3. **Core orchestration layer**
   `src/core/generation.py`, `src/core/infer.py`, and `src/core/model_manager.py` are the main control plane. They handle model selection, lazy loading/unloading, inference execution, and output assembly. This is the layer that ties together preprocessing, model execution, memory policy, and final export.

4. **Input flow and preprocessing**
   The data path begins with video or image frames from either ComfyUI or the CLI. Frame handling and spatial normalization live under `src/data/image/transforms`, including resize and crop utilities. These transforms prepare frames to the model’s expected dimensions and aspect-ratio rules before inference.

5. **Model families**
   The repository contains three major model subsystems: SeedVR2 DiT models in `src/models/dit`, a newer variant in `src/models/dit_v2`, and the video VAE stack in `src/models/video_vae_v3`. The DiT modules implement the diffusion transformer backbone; the VAE modules handle video latent encoding/decoding; and the two DiT trees suggest an evolution path or compatibility split between model generations.

6. **Diffusion runtime**
   Shared diffusion logic lives in `src/common/diffusion`, including samplers, schedules, and timestep utilities. This layer is model-agnostic and provides the numerical machinery used during generation, while `src/common/diffusion/config.py` and related types keep the runtime configuration separated from model definitions.

7. **Memory and performance boundary**
   `src/optimization` is a dedicated subsystem for low-VRAM execution and performance tuning. It includes BlockSwap support, memory management, GGUF dequantization/ops, compatibility helpers, and performance utilities. This is the main boundary where the repo trades speed for memory savings, especially important because the README emphasizes high VRAM usage and optional CPU offload.

8. **Distributed and multi-GPU support**
   `src/common/distributed` contains the multi-device execution helpers used by standalone mode. The CLI supports a comma-separated CUDA device list, so the pipeline can shard work across GPUs. This is a separate execution path from ComfyUI’s typical single-process node flow.

9. **Model acquisition and registry**
   `src/utils/downloads.py` and `src/utils/model_registry.py` support automatic model discovery and fetching from external model sources, primarily Hugging Face as described in the README. The repo also includes local config files for 3B and 7B variants, plus embedded prompt/negative embeddings (`pos_emb.pt`, `neg_emb.pt`), so model selection is driven by local registry/config rather than hardcoded paths.

10. **Common infrastructure**
   The `src/common` package holds reusable plumbing: config parsing, caching, logging, seeding, decorators, partitioning, and half-precision fixes. This is the shared foundation used by both the ComfyUI node and the CLI to keep behavior consistent across execution modes.

11. **ComfyUI workflow integration**
   The example JSON workflows show the repo is meant to be embedded in a node graph with optional BlockSwap configuration. The output boundary is therefore node-to-node rather than service-to-service: frames enter the SeedVR2 node, are processed internally, and return as upscaled video frames or image sequences for downstream ComfyUI nodes.

12. **Output path**
   The pipeline can emit either video or PNG sequences, as shown by the standalone CLI options. The final stage is responsible for reassembling processed frames into the requested format and writing to the chosen output path, making export format a configurable terminal boundary rather than part of the model core.

13. **Operational characteristics**
   The architecture is deliberately GPU-centric and memory constrained. The README’s warnings, batch-size guidance, and BlockSwap options reflect the main runtime reality: inference is dominated by VRAM management, not just compute. That makes memory policy an architectural concern, not just an optimization detail.

14. **Repository shape for diagramming**
   If you draw this architecture, the cleanest split is: user entry points on the left, orchestration in the middle, shared runtime and model families below, and optimization/distributed support as side-car subsystems. External dependencies to show are ComfyUI, CUDA/PyTorch, local model files, and Hugging Face for automatic model download.

Main entry points: [`src/core/generation.py`](https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler/blob/nightly/src/core/generation.py) (Generation), [`src/core/infer.py`](https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler/blob/nightly/src/core/infer.py) (Infer), [`src/core/model_manager.py`](https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler/blob/nightly/src/core/model_manager.py) (Model manager), [`src/interfaces/comfyui_node.py`](https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler/blob/nightly/src/interfaces/comfyui_node.py) (ComfyUI node)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["Entry points"]
  node_comfyui_node["ComfyUI node<br/>interface<br/>[comfyui_node.py]"]
  node_inference_cli["CLI runner<br/>interface<br/>[inference_cli.py]"]
end

subgraph group_orchestration["Orchestration"]
  node_generation["Generation<br/>orchestrator<br/>[generation.py]"]
  node_infer["Infer<br/>executor<br/>[infer.py]"]
  node_model_manager[("Model manager<br/>lifecycle<br/>[model_manager.py]")]
  node_img_transforms["Image transforms<br/>preprocess"]
end

subgraph group_runtime["Runtime & models"]
  node_diffusion_cfg["Diffusion config<br/>runtime config<br/>[config.py]"]
  node_diffusion_utils["Diffusion utils<br/>diffusion runtime"]
  node_dit_v1{{"DiT<br/>model backbone"}}
  node_dit_v2{{"DiT v2<br/>model backbone"}}
  node_video_vae{{"Video VAE<br/>latent codec"}}
end

subgraph group_support["Support systems"]
  node_blockswap["BlockSwap<br/>memory policy<br/>[blockswap.py]"]
  node_memory_manager["Memory manager<br/>memory control<br/>[memory_manager.py]"]
  node_distributed["Distributed<br/>multi-GPU"]
  node_registry[("Model registry<br/>acquisition<br/>[model_registry.py]")]
  node_downloads["Downloads<br/>acquisition<br/>[downloads.py]"]
  node_config_3b["3B config<br/>model config<br/>[main.yaml]"]
  node_config_7b["7B config<br/>model config<br/>[main.yaml]"]
  node_examples["Example workflows<br/>integration examples"]
end

node_comfyui_node -->|"invokes"| node_generation
node_inference_cli -->|"invokes"| node_generation
node_generation -->|"preprocess"| node_img_transforms
node_generation -->|"selects model"| node_model_manager
node_generation -->|"runs"| node_infer
node_generation -->|"applies memory policy"| node_blockswap
node_generation -->|"shards"| node_distributed
node_infer -->|"uses"| node_diffusion_utils
node_infer -->|"reads config"| node_diffusion_cfg
node_infer -->|"executes"| node_dit_v1
node_infer -->|"executes"| node_dit_v2
node_infer -->|"encodes/decodes"| node_video_vae
node_model_manager -->|"resolves"| node_registry
node_model_manager -->|"fetches"| node_downloads
node_model_manager -->|"loads"| node_config_3b
node_model_manager -->|"loads"| node_config_7b
node_registry -->|"requests"| node_downloads
node_examples -.->|"documents"| node_comfyui_node
node_blockswap -->|"coordinates"| node_memory_manager
node_distributed -->|"supports"| node_infer

click node_comfyui_node "https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler/blob/nightly/src/interfaces/comfyui_node.py"
click node_inference_cli "https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler/blob/nightly/inference_cli.py"
click node_generation "https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler/blob/nightly/src/core/generation.py"
click node_infer "https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler/blob/nightly/src/core/infer.py"
click node_model_manager "https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler/blob/nightly/src/core/model_manager.py"
click node_img_transforms "https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler/tree/nightly/src/data/image/transforms"
click node_diffusion_cfg "https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler/blob/nightly/src/common/diffusion/config.py"
click node_diffusion_utils "https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler/tree/nightly/src/common/diffusion"
click node_dit_v1 "https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler/tree/nightly/src/models/dit"
click node_dit_v2 "https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler/tree/nightly/src/models/dit_v2"
click node_video_vae "https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler/tree/nightly/src/models/video_vae_v3"
click node_blockswap "https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler/blob/nightly/src/optimization/blockswap.py"
click node_memory_manager "https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler/blob/nightly/src/optimization/memory_manager.py"
click node_distributed "https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler/tree/nightly/src/common/distributed"
click node_registry "https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler/blob/nightly/src/utils/model_registry.py"
click node_downloads "https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler/blob/nightly/src/utils/downloads.py"
click node_config_3b "https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler/blob/nightly/configs_3b/main.yaml"
click node_config_7b "https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler/blob/nightly/configs_7b/main.yaml"
click node_examples "https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler/tree/nightly/example_workflows"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_comfyui_node,node_inference_cli toneBlue
class node_generation,node_infer,node_model_manager,node_img_transforms toneAmber
class node_diffusion_cfg,node_diffusion_utils,node_dit_v1,node_dit_v2,node_video_vae toneMint
class node_blockswap,node_memory_manager,node_distributed,node_registry,node_downloads,node_config_3b,node_config_7b,node_examples toneRose
```

## Components

### Entry points

User-facing ways to run the pipeline

- [ComfyUI node](https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler/blob/nightly/src/interfaces/comfyui_node.py): interface · ComfyUI custom node entry point · `src/interfaces/comfyui_node.py`
- [CLI runner](https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler/blob/nightly/inference_cli.py): interface · Standalone batch inference entry point · `inference_cli.py`

### Orchestration

Control plane that prepares inputs, runs inference, and assembles outputs

- [Generation](https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler/blob/nightly/src/core/generation.py): orchestrator · Top-level pipeline orchestration and output assembly · `src/core/generation.py`
- [Infer](https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler/blob/nightly/src/core/infer.py): executor · Inference execution and runtime coordination · `src/core/infer.py`
- [Model manager](https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler/blob/nightly/src/core/model_manager.py): lifecycle · Model selection, lazy loading, and unloading · `src/core/model_manager.py`
- [Image transforms](https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler/tree/nightly/src/data/image/transforms): preprocess · Resize and crop utilities for frame normalization · `src/data/image/transforms`

### Runtime & models

Shared diffusion runtime and model families

- [Diffusion config](https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler/blob/nightly/src/common/diffusion/config.py): runtime config · Shared diffusion runtime configuration and types · `src/common/diffusion/config.py`
- [Diffusion utils](https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler/tree/nightly/src/common/diffusion): diffusion runtime · Samplers, schedules, timestep utilities, and shared diffusion helpers · `src/common/diffusion`
- [DiT](https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler/tree/nightly/src/models/dit): model backbone · SeedVR2 diffusion transformer backbone · `src/models/dit`
- [DiT v2](https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler/tree/nightly/src/models/dit_v2): model backbone · Updated diffusion transformer variant · `src/models/dit_v2`
- [Video VAE](https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler/tree/nightly/src/models/video_vae_v3): latent codec · Video latent encoder/decoder stack · `src/models/video_vae_v3`

### Support systems

Memory, distribution, registry, and example integration assets

- [BlockSwap](https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler/blob/nightly/src/optimization/blockswap.py): memory policy · Low-VRAM execution and module swapping · `src/optimization/blockswap.py`
- [Memory manager](https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler/blob/nightly/src/optimization/memory_manager.py): memory control · VRAM management and allocation helpers · `src/optimization/memory_manager.py`
- [Distributed](https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler/tree/nightly/src/common/distributed): multi-GPU · Multi-device execution helpers for standalone mode · `src/common/distributed`
- [Model registry](https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler/blob/nightly/src/utils/model_registry.py): acquisition · Model discovery and download coordination · `src/utils/model_registry.py`
- [Downloads](https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler/blob/nightly/src/utils/downloads.py): acquisition · External model fetching helpers · `src/utils/downloads.py`
- [3B config](https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler/blob/nightly/configs_3b/main.yaml): model config · Configuration for the smaller model variant · `configs_3b/main.yaml`
- [7B config](https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler/blob/nightly/configs_7b/main.yaml): model config · Configuration for the larger model variant · `configs_7b/main.yaml`
- [Example workflows](https://github.com/lihaoyun6/comfyui-seedvr2_videoupscaler/tree/nightly/example_workflows): integration examples · ComfyUI example graphs for image and video upscaling · `example_workflows`

## Connections

- ComfyUI node → Generation: invokes · ComfyUI node delegates execution into the shared pipeline
- CLI runner → Generation: invokes · CLI uses the same orchestration path as the node frontend
- Generation → Image transforms: preprocess · Frames are normalized before model execution
- Generation → Model manager: selects model · Orchestration asks for the appropriate model variant
- Generation → Infer: runs · Generation coordinates the actual inference pass
- Generation → BlockSwap: applies memory policy · Low-VRAM behavior is wired into the pipeline
- Generation → Distributed: shards · Standalone mode can spread work across GPUs
- Infer → Diffusion utils: uses · Inference depends on shared samplers, schedules, and timestep logic
- Infer → Diffusion config: reads config · Runtime parameters are separated from model definitions
- Infer → DiT: executes · One supported backbone family
- Infer → DiT v2: executes · Alternative backbone family
- Infer → Video VAE: encodes/decodes · Video latent codec participates in the pipeline
- Model manager → Model registry: resolves · Model lookup is driven by registry metadata
- Model manager → Downloads: fetches · Missing models can be acquired automatically
- Model manager → 3B config: loads · One model family configuration
- Model manager → 7B config: loads · Another model family configuration
- Model registry → Downloads: requests · Registry-backed acquisition can trigger downloads
- Example workflows → ComfyUI node: documents · Example workflows show how the custom node is embedded
- BlockSwap → Memory manager: coordinates · Block swapping relies on shared memory controls
- Distributed → Infer: supports · Multi-GPU helpers extend the standalone inference path
