How numz/comfyui-seedvr2_videoupscaler works
Project Type and Purpose
• This is an open-source ComfyUI custom-node plugin and standalone CLI tool for high-quality video/image upscaling using the SeedVR2 diffusion models.
• It integrates directly into ComfyUI as a “SeedVR2 Video Upscaler” node, and can also run independently via inference_cli.py with multi-GPU support.
File Structure Overview
• Top level
• configs_3b, configs_7b: YAML presets for 3B/7B models
• inference_cli.py: CLI entry point for standalone mode
• neg_emb.pt / pos_emb.pt: precomputed embeddings
• requirements.txt, LICENSE, README.md, docs/
• src/
• common/: shared utilities (config loading, caching, diffusion samplers/schedules, distributed helpers, logging, seeding)
• core/: orchestration of generation and inference (model_manager, generation.py, infer.py)
• data/: image/video transforms pipeline
• interfaces/: ComfyUI node wrapper (comfyui_node.py)
• models/: model definitions for DIT and video-VAE modules (v1, v2, variants)
• optimization/: BlockSwap, memory manager, performance tweaks
README Insights
• Describes two modes: ComfyUI node vs. standalone CLI
• Highlights requirements (PyTorch, optional flash_attn/triton), multi-GPU batching, memory optimizations (BlockSwap, preserve_vram)
• Details node parameters and CLI flags
a. Main Components
1. ComfyUI Frontend (Node Menu)
2. CLI Frontend (inference_cli.py)
3. Configuration Loader (src/common/config.py + YAML)
4. Model Manager (src/core/model_manager.py)
5. Diffusion Engine
• Samplers (Euler, etc.)
• Schedules / Timesteps
• Core model definitions (src/models/…)
6. Data Pipeline (src/data/image/transforms + video frame loader)
7. Optimization Layer (BlockSwap, Memory Manager)
8. Distributed Runner (multi-GPU orchestration)
9. Logging & Debug (src/common/logger.py)
10. Output Writer (video/png export)
b. Relationships & Data Flow
• Frontend (UI or CLI) → reads user parameters → passes to Config Loader
• Config Loader → yields model settings + sampler/timestep settings → Model Manager
• Model Manager → loads weights into PyTorch modules → hands off to Diffusion Engine
• Data Pipeline → batches frames, applies transforms → feeds Diffusion Engine
• Diffusion Engine ↔ Optimization Layer (BlockSwap, VRAM management)
• Diffusion Engine → generates upscaled frames → Output Writer
• Distributed Runner orchestrates parallel model/data shards across GPUs
• Logger collects metrics/stats, Debug hooks can be enabled
c. Architectural Patterns & Principles
• Plugin architecture (ComfyUI interface wrapping core inference)
• Config-driven design (YAML + code defaults)
• Modular diffusion pipeline (sampler/schedule abstraction)
• Separation of concerns: data, models, optimization, orchestration, interface
• Extensibility points: new samplers, model variants, transforms
d. Key Technologies & Frameworks
• Python 3.12, PyTorch 2.x
• ComfyUI Node API
• flash_attn / triton (optional performance libs)
• Multi-GPU using torch.distributed / custom common/distributed
• YAML for configs, safetensors weights