# modeltc/lightx2v architecture

> Architecture diagram of the GitHub repository modeltc/lightx2v, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/modeltc/lightx2v
- Repository: https://github.com/modeltc/lightx2v

## Overview

##### Project Type and Purpose
- LightX2V is a full-stack, modular video-generation inference framework. It provides:
- A Python-based service/API layer for orchestrating requests
- A core inference engine (lightx2v Python package) that loads models, runs schedulers/runners, manages offloading, quantization, caching, and parallelism
- A high-performance CUDA/C++ kernel library (lightx2v_kernel) for custom operators (attention, GEMM, quant kernels)
- Multiple frontends: Gradio demo, ComfyUI node interface, Windows one-click, and a service deployment mode
- Configuration-driven architecture (configs/* folder) to select models, offload strategies, caching, quantization, etc.

##### Key Directories and Architectural Patterns
- Top-level:
- app/ – frontend entrypoints (Gradio scripts & wrappers)
- lightx2v/ – core Python inference package
– server/ – API, task manager, GPU manager, service loops
– common/ops, offload, modules – plugin registry for operators/offload managers
– models/ – input encoders, video encoders, network definitions, runners, schedulers, VFI
– utils/ – async I/O, config loading, profiling, prompts
- lightx2v_kernel/ – CMake project, CUDA kernels, Python bindings
- configs/ – JSON-based definitions for every model/strategy combination
- scripts/ – shell wrappers for benchmarks, caching tests, deployments
- docs/ – Sphinx documentation for tutorials, deploy guides
- Dockerfile(s), CI workflows, pyproject.toml, requirements – build and dev setup

- Architectural Patterns:
- Plugin pattern for operators and offload managers (registry_factory)
- Scheduler–Runner separation: scheduler decides timesteps/blocks; runner executes transformer passes
- Three-tier offloading (disk–CPU–GPU) and caching sub-systems
- Modular service API that can spin up multiple workers or a multi-server farm (api_multi_servers.py)
- Config-driven microservices: each model/strategy is selected by passing a JSON config

a. Frontend Layer (User Interfaces)
– Gradio App (app/gradio_demo.py)
– ComfyUI Nodes (via deploy_comfyui guide)
– Windows One-Click Script
– CLI Scripts (scripts/server/start_server.sh, etc.)
b. API / Service Layer (lightx2v.server)
– HTTP/gRPC endpoints → request router
– TaskManager & GPUManager
– Schema validation
– Distributed service entrypoint (api_multi_servers.py)
c. Inference Engine (lightx2v.*)
– ConfigLoader → loads JSON from configs/
– ModelLoader → HF checkpoints or quantized binaries
– Scheduler → generates denoise timesteps or caching schedules
– Runner → orchestrates encoder→transformer→decoder for each step
– TransformerInfer → dispatches to CPU/GPU/offload/trt or custom ops (flash, sage, radial)
– Feature Caching & Parallel Inference sub-systems
– Video Frame Interpolation (RIFE module)
d. Operator Kernel Layer (lightx2v_kernel)
– Custom CUDA kernels for quantized GEMM, attention, NV/FP4
– Python bindings for ops used by transformer_infer
e. Storage & Offload Layer
– In-memory caching
– Disk offload manager (phase/block/disk)
f. External Dependencies & Services
– HuggingFace model hub
– Docker Engine (for containerized runs)
– Optional DB or object store if extended for multi-tenant
– ComfyUI or Gradio frameworks

##### Relationships and Data Flow
1. User submits Text/Image via Frontend → HTTP POST or CLI
2. API Layer authenticates, assigns TaskID, selects GPU
3. ConfigLoader reads relevant JSON → builds scheduler & runner
4. Runner passes input through InputEncoder → video latents
5. Scheduler loops over timesteps:
– TransformerInfer fetches parameters (may offload others)
– Custom ops executed via lightx2v_kernel
– FeatureCache writes/reads intermediate caches
6. After denoising, outputs decoded via VideoEncoder → frames
7. (Optional) Frame interpolation via RIFE module
8. Output video saved, returned to user or uploaded

- Blue for Frontend
- Green for Service/API
- Orange for Inference Engine
- Red for Kernel Layer
- Brown for Storage/Offload
- Gray for External Services

##### Design Principles to Highlight
- Modular plugin-based operators and offload managers
- Config-driven behavior enabling dozens of model/strategy combos
- Separation of concerns: encoding, scheduling, inference, decoding, I/O
- High-performance path: custom ops → CUDA kernels → step-distilled loop
- Scalability: multi-server API layer for distributed inference

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Frontend Layer
    subgraph "Frontend Layer"
        direction TB
        Gradio["Gradio App"]:::frontend
        WinClick["Windows One-Click"]:::frontend
        CLI_Scripts["CLI Server Scripts"]:::frontend
        ComfyUI["ComfyUI Node Interface"]:::frontend
    end

    %% API / Service Layer
    subgraph "Service/API Layer"
        direction TB
        API_Multi["Multi-server Entrypoint"]:::api
        API_Single["Single-server Entrypoint"]:::api
        ServiceLoop["Main Service Loop & HTTP/gRPC"]:::api
        TaskGPU["TaskManager & GPUManager"]:::api
        SchemaConfig["Schema Validation & Config"]:::api
    end

    %% Inference Engine
    subgraph "Inference Engine" 
        direction TB
        CLI_Infer["Core CLI Inference Script"]:::inference
        ConfigLoader["ConfigLoader & Utils"]:::inference
        Registry["Plugin Registry Factory"]:::inference
        OffloadMgr["Offload Manager"]:::inference
        TransformerInfer["TransformerInfer Core"]:::inference
        InputEnc["Input Encoders"]:::inference
        VideoEnc["Video Encoders"]:::inference
        Scheduler["Schedulers"]:::inference
        Runner["Runners"]:::inference
        FeatureCache["Feature Caching Subsystem"]:::inference
        RIFE["Video Frame Interpolation (RIFE)"]:::inference
    end

    %% Kernel Layer
    subgraph "Kernel Layer"
        direction TB
        CUDA_Kernels["C++/CUDA Kernels"]:::kernel
        PyBind["Python Bindings"]:::kernel
        KernelDocs["Kernel Documentation"]:::kernel
    end

    %% Storage & Offload Layer
    subgraph "Storage & Offload"
        direction TB
        InMemory["In-Memory Cache"]:::storage
        DiskOffload["Disk Offload Manager"]:::storage
    end

    %% Configuration Files
    Configs["/configs/**/*.json"]:::config

    %% Docker & CI
    subgraph "Build & CI"
        Dockerfiles["Docker & CI Setup"]:::external
        BenchScripts["Benchmarks & Deployment Scripts"]:::external
        Docs["Documentation & Tutorials"]:::external
    end

    %% External Services
    subgraph "External Dependencies"
        HFHub["HuggingFace Model Hub"]:::external
        DockerEngine["Docker Engine"]:::external
        DBStore["Optional DB/Object Store"]:::external
        GradioFW["Gradio/ComfyUI Frameworks"]:::external
    end

    %% Data Flows
    %% User to Frontend
    Gradio -->|"HTTP POST"/CLI| API_Single
    WinClick -->|launch| Gradio
    CLI_Scripts -->|start| API_Single
    ComfyUI -->|node call| API_Single

    %% Service Flow
    API_Single -->|delegate| ServiceLoop
    API_Multi -->|delegate| ServiceLoop
    ServiceLoop -->|validate| SchemaConfig
    ServiceLoop -->|manage| TaskGPU
    ServiceLoop -->|load config| ConfigLoader

    %% Inference Flow
    ConfigLoader -->|load JSON| Configs
    ConfigLoader -->|init| Registry
    Registry -->|provide ops| TransformerInfer
    ServiceLoop -->|invoke| CLI_Infer
    CLI_Infer -->|setup| ConfigLoader
    CLI_Infer -->|start loop| Scheduler
    Scheduler -->|next step| Runner
    Runner -->|encode| InputEnc
    Runner -->|infer| TransformerInfer
    TransformerInfer -->|call| CUDA_Kernels
    TransformerInfer -->|via PyBind11| PyBind
    TransformerInfer -->|cache ops| FeatureCache
    FeatureCache -->|read/write| InMemory
    FeatureCache -->|spill| DiskOffload
    DiskOffload -->|load/offload| OffloadMgr
    Runner -->|decode| VideoEnc
    VideoEnc -->|output frames| RIFE
    RIFE -->|interpolate| VideoEnc

    %% Final Output
    VideoEnc -->|return video| ServiceLoop
    ServiceLoop -->|response| GradioFW
    ServiceLoop -->|store| DBStore

    %% External interactions
    ConfigLoader -->|fetch model| HFHub
    Dockerfiles -->|package| DockerEngine

    %% Click Events
    click Gradio "https://github.com/modeltc/lightx2v/blob/main/app/gradio_demo.py"
    click WinClick "https://github.com/modeltc/lightx2v/blob/main/app/run_gradio_win.bat"
    click CLI_Scripts "https://github.com/modeltc/lightx2v/blob/main/scripts/server/start_server.sh"
    click CLI_Scripts "https://github.com/modeltc/lightx2v/blob/main/scripts/server/start_multi_servers.sh"
    click ComfyUI "https://github.com/modeltc/lightx2v/blob/main/lightx2v/models/vfi/rife/rife_comfyui_wrapper.py"
    click API_Multi "https://github.com/modeltc/lightx2v/blob/main/lightx2v/api_multi_servers.py"
    click API_Single "https://github.com/modeltc/lightx2v/blob/main/lightx2v/api_server.py"
    click ServiceLoop "https://github.com/modeltc/lightx2v/blob/main/lightx2v/server/main.py"
    click ServiceLoop "https://github.com/modeltc/lightx2v/blob/main/lightx2v/server/service.py"
    click ServiceLoop "https://github.com/modeltc/lightx2v/blob/main/lightx2v/server/api.py"
    click TaskGPU "https://github.com/modeltc/lightx2v/blob/main/lightx2v/server/task_manager.py"
    click TaskGPU "https://github.com/modeltc/lightx2v/blob/main/lightx2v/server/gpu_manager.py"
    click SchemaConfig "https://github.com/modeltc/lightx2v/blob/main/lightx2v/server/schema.py"
    click SchemaConfig "https://github.com/modeltc/lightx2v/blob/main/lightx2v/server/config.py"
    click SchemaConfig "https://github.com/modeltc/lightx2v/blob/main/lightx2v/utils/set_config.py"
    click CLI_Infer "https://github.com/modeltc/lightx2v/blob/main/lightx2v/infer.py"
    click ConfigLoader "https://github.com/modeltc/lightx2v/blob/main/lightx2v/utils/set_config.py"
    click ConfigLoader "https://github.com/modeltc/lightx2v/blob/main/lightx2v/utils/service_utils.py"
    click Registry "https://github.com/modeltc/lightx2v/blob/main/lightx2v/utils/registry_factory.py"
    click OffloadMgr "https://github.com/modeltc/lightx2v/blob/main/lightx2v/common/offload/manager.py"
    click TransformerInfer "https://github.com/modeltc/lightx2v/blob/main/lightx2v/common/transformer_infer/transformer_infer.py"
    click InputEnc "https://github.com/modeltc/lightx2v/tree/main/lightx2v/models/input_encoders/"
    click VideoEnc "https://github.com/modeltc/lightx2v/tree/main/lightx2v/models/video_encoders/"
    click Scheduler "https://github.com/modeltc/lightx2v/tree/main/lightx2v/models/schedulers/"
    click Runner "https://github.com/modeltc/lightx2v/tree/main/lightx2v/models/runners/"
    click FeatureCache "https://github.com/modeltc/lightx2v/tree/main/lightx2v/models/networks/hunyuan/infer/feature_caching"
    click FeatureCache "https://github.com/modeltc/lightx2v/tree/main/lightx2v/models/schedulers/hunyuan/feature_caching"
    click RIFE "https://github.com/modeltc/lightx2v/tree/main/lightx2v/models/vfi/rife/"
    click CUDA_Kernels "https://github.com/modeltc/lightx2v/tree/main/lightx2v_kernel/csrc/"
    click CUDA_Kernels "https://github.com/modeltc/lightx2v/blob/main/lightx2v_kernel/CMakeLists.txt"
    click PyBind "https://github.com/modeltc/lightx2v/tree/main/lightx2v_kernel/python/lightx2v_kernel/"
    click KernelDocs "https://github.com/modeltc/lightx2v/tree/main/lightx2v_kernel/docs/"
    click DiskOffload "https://github.com/modeltc/lightx2v/blob/main/lightx2v/common/offload/manager.py"
    click DiskOffload "https://github.com/modeltc/lightx2v/tree/main/lightx2v/models/networks//infer/offload/"
    click Configs "https://github.com/modeltc/lightx2v/tree/main/configs/"
    click Dockerfiles "https://github.com/modeltc/lightx2v/tree/main/Dockerfile"
    click Dockerfiles "https://github.com/modeltc/lightx2v/tree/main/Dockerfile_cu124"
    click Dockerfiles "https://github.com/modeltc/lightx2v/blob/main/.github/workflows/lint.yml"
    click BenchScripts "https://github.com/modeltc/lightx2v/tree/main/scripts/bench/"
    click BenchScripts "https://github.com/modeltc/lightx2v/tree/main/scripts/dist_infer/"
    click BenchScripts "https://github.com/modeltc/lightx2v/tree/main/scripts/cache/"
    click BenchScripts "https://github.com/modeltc/lightx2v/tree/main/scripts/quantization/"
    click BenchScripts "https://github.com/modeltc/lightx2v/tree/main/scripts/server/"
    click Docs "https://github.com/modeltc/lightx2v/tree/main/docs/EN/source/"
    click Docs "https://github.com/modeltc/lightx2v/tree/main/docs/ZH_CN/source/"

    %% Styles
    classDef frontend fill:#E0F7FA,stroke:#006064,color:#004D40;
    classDef api fill:#E8F5E9,stroke:#1B5E20,color:#2E7D32;
    classDef inference fill:#FFF3E0,stroke:#E65100,color:#BF360C;
    classDef kernel fill:#FFEBEE,stroke:#B71C1C,color:#C62828;
    classDef storage fill:#EFEBE9,stroke:#4E342E,color:#3E2723;
    classDef config fill:#F3E5F5,stroke:#6A1B9A,color:#4A148C;
    classDef external fill:#ECEFF1,stroke:#37474F,color:#263238;
```
