# lightricks/comfyui-ltxvideo architecture

> Architecture diagram of the GitHub repository lightricks/comfyui-ltxvideo, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/lightricks/comfyui-ltxvideo
- Repository: https://github.com/lightricks/comfyui-ltxvideo
- Video tour (about a minute): https://gitdiagram.com/lightricks/comfyui-ltxvideo/video

## Overview

##### Project type and purpose
- This is not a web app or microservice—it’s an extension/plugin for ComfyUI, providing a collection of custom “nodes” that integrate the LTXVideo model into the ComfyUI node‐based diffusion pipeline.
- Purpose: augment ComfyUI with video‐diffusion nodes (samplers, guides, upsamplers, decoders, “tricks”) and provide JSON example workflows for common use cases (image→video, frame interpolation, flow/RF edits, multiscale upscaling).

##### File structure
Top level:
- Python modules implementing individual nodes and helper functions:
– decoder_noise.py, easy_samplers.py, latent_adain.py, latent_upsampler.py, latents.py, prompt_enhancer_nodes.py, prompt_enhancer_utils.py, recurrent_sampler.py, tiled_sampler.py, guide.py, q8_nodes.py, stg.py, film_grain.py, nodes_registry.py
- presets/ (pre‐defined parameter sets)
- tricks/ (LTXTricks code integrated here)
- example_workflows/ (JSON flows + sample videos illustrating how to chain nodes)
- config: pyproject.toml, requirements.txt, .pre-commit-config.yaml
Key patterns:
– Plugin architecture: nodes are discovered & registered via nodes_registry.py
– Functional grouping:
- “nodes” implement ComfyUI extensibility points
- “utils” support those nodes (attn_bank, noise_utils, module_utils)
- “tricks” folder contains advanced/custom modules & nodes
- “presets” supply JSON parameter bundles for flows

##### Extract insights from the README
- Confirms ComfyUI integration: custom nodes must be placed in ComfyUI’s custom-nodes folder
- Dependencies: LTXVideo model checkpoint files (in models/checkpoints), optional quantized Q8 kernels, upscaler models (in models/upscale_models), other custom nodes (VideoHelperSuite, etc.)
- Example workflows: image→video, keyframe interpolation, flow/RF edit, multiscale generation
- Notes on new nodes: STGGuiderAdvanced, PromptEnhancer, frame/sequence conditioning

1. ComfyUI Core
– Node loader / graph execution engine
2. ComfyUI-LTXVideo Plugin
- Node Registry (nodes_registry.py)
- Node Definitions (decoder_noise, samplers, guide, latent_upsampler, q8_nodes, stg, etc.)
- Utility Modules (prompt_enhancer_utils, tricks/utils, latents, film_grain)
- Presets (JSONs under presets/)
- Tricks (modules & nodes under tricks/)
3. Example Workflows
– JSON files in example_workflows/ consumed by ComfyUI to instantiate flows
4. External Dependencies
– LTXVideo model checkpoints (models/checkpoints)
– Upscaler models (models/upscale_models)
– ComfyUI-VideoHelperSuite and other custom-node prerequisites
– Hugging Face / GitHub as artifact sources

1. ComfyUI Core → loads Plugin → reads nodes_registry → registers each custom node
2. User selects an Example Workflow JSON → ComfyUI Core parses JSON → instantiates nodes in a directed acyclic graph
3. Node invocations (per‐step in the flow):
- PromptEnhancerNode → produces enhanced prompt embedding
- SamplerNode (easy_samplers / recurrent_sampler / tiled_sampler) → generates/interpolates latent tensors
- STGGuiderAdvanced → applies stepwise guidance parameters
- LatentUpsampler → upsamples latent space for higher resolution
- DecoderNoise / film_grain → adds noise or film‐grain effects
- Latents → handles representation conversions
- Final decoder (ComfyUI core or plugin) → decodes latent→image frames
4. Video assembly → output mp4/video files saved or streamed via VideoHelperSuite

c. Architectural patterns & design principles
- Plugin / modular architecture: each node is isolated, registered dynamically
- Separation of concerns: node logic vs. utility code vs. workflow definitions vs. presets
- Extensibility: new nodes can be dropped in the tricks/ folder or root and auto-registered
- Dataflow paradigm: directed graph of nodes, each consuming & emitting tensors

d. Key technologies & libraries
- Python 3.x, ComfyUI framework
- Hugging Face model files & ComfyUI Model Manager
- Custom node loader (nodes_registry.py)
- Third-party custom-node dependencies (VideoHelperSuite, LTXVideo-Q8-Kernels)

##### Detail
- Every node group and utility module
- How presets JSON feed into node parameterization
- Composition of typical flows (e.g., “Prompt → Sampler → STG → Upsampler → Decoder → Video Output”)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Core
    subgraph "ComfyUI Core"
        CoreLoader["Node Loader & Graph Engine"]
    end

    %% Plugin
    subgraph "ComfyUI-LTXVideo Plugin"
        Registry["Node Registry"]
        subgraph "Node Definitions"
            DN["decoder_noise.py"]
            ES["easy_samplers.py"]
            LA["latent_adain.py"]
            LU["latent_upsampler.py"]
            LT["latents.py"]
            PEN["prompt_enhancer_nodes.py"]
            GU["guide.py"]
            Q8["q8_nodes.py"]
            RS["recurrent_sampler.py"]
            TS["tiled_sampler.py"]
            STG["stg.py"]
            FG["film_grain.py"]
        end
        subgraph "Utility Modules"
            PEU["prompt_enhancer_utils.py"]
            ABu["attn_bank.py"]
            NU["noise_utils.py"]
            MU["module_utils.py"]
            FEU["feta_enhance_utils.py"]
            LGU["latent_guide.py"]
        end
        subgraph "Presets"
            PresetsDir["presets/"]
            STGPre["stg_advanced_presets.json"]
        end
        subgraph "Tricks"
            Init["__init__.py"]
            LTM["ltx_model.py"]
            ABn["attn_bank_nodes.py"]
            AON["attn_override_node.py"]
            LGn["latent_guide_node.py"]
            FEN["ltx_feta_enhance_node.py"]
            FEN2["ltx_flowedit_nodes.py"]
            IMP["ltx_inverse_model_pred_nodes.py"]
            PAG["ltx_pag_node.py"]
            MLN["modify_ltx_model_node.py"]
            RSN["rectified_sampler_nodes.py"]
            RFN["rf_edit_sampler_nodes.py"]
        end
    end

    %% Workflows
    subgraph "Example Workflows"
        EWDir["example_workflows/"]
        LLDir["low_level/"]
        T2V["ltxvideo-t2v.json"]
        FInt["ltxvideo-frame-interpolation.json"]
    end

    %% External
    subgraph "External Dependencies"
        CP["model checkpoints\n(models/checkpoints)"]
        UM["upscaler models\n(models/upscale_models)"]
        VHS["VideoHelperSuite &\nother custom-node deps"]
        HF["Hugging Face / GitHub"]
    end

    %% Connections
    CoreLoader --> Registry
    EWDir --> LLDir --> T2V
    LLDir --> FInt
    LLDir --> CoreLoader
    Registry --> DN
    Registry --> ES
    Registry --> LA
    Registry --> LU
    Registry --> LT
    Registry --> PEN
    Registry --> GU
    Registry --> Q8
    Registry --> RS
    Registry --> TS
    Registry --> STG
    Registry --> FG
    DN --> CP
    Q8 --> UM
    ES --> HF
    VHS --> CoreLoader

    %% Click Events
    click Registry "https://github.com/lightricks/comfyui-ltxvideo/blob/master/nodes_registry.py"
    click DN "https://github.com/lightricks/comfyui-ltxvideo/blob/master/decoder_noise.py"
    click ES "https://github.com/lightricks/comfyui-ltxvideo/blob/master/easy_samplers.py"
    click LA "https://github.com/lightricks/comfyui-ltxvideo/blob/master/latent_adain.py"
    click LU "https://github.com/lightricks/comfyui-ltxvideo/blob/master/latent_upsampler.py"
    click LT "https://github.com/lightricks/comfyui-ltxvideo/blob/master/latents.py"
    click PEN "https://github.com/lightricks/comfyui-ltxvideo/blob/master/prompt_enhancer_nodes.py"
    click GU "https://github.com/lightricks/comfyui-ltxvideo/blob/master/guide.py"
    click Q8 "https://github.com/lightricks/comfyui-ltxvideo/blob/master/q8_nodes.py"
    click RS "https://github.com/lightricks/comfyui-ltxvideo/blob/master/recurrent_sampler.py"
    click TS "https://github.com/lightricks/comfyui-ltxvideo/blob/master/tiled_sampler.py"
    click STG "https://github.com/lightricks/comfyui-ltxvideo/blob/master/stg.py"
    click FG "https://github.com/lightricks/comfyui-ltxvideo/blob/master/film_grain.py"
    click PEU "https://github.com/lightricks/comfyui-ltxvideo/blob/master/prompt_enhancer_utils.py"
    click ABu "https://github.com/lightricks/comfyui-ltxvideo/blob/master/tricks/utils/attn_bank.py"
    click NU "https://github.com/lightricks/comfyui-ltxvideo/blob/master/tricks/utils/noise_utils.py"
    click MU "https://github.com/lightricks/comfyui-ltxvideo/blob/master/tricks/utils/module_utils.py"
    click FEU "https://github.com/lightricks/comfyui-ltxvideo/blob/master/tricks/utils/feta_enhance_utils.py"
    click LGU "https://github.com/lightricks/comfyui-ltxvideo/blob/master/tricks/utils/latent_guide.py"
    click PresetsDir "https://github.com/lightricks/comfyui-ltxvideo/tree/master/presets/"
    click STGPre "https://github.com/lightricks/comfyui-ltxvideo/blob/master/presets/stg_advanced_presets.json"
    click Init "https://github.com/lightricks/comfyui-ltxvideo/blob/master/tricks/__init__.py"
    click LTM "https://github.com/lightricks/comfyui-ltxvideo/blob/master/tricks/modules/ltx_model.py"
    click ABn "https://github.com/lightricks/comfyui-ltxvideo/blob/master/tricks/nodes/attn_bank_nodes.py"
    click AON "https://github.com/lightricks/comfyui-ltxvideo/blob/master/tricks/nodes/attn_override_node.py"
    click LGn "https://github.com/lightricks/comfyui-ltxvideo/blob/master/tricks/nodes/latent_guide_node.py"
    click FEN "https://github.com/lightricks/comfyui-ltxvideo/blob/master/tricks/nodes/ltx_feta_enhance_node.py"
    click FEN2 "https://github.com/lightricks/comfyui-ltxvideo/blob/master/tricks/nodes/ltx_flowedit_nodes.py"
    click IMP "https://github.com/lightricks/comfyui-ltxvideo/blob/master/tricks/nodes/ltx_inverse_model_pred_nodes.py"
    click PAG "https://github.com/lightricks/comfyui-ltxvideo/blob/master/tricks/nodes/ltx_pag_node.py"
    click MLN "https://github.com/lightricks/comfyui-ltxvideo/blob/master/tricks/nodes/modify_ltx_model_node.py"
    click RSN "https://github.com/lightricks/comfyui-ltxvideo/blob/master/tricks/nodes/rectified_sampler_nodes.py"
    click RFN "https://github.com/lightricks/comfyui-ltxvideo/blob/master/tricks/nodes/rf_edit_sampler_nodes.py"
    click EWDir "https://github.com/lightricks/comfyui-ltxvideo/tree/master/example_workflows/"
    click LLDir "https://github.com/lightricks/comfyui-ltxvideo/tree/master/example_workflows/low_level/"
    click T2V "https://github.com/lightricks/comfyui-ltxvideo/blob/master/example_workflows/low_level/ltxvideo-t2v.json"
    click FInt "https://github.com/lightricks/comfyui-ltxvideo/blob/master/example_workflows/low_level/ltxvideo-frame-interpolation.json"
    click CP "https://github.com/lightricks/comfyui-ltxvideo/tree/master/models/checkpoints"
    click UM "https://github.com/lightricks/comfyui-ltxvideo/tree/master/models/upscale_models"

    %% Styles
    classDef core fill:#D0E8FF,stroke:#0366d6,color:#0366d6;
    classDef plugin fill:#D8FFE0,stroke:#228B22,color:#006400;
    classDef external fill:#E0E0E0,stroke:#555,color:#333;
    classDef workflow fill:#FFE0B2,stroke:#FB8C00,color:#E65100;
    class CoreLoader core
    class Registry,DN,ES,LA,LU,LT,PEN,GU,Q8,RS,TS,STG,FG,PEU,ABu,NU,MU,FEU,LGU,PresetsDir,STGPre,Init,LTM,ABn,AON,LGn,FEN,FEN2,IMP,PAG,MLN,RSN,RFN plugin
    class EWDir,LLDir,T2V,FInt workflow
    class CP,UM,VHS,HF external
```
