# nvlabs/longlive architecture

> Architecture diagram of the GitHub repository nvlabs/longlive, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nvlabs/longlive
- Repository: https://github.com/nvlabs/longlive

## Overview

The repository contains three generations of long-video generation software: LongLive-Plug for reusable capability distillation, LongLive 1.0 for real-time interactive generation, and LongLive 2.0 for parallel, quantized training and inference. The main user workflow is prompt input through a generation pipeline to video output; training and distillation form distinct model-production workflows. README documents additional training and serving capabilities beyond the sampled excerpts, so those branches are represented at their supported directory paths; wiring not evidenced by the excerpts is omitted.

Main entry points: [`LongLive-Plug/inference.py`](https://github.com/nvlabs/longlive/blob/main/LongLive-Plug/inference.py) (Video inference), [`LongLive-Plug/trainer/distillation.py`](https://github.com/nvlabs/longlive/blob/main/LongLive-Plug/trainer/distillation.py) (Distillation trainer), [`LongLive2.0/inference.py`](https://github.com/nvlabs/longlive/blob/main/LongLive2.0/inference.py) (Inference entry), [`LongLive1.0/interactive_inference.py`](https://github.com/nvlabs/longlive/blob/main/LongLive1.0/interactive_inference.py) (Interactive inference)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_plug["LongLive Plug"]
  node_plug_train["Capability distillation<br/>[train.py]"]
  node_plug_trainer["Distillation trainer<br/>[distillation.py]"]
  node_plug_loss["Distillation methods"]
  node_plug_data["Prompt dataset<br/>[dataset.py]"]
  node_plug_infer["Video inference<br/>[inference.py]"]
  node_plug_lora["Functional LoRA<br/>[lora_utils.py]"]
  node_plug_wan["Wan components"]
  node_plug_video["Generated video"]
end

subgraph group_v1["LongLive 1.0"]
  node_v1_train["Model training<br/>[train.py]"]
  node_v1_trainer["Distillation trainer<br/>[distillation.py]"]
  node_v1_interactive["Interactive inference"]
  node_v1_pipeline["Interactive pipeline"]
  node_v1_causal["Causal generation"]
  node_v1_text["Text conditioning<br/>[wan_wrapper.py]"]
  node_v1_model["Causal Wan model<br/>[causal_model.py]"]
  node_v1_video["Generated video"]
end

subgraph group_v2["LongLive 2.0"]
  node_v2_train["Training entry<br/>[train.py]"]
  node_v2_trainer["Training workflows<br/>[trainer]"]
  node_v2_infer["Inference entry<br/>[inference.py]"]
  node_v2_sp_infer["Parallel inference<br/>[inference_sp.py]"]
  node_v2_pipeline["Causal diffusion"]
  node_v2_sp_pipeline["Sequence-parallel pipeline"]
  node_v2_sp["Ulysses collectives"]
  node_v2_quant_setup["Quantized setup<br/>[inference_utils.py]"]
  node_v2_quant["Quantization runtime<br/>[quant.py]"]
  node_v2_kernel["FourOverSix kernels<br/>[quantize]"]
  node_v2_video["Generated video"]
end

subgraph group_models["Model foundations"]
  node_wan1["Wan 1.0 modules<br/>[modules]"]
  node_wan2["Wan 2.0 modules<br/>[modules]"]
  node_plug_models["Plug Wan models"]
end

node_user(("Prompt user"))
node_data["Training data"]

node_user -->|"provides prompts"| node_plug_infer
node_plug_infer -->|"loads adapter"| node_plug_lora
node_plug_infer -->|"runs model"| node_plug_wan
node_plug_wan -->|"decodes frames"| node_plug_video
node_plug_infer -->|"saves output"| node_plug_video
node_data -->|"supplies examples"| node_plug_data
node_plug_train -->|"loads data"| node_plug_data
node_plug_train -->|"starts training"| node_plug_trainer
node_plug_trainer -->|"uses objectives"| node_plug_loss
node_plug_trainer -->|"trains adapters"| node_plug_lora
node_plug_trainer -->|"distills model"| node_plug_wan
node_user -->|"provides prompts"| node_v1_interactive
node_v1_interactive -->|"invokes"| node_v1_pipeline
node_v1_pipeline -->|"inherits inference"| node_v1_causal
node_v1_causal -->|"encodes prompts"| node_v1_text
node_v1_causal -->|"denoises chunks"| node_v1_model
node_v1_model -->|"uses modules"| node_wan1
node_v1_interactive -->|"writes video"| node_v1_video
node_data -->|"supplies training data"| node_v1_train
node_v1_train -->|"starts training"| node_v1_trainer
node_user -->|"provides prompts"| node_v2_infer
node_v2_infer -->|"prepares quantization"| node_v2_quant_setup
node_v2_quant_setup -->|"configures weights"| node_v2_quant
node_v2_quant -->|"dispatches kernels"| node_v2_kernel
node_v2_infer -->|"runs inference"| node_v2_pipeline
node_v2_pipeline -->|"uses Wan components"| node_wan2
node_v2_pipeline -->|"decodes output"| node_v2_video
node_v2_infer -->|"saves video"| node_v2_video
node_user -->|"provides prompts"| node_v2_sp_infer
node_v2_sp_infer -->|"constructs pipeline"| node_v2_sp_pipeline
node_v2_sp_pipeline -->|"uses collectives"| node_v2_sp
node_v2_sp_pipeline -->|"runs causal model"| node_wan2
node_data -->|"supplies training data"| node_v2_train
node_v2_train -->|"starts training"| node_v2_trainer
node_v2_trainer -->|"trains models"| node_wan2
node_plug_infer -->|"loads architecture"| node_plug_models

click node_plug_train "https://github.com/nvlabs/longlive/blob/main/LongLive-Plug/train.py"
click node_plug_trainer "https://github.com/nvlabs/longlive/blob/main/LongLive-Plug/trainer/distillation.py"
click node_plug_loss "https://github.com/nvlabs/longlive/tree/main/LongLive-Plug/model"
click node_plug_data "https://github.com/nvlabs/longlive/blob/main/LongLive-Plug/utils/dataset.py"
click node_plug_infer "https://github.com/nvlabs/longlive/blob/main/LongLive-Plug/inference.py"
click node_plug_lora "https://github.com/nvlabs/longlive/blob/main/LongLive-Plug/utils/lora_utils.py"
click node_plug_wan "https://github.com/nvlabs/longlive/tree/main/LongLive-Plug/wan_5b/modules"
click node_v1_train "https://github.com/nvlabs/longlive/blob/main/LongLive1.0/train.py"
click node_v1_trainer "https://github.com/nvlabs/longlive/blob/main/LongLive1.0/trainer/distillation.py"
click node_v1_interactive "https://github.com/nvlabs/longlive/blob/main/LongLive1.0/interactive_inference.py"
click node_v1_pipeline "https://github.com/nvlabs/longlive/blob/main/LongLive1.0/pipeline/interactive_causal_inference.py"
click node_v1_causal "https://github.com/nvlabs/longlive/blob/main/LongLive1.0/pipeline/causal_inference.py"
click node_v1_text "https://github.com/nvlabs/longlive/blob/main/LongLive1.0/utils/wan_wrapper.py"
click node_v1_model "https://github.com/nvlabs/longlive/blob/main/LongLive1.0/wan/modules/causal_model.py"
click node_v2_train "https://github.com/nvlabs/longlive/blob/main/LongLive2.0/train.py"
click node_v2_trainer "https://github.com/nvlabs/longlive/tree/main/LongLive2.0/trainer"
click node_v2_infer "https://github.com/nvlabs/longlive/blob/main/LongLive2.0/inference.py"
click node_v2_sp_infer "https://github.com/nvlabs/longlive/blob/main/LongLive2.0/inference_sp.py"
click node_v2_pipeline "https://github.com/nvlabs/longlive/blob/main/LongLive2.0/pipeline/causal_diffusion_inference.py"
click node_v2_sp_pipeline "https://github.com/nvlabs/longlive/blob/main/LongLive2.0/pipeline/causal_diffusion_inference_sp.py"
click node_v2_sp "https://github.com/nvlabs/longlive/blob/main/LongLive2.0/wan_5b/distributed/sp_ulysses_inference.py"
click node_v2_quant_setup "https://github.com/nvlabs/longlive/blob/main/LongLive2.0/utils/inference_utils.py"
click node_v2_quant "https://github.com/nvlabs/longlive/blob/main/LongLive2.0/utils/quant.py"
click node_v2_kernel "https://github.com/nvlabs/longlive/tree/main/LongLive2.0/fouroversix/src/fouroversix/quantize"
click node_wan1 "https://github.com/nvlabs/longlive/tree/main/LongLive1.0/wan/modules"
click node_wan2 "https://github.com/nvlabs/longlive/tree/main/LongLive2.0/wan_5b/modules"
click node_plug_models "https://github.com/nvlabs/longlive/tree/main/LongLive-Plug/wan_5b/modules"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_plug_train,node_plug_trainer,node_plug_loss,node_plug_data,node_plug_infer,node_plug_lora,node_plug_wan,node_plug_video,node_user toneBlue
class node_v1_train,node_v1_trainer,node_v1_interactive,node_v1_pipeline,node_v1_causal,node_v1_text,node_v1_model,node_v1_video toneAmber
class node_v2_train,node_v2_trainer,node_v2_infer,node_v2_sp_infer,node_v2_pipeline,node_v2_sp_pipeline,node_v2_sp,node_v2_quant_setup,node_v2_quant,node_v2_kernel,node_v2_video toneMint
class node_wan1,node_wan2,node_plug_models toneRose
class node_data toneIndigo
```

## Components

### LongLive Plug

- [Capability distillation](https://github.com/nvlabs/longlive/blob/main/LongLive-Plug/train.py): `LongLive-Plug/train.py`
- [Distillation trainer](https://github.com/nvlabs/longlive/blob/main/LongLive-Plug/trainer/distillation.py): `LongLive-Plug/trainer/distillation.py`
- [Distillation methods](https://github.com/nvlabs/longlive/tree/main/LongLive-Plug/model): `LongLive-Plug/model`
- [Prompt dataset](https://github.com/nvlabs/longlive/blob/main/LongLive-Plug/utils/dataset.py): `LongLive-Plug/utils/dataset.py`
- [Video inference](https://github.com/nvlabs/longlive/blob/main/LongLive-Plug/inference.py): `LongLive-Plug/inference.py`
- [Functional LoRA](https://github.com/nvlabs/longlive/blob/main/LongLive-Plug/utils/lora_utils.py): `LongLive-Plug/utils/lora_utils.py`
- [Wan components](https://github.com/nvlabs/longlive/tree/main/LongLive-Plug/wan_5b/modules): `LongLive-Plug/wan_5b/modules`
- **Generated video**

### LongLive 1.0

- [Model training](https://github.com/nvlabs/longlive/blob/main/LongLive1.0/train.py): `LongLive1.0/train.py`
- [Distillation trainer](https://github.com/nvlabs/longlive/blob/main/LongLive1.0/trainer/distillation.py): `LongLive1.0/trainer/distillation.py`
- [Interactive inference](https://github.com/nvlabs/longlive/blob/main/LongLive1.0/interactive_inference.py): `LongLive1.0/interactive_inference.py`
- [Interactive pipeline](https://github.com/nvlabs/longlive/blob/main/LongLive1.0/pipeline/interactive_causal_inference.py): `LongLive1.0/pipeline/interactive_causal_inference.py`
- [Causal generation](https://github.com/nvlabs/longlive/blob/main/LongLive1.0/pipeline/causal_inference.py): `LongLive1.0/pipeline/causal_inference.py`
- [Text conditioning](https://github.com/nvlabs/longlive/blob/main/LongLive1.0/utils/wan_wrapper.py): `LongLive1.0/utils/wan_wrapper.py`
- [Causal Wan model](https://github.com/nvlabs/longlive/blob/main/LongLive1.0/wan/modules/causal_model.py): `LongLive1.0/wan/modules/causal_model.py`
- **Generated video**

### LongLive 2.0

- [Training entry](https://github.com/nvlabs/longlive/blob/main/LongLive2.0/train.py): `LongLive2.0/train.py`
- [Training workflows](https://github.com/nvlabs/longlive/tree/main/LongLive2.0/trainer): `LongLive2.0/trainer`
- [Inference entry](https://github.com/nvlabs/longlive/blob/main/LongLive2.0/inference.py): `LongLive2.0/inference.py`
- [Parallel inference](https://github.com/nvlabs/longlive/blob/main/LongLive2.0/inference_sp.py): `LongLive2.0/inference_sp.py`
- [Causal diffusion](https://github.com/nvlabs/longlive/blob/main/LongLive2.0/pipeline/causal_diffusion_inference.py): `LongLive2.0/pipeline/causal_diffusion_inference.py`
- [Sequence-parallel pipeline](https://github.com/nvlabs/longlive/blob/main/LongLive2.0/pipeline/causal_diffusion_inference_sp.py): `LongLive2.0/pipeline/causal_diffusion_inference_sp.py`
- [Ulysses collectives](https://github.com/nvlabs/longlive/blob/main/LongLive2.0/wan_5b/distributed/sp_ulysses_inference.py): `LongLive2.0/wan_5b/distributed/sp_ulysses_inference.py`
- [Quantized setup](https://github.com/nvlabs/longlive/blob/main/LongLive2.0/utils/inference_utils.py): `LongLive2.0/utils/inference_utils.py`
- [Quantization runtime](https://github.com/nvlabs/longlive/blob/main/LongLive2.0/utils/quant.py): `LongLive2.0/utils/quant.py`
- [FourOverSix kernels](https://github.com/nvlabs/longlive/tree/main/LongLive2.0/fouroversix/src/fouroversix/quantize): `LongLive2.0/fouroversix/src/fouroversix/quantize`
- **Generated video**

### Model foundations

- [Wan 1.0 modules](https://github.com/nvlabs/longlive/tree/main/LongLive1.0/wan/modules): `LongLive1.0/wan/modules`
- [Wan 2.0 modules](https://github.com/nvlabs/longlive/tree/main/LongLive2.0/wan_5b/modules): `LongLive2.0/wan_5b/modules`
- [Plug Wan models](https://github.com/nvlabs/longlive/tree/main/LongLive-Plug/wan_5b/modules): `LongLive-Plug/wan_5b/modules`

### Other components

- **Prompt user**
- **Training data**

## Connections

- Prompt user → Video inference: provides prompts
- Video inference → Functional LoRA: loads adapter
- Video inference → Wan components: runs model
- Wan components → Generated video: decodes frames
- Video inference → Generated video: saves output
- Training data → Prompt dataset: supplies examples
- Capability distillation → Prompt dataset: loads data
- Capability distillation → Distillation trainer: starts training
- Distillation trainer → Distillation methods: uses objectives
- Distillation trainer → Functional LoRA: trains adapters
- Distillation trainer → Wan components: distills model
- Prompt user → Interactive inference: provides prompts
- Interactive inference → Interactive pipeline: invokes
- Interactive pipeline → Causal generation: inherits inference
- Causal generation → Text conditioning: encodes prompts
- Causal generation → Causal Wan model: denoises chunks
- Causal Wan model → Wan 1.0 modules: uses modules
- Interactive inference → Generated video: writes video
- Training data → Model training: supplies training data
- Model training → Distillation trainer: starts training
- Prompt user → Inference entry: provides prompts
- Inference entry → Quantized setup: prepares quantization
- Quantized setup → Quantization runtime: configures weights
- Quantization runtime → FourOverSix kernels: dispatches kernels
- Inference entry → Causal diffusion: runs inference
- Causal diffusion → Wan 2.0 modules: uses Wan components
- Causal diffusion → Generated video: decodes output
- Inference entry → Generated video: saves video
- Prompt user → Parallel inference: provides prompts
- Parallel inference → Sequence-parallel pipeline: constructs pipeline
- Sequence-parallel pipeline → Ulysses collectives: uses collectives
- Sequence-parallel pipeline → Wan 2.0 modules: runs causal model
- Training data → Training entry: supplies training data
- Training entry → Training workflows: starts training
- Training workflows → Wan 2.0 modules: trains models
- Video inference → Plug Wan models: loads architecture