# alibaba-quark/liveavatar architecture

> Architecture diagram of the GitHub repository alibaba-quark/liveavatar, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/alibaba-quark/liveavatar
- Repository: https://github.com/alibaba-quark/liveavatar

## Overview

LiveAvatar is a streaming audio-driven avatar-video generation system, exposed through a Gradio interface and command-line inference. The documented workflow accepts audio with a reference image and optional prompt, then uses a causal speech-to-video pipeline for streaming, block-wise autoregressive generation; distributed inference and streaming VAE support are key internal layers. The repository also includes Wan text/video and image/video capabilities. Connections below follow the README and source-index references; unsampled implementation details are represented by their exact tree paths, without inferring unshown calls.

Main entry points: [`liveavatar/models/wan/causal_s2v_pipeline.py`](https://github.com/alibaba-quark/liveavatar/blob/main/liveavatar/models/wan/causal_s2v_pipeline.py) (Causal S2V), [`liveavatar/models/wan/causal_s2v_pipeline_tpp.py`](https://github.com/alibaba-quark/liveavatar/blob/main/liveavatar/models/wan/causal_s2v_pipeline_tpp.py) (TPP streaming), [`minimal_inference/s2v_streaming_interact.py`](https://github.com/alibaba-quark/liveavatar/blob/main/minimal_inference/s2v_streaming_interact.py) (Streaming CLI), [`minimal_inference/batch_eval.py`](https://github.com/alibaba-quark/liveavatar/blob/main/minimal_inference/batch_eval.py) (Batch inference)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interfaces["Inference interfaces"]
  node_gradio["Gradio UI<br/>[gradio_app.py]"]
  node_stream_cli["Streaming CLI"]
  node_batch_cli["Batch inference<br/>[batch_eval.py]"]
end

subgraph group_generation["Generation pipelines"]
  node_causal_pipeline["Causal S2V"]
  node_tpp_pipeline["TPP streaming"]
  node_blockwise_pipeline["Blockwise TPP"]
  node_wan_s2v["Speech-video pipeline<br/>[speech2video.py]"]
end

subgraph group_models["Model components"]
  node_causal_model["Causal S2V model"]
  node_s2v_model["S2V model blocks<br/>[model_s2v.py]"]
  node_t5["Text encoder<br/>[t5.py]"]
  node_vae["Video VAE<br/>[vae2_1.py]"]
  node_stream_vae["Streaming VAE<br/>[vae_streaming.py]"]
  node_sampler["Flow samplers<br/>[fm_solvers.py]"]
  node_flow_match["Flow matching<br/>[flow_match.py]"]
  node_base_model["Wan base model<br/>[model.py]"]
end

subgraph group_runtime["Distributed runtime"]
  node_sequence_parallel["Sequence parallel"]
  node_fsdp["FSDP sharding<br/>[fsdp.py]"]
end

subgraph group_support["Prompt and checkpoint support"]
  node_args["Inference options<br/>[args_config.py]"]
  node_weights["Weight loading"]
  node_prompt_extend["Prompt extension<br/>[prompt_extend.py]"]
end

node_user(("User"))

node_user -->|"submits request"| node_gradio
node_user -->|"provides inputs"| node_stream_cli
node_user -->|"provides inputs"| node_batch_cli
node_gradio -->|"uses pipeline"| node_tpp_pipeline
node_gradio -->|"uses pipeline"| node_causal_pipeline
node_stream_cli -->|"uses pipeline"| node_tpp_pipeline
node_stream_cli -->|"uses pipeline"| node_causal_pipeline
node_batch_cli -->|"uses pipeline"| node_tpp_pipeline
node_batch_cli -->|"uses pipeline"| node_causal_pipeline
node_gradio -->|"loads options"| node_args
node_stream_cli -->|"loads options"| node_args
node_batch_cli -->|"loads options"| node_args
node_causal_pipeline -->|"imports model"| node_causal_model
node_causal_pipeline -->|"imports encoder"| node_t5
node_causal_pipeline -->|"imports VAE"| node_vae
node_causal_pipeline -->|"imports solver"| node_sampler
node_causal_pipeline -->|"imports utilities"| node_flow_match
node_causal_pipeline -->|"loads weights"| node_weights
node_tpp_pipeline -->|"imports VAE"| node_stream_vae
node_tpp_pipeline -->|"imports parallel runtime"| node_sequence_parallel
node_tpp_pipeline -->|"imports sharding"| node_fsdp
node_tpp_pipeline -->|"imports model"| node_causal_model
node_tpp_pipeline -->|"imports solver"| node_sampler
node_wan_s2v -->|"imports model"| node_s2v_model
node_wan_s2v -->|"imports encoder"| node_t5
node_wan_s2v -->|"imports VAE"| node_vae
node_causal_model -->|"imports model blocks"| node_s2v_model
node_sequence_parallel -->|"imports embeddings"| node_base_model
node_batch_cli -->|"imports extension"| node_prompt_extend
node_stream_cli -->|"imports extension"| node_prompt_extend

click node_gradio "https://github.com/alibaba-quark/liveavatar/blob/main/minimal_inference/gradio_app.py"
click node_stream_cli "https://github.com/alibaba-quark/liveavatar/blob/main/minimal_inference/s2v_streaming_interact.py"
click node_batch_cli "https://github.com/alibaba-quark/liveavatar/blob/main/minimal_inference/batch_eval.py"
click node_args "https://github.com/alibaba-quark/liveavatar/blob/main/liveavatar/utils/args_config.py"
click node_causal_pipeline "https://github.com/alibaba-quark/liveavatar/blob/main/liveavatar/models/wan/causal_s2v_pipeline.py"
click node_tpp_pipeline "https://github.com/alibaba-quark/liveavatar/blob/main/liveavatar/models/wan/causal_s2v_pipeline_tpp.py"
click node_blockwise_pipeline "https://github.com/alibaba-quark/liveavatar/blob/main/liveavatar/models/wan/causal_s2v_pipeline_tpp_blockwise.py"
click node_wan_s2v "https://github.com/alibaba-quark/liveavatar/blob/main/liveavatar/models/wan/wan_2_2/speech2video.py"
click node_causal_model "https://github.com/alibaba-quark/liveavatar/blob/main/liveavatar/models/wan/causal_model_s2v.py"
click node_s2v_model "https://github.com/alibaba-quark/liveavatar/blob/main/liveavatar/models/wan/wan_2_2/modules/s2v/model_s2v.py"
click node_t5 "https://github.com/alibaba-quark/liveavatar/blob/main/liveavatar/models/wan/wan_2_2/modules/t5.py"
click node_vae "https://github.com/alibaba-quark/liveavatar/blob/main/liveavatar/models/wan/wan_2_2/modules/vae2_1.py"
click node_stream_vae "https://github.com/alibaba-quark/liveavatar/blob/main/liveavatar/models/wan/wan_2_2/modules/vae_streaming.py"
click node_sampler "https://github.com/alibaba-quark/liveavatar/blob/main/liveavatar/models/wan/wan_2_2/utils/fm_solvers.py"
click node_flow_match "https://github.com/alibaba-quark/liveavatar/blob/main/liveavatar/models/wan/flow_match.py"
click node_base_model "https://github.com/alibaba-quark/liveavatar/blob/main/liveavatar/models/wan/wan_base/modules/model.py"
click node_sequence_parallel "https://github.com/alibaba-quark/liveavatar/blob/main/liveavatar/models/wan/wan_2_2/distributed/sequence_parallel.py"
click node_fsdp "https://github.com/alibaba-quark/liveavatar/blob/main/liveavatar/models/wan/wan_2_2/distributed/fsdp.py"
click node_weights "https://github.com/alibaba-quark/liveavatar/blob/main/liveavatar/utils/load_weight_utils.py"
click node_prompt_extend "https://github.com/alibaba-quark/liveavatar/blob/main/liveavatar/models/wan/wan_2_2/utils/prompt_extend.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_gradio,node_stream_cli,node_batch_cli,node_user toneBlue
class node_causal_pipeline,node_tpp_pipeline,node_blockwise_pipeline,node_wan_s2v toneAmber
class node_causal_model,node_s2v_model,node_t5,node_vae,node_stream_vae,node_sampler,node_flow_match,node_base_model toneMint
class node_sequence_parallel,node_fsdp toneRose
class node_args,node_weights,node_prompt_extend toneIndigo
```

## Components

### Inference interfaces

- [Gradio UI](https://github.com/alibaba-quark/liveavatar/blob/main/minimal_inference/gradio_app.py): `minimal_inference/gradio_app.py`
- [Streaming CLI](https://github.com/alibaba-quark/liveavatar/blob/main/minimal_inference/s2v_streaming_interact.py): `minimal_inference/s2v_streaming_interact.py`
- [Batch inference](https://github.com/alibaba-quark/liveavatar/blob/main/minimal_inference/batch_eval.py): `minimal_inference/batch_eval.py`

### Generation pipelines

- [Causal S2V](https://github.com/alibaba-quark/liveavatar/blob/main/liveavatar/models/wan/causal_s2v_pipeline.py): `liveavatar/models/wan/causal_s2v_pipeline.py`
- [TPP streaming](https://github.com/alibaba-quark/liveavatar/blob/main/liveavatar/models/wan/causal_s2v_pipeline_tpp.py): `liveavatar/models/wan/causal_s2v_pipeline_tpp.py`
- [Blockwise TPP](https://github.com/alibaba-quark/liveavatar/blob/main/liveavatar/models/wan/causal_s2v_pipeline_tpp_blockwise.py): `liveavatar/models/wan/causal_s2v_pipeline_tpp_blockwise.py`
- [Speech-video pipeline](https://github.com/alibaba-quark/liveavatar/blob/main/liveavatar/models/wan/wan_2_2/speech2video.py): `liveavatar/models/wan/wan_2_2/speech2video.py`

### Model components

- [Causal S2V model](https://github.com/alibaba-quark/liveavatar/blob/main/liveavatar/models/wan/causal_model_s2v.py): `liveavatar/models/wan/causal_model_s2v.py`
- [S2V model blocks](https://github.com/alibaba-quark/liveavatar/blob/main/liveavatar/models/wan/wan_2_2/modules/s2v/model_s2v.py): `liveavatar/models/wan/wan_2_2/modules/s2v/model_s2v.py`
- [Text encoder](https://github.com/alibaba-quark/liveavatar/blob/main/liveavatar/models/wan/wan_2_2/modules/t5.py): `liveavatar/models/wan/wan_2_2/modules/t5.py`
- [Video VAE](https://github.com/alibaba-quark/liveavatar/blob/main/liveavatar/models/wan/wan_2_2/modules/vae2_1.py): `liveavatar/models/wan/wan_2_2/modules/vae2_1.py`
- [Streaming VAE](https://github.com/alibaba-quark/liveavatar/blob/main/liveavatar/models/wan/wan_2_2/modules/vae_streaming.py): `liveavatar/models/wan/wan_2_2/modules/vae_streaming.py`
- [Flow samplers](https://github.com/alibaba-quark/liveavatar/blob/main/liveavatar/models/wan/wan_2_2/utils/fm_solvers.py): `liveavatar/models/wan/wan_2_2/utils/fm_solvers.py`
- [Flow matching](https://github.com/alibaba-quark/liveavatar/blob/main/liveavatar/models/wan/flow_match.py): `liveavatar/models/wan/flow_match.py`
- [Wan base model](https://github.com/alibaba-quark/liveavatar/blob/main/liveavatar/models/wan/wan_base/modules/model.py): `liveavatar/models/wan/wan_base/modules/model.py`

### Distributed runtime

- [Sequence parallel](https://github.com/alibaba-quark/liveavatar/blob/main/liveavatar/models/wan/wan_2_2/distributed/sequence_parallel.py): `liveavatar/models/wan/wan_2_2/distributed/sequence_parallel.py`
- [FSDP sharding](https://github.com/alibaba-quark/liveavatar/blob/main/liveavatar/models/wan/wan_2_2/distributed/fsdp.py): `liveavatar/models/wan/wan_2_2/distributed/fsdp.py`

### Prompt and checkpoint support

- [Inference options](https://github.com/alibaba-quark/liveavatar/blob/main/liveavatar/utils/args_config.py): `liveavatar/utils/args_config.py`
- [Weight loading](https://github.com/alibaba-quark/liveavatar/blob/main/liveavatar/utils/load_weight_utils.py): `liveavatar/utils/load_weight_utils.py`
- [Prompt extension](https://github.com/alibaba-quark/liveavatar/blob/main/liveavatar/models/wan/wan_2_2/utils/prompt_extend.py): `liveavatar/models/wan/wan_2_2/utils/prompt_extend.py`

### Other components

- **User**

## Connections

- User → Gradio UI: submits request
- User → Streaming CLI: provides inputs
- User → Batch inference: provides inputs
- Gradio UI → TPP streaming: uses pipeline (evidence: `minimal_inference/gradio_app.py`)
- Gradio UI → Causal S2V: uses pipeline (evidence: `minimal_inference/gradio_app.py`)
- Streaming CLI → TPP streaming: uses pipeline (evidence: `minimal_inference/s2v_streaming_interact.py`)
- Streaming CLI → Causal S2V: uses pipeline (evidence: `minimal_inference/s2v_streaming_interact.py`)
- Batch inference → TPP streaming: uses pipeline (evidence: `minimal_inference/batch_eval.py`)
- Batch inference → Causal S2V: uses pipeline (evidence: `minimal_inference/batch_eval.py`)
- Gradio UI → Inference options: loads options (evidence: `minimal_inference/gradio_app.py`)
- Streaming CLI → Inference options: loads options (evidence: `minimal_inference/s2v_streaming_interact.py`)
- Batch inference → Inference options: loads options (evidence: `minimal_inference/batch_eval.py`)
- Causal S2V → Causal S2V model: imports model (evidence: `liveavatar/models/wan/causal_s2v_pipeline.py`)
- Causal S2V → Text encoder: imports encoder (evidence: `liveavatar/models/wan/causal_s2v_pipeline.py`)
- Causal S2V → Video VAE: imports VAE (evidence: `liveavatar/models/wan/causal_s2v_pipeline.py`)
- Causal S2V → Flow samplers: imports solver (evidence: `liveavatar/models/wan/causal_s2v_pipeline.py`)
- Causal S2V → Flow matching: imports utilities (evidence: `liveavatar/models/wan/causal_s2v_pipeline.py`)
- Causal S2V → Weight loading: loads weights (evidence: `liveavatar/models/wan/causal_s2v_pipeline.py`)
- TPP streaming → Streaming VAE: imports VAE (evidence: `liveavatar/models/wan/causal_s2v_pipeline_tpp.py`)
- TPP streaming → Sequence parallel: imports parallel runtime (evidence: `liveavatar/models/wan/causal_s2v_pipeline_tpp.py`)
- TPP streaming → FSDP sharding: imports sharding (evidence: `liveavatar/models/wan/causal_s2v_pipeline_tpp.py`)
- TPP streaming → Causal S2V model: imports model (evidence: `liveavatar/models/wan/causal_s2v_pipeline_tpp.py`)
- TPP streaming → Flow samplers: imports solver (evidence: `liveavatar/models/wan/causal_s2v_pipeline_tpp.py`)
- Speech-video pipeline → S2V model blocks: imports model (evidence: `liveavatar/models/wan/wan_2_2/speech2video.py`)
- Speech-video pipeline → Text encoder: imports encoder (evidence: `liveavatar/models/wan/wan_2_2/speech2video.py`)
- Speech-video pipeline → Video VAE: imports VAE (evidence: `liveavatar/models/wan/wan_2_2/speech2video.py`)
- Causal S2V model → S2V model blocks: imports model blocks (evidence: `liveavatar/models/wan/causal_model_s2v.py`)
- Sequence parallel → Wan base model: imports embeddings (evidence: `liveavatar/models/wan/wan_2_2/distributed/sequence_parallel.py`)
- Batch inference → Prompt extension: imports extension (evidence: `minimal_inference/batch_eval.py`)
- Streaming CLI → Prompt extension: imports extension (evidence: `minimal_inference/s2v_streaming_interact.py`)