# meigen-ai/infinitetalk architecture

> Architecture diagram of the GitHub repository meigen-ai/infinitetalk, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/meigen-ai/infinitetalk
- Repository: https://github.com/meigen-ai/infinitetalk

## Overview

InfiniteTalk takes audio with a source video or image, extracts audio features, and generates an identity-conditioned talking video intended to support long durations. The graph follows the visible Gradio preprocessing and generation path and the documented image/audio and video/audio modes. Source excerpts are partial, so some model-internal wiring and the standalone CLI path are represented only where the tree or excerpts support them.

Main entry points: [`wan/multitalk.py`](https://github.com/meigen-ai/infinitetalk/blob/main/wan/multitalk.py) (InfiniteTalk pipeline), [`app.py`](https://github.com/meigen-ai/infinitetalk/blob/main/app.py) (Gradio demo), [`wan/utils/prompt_extend.py`](https://github.com/meigen-ai/infinitetalk/blob/main/wan/utils/prompt_extend.py) (Prompt extension), [`generate_infinitetalk.py`](https://github.com/meigen-ai/infinitetalk/blob/main/generate_infinitetalk.py) (Inference entry)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_input["Inputs and interface"]
  node_app["Gradio demo<br/>[app.py]"]
  node_cli["Inference entry"]
end

subgraph group_conditioning["Conditioning"]
  node_audio_prep["Audio preparation<br/>[app.py]"]
  node_audio_features["Wav2Vec2 features<br/>[wav2vec2.py]"]
  node_text_models["Text encoders<br/>[t5.py]"]
  node_tokenizers["Tokenization<br/>[tokenizers.py]"]
  node_prompt_extend["Prompt extension<br/>[prompt_extend.py]"]
end

subgraph group_generation["Video generation"]
  node_pipeline["InfiniteTalk pipeline<br/>[multitalk.py]"]
  node_dit["Audio-conditioned DiT<br/>[multitalk_model.py]"]
  node_attention["Attention operations<br/>[attention.py]"]
  node_vae["Video VAE<br/>[vae.py]"]
end

subgraph group_runtime["Runtime and output"]
  node_parallel["Distributed inference"]
  node_vram["VRAM management<br/>[layers.py]"]
  node_encode["Video and audio export<br/>[app.py]"]
end

node_user(("User"))
node_media(("Audio and reference media"))
node_dashscope{{"DashScope service"}}
node_result["Generated video"]

node_user -->|"submits request"| node_app
node_media -->|"provides inputs"| node_app
node_app -->|"prepares audio"| node_audio_prep
node_audio_prep -->|"extracts embeddings"| node_audio_features
node_app -->|"requests generation"| node_pipeline
node_cli -->|"runs inference"| node_pipeline
node_pipeline -->|"denoises latents"| node_dit
node_pipeline -->|"decodes latents"| node_vae
node_pipeline -->|"encodes prompts"| node_text_models
node_pipeline -->|"tokenizes prompts"| node_tokenizers
node_pipeline -.->|"uses parallel inference"| node_parallel
node_pipeline -.->|"manages memory"| node_vram
node_app -.->|"may extend prompt"| node_prompt_extend
node_prompt_extend -.->|"calls service"| node_dashscope
node_app -->|"saves generated video"| node_encode
node_encode -->|"produces"| node_result

click node_app "https://github.com/meigen-ai/infinitetalk/blob/main/app.py"
click node_cli "https://github.com/meigen-ai/infinitetalk/blob/main/generate_infinitetalk.py"
click node_audio_prep "https://github.com/meigen-ai/infinitetalk/blob/main/app.py"
click node_audio_features "https://github.com/meigen-ai/infinitetalk/blob/main/src/audio_analysis/wav2vec2.py"
click node_text_models "https://github.com/meigen-ai/infinitetalk/blob/main/wan/modules/t5.py"
click node_tokenizers "https://github.com/meigen-ai/infinitetalk/blob/main/wan/modules/tokenizers.py"
click node_prompt_extend "https://github.com/meigen-ai/infinitetalk/blob/main/wan/utils/prompt_extend.py"
click node_pipeline "https://github.com/meigen-ai/infinitetalk/blob/main/wan/multitalk.py"
click node_dit "https://github.com/meigen-ai/infinitetalk/blob/main/wan/modules/multitalk_model.py"
click node_attention "https://github.com/meigen-ai/infinitetalk/blob/main/wan/modules/attention.py"
click node_vae "https://github.com/meigen-ai/infinitetalk/blob/main/wan/modules/vae.py"
click node_parallel "https://github.com/meigen-ai/infinitetalk/blob/main/wan/distributed/xdit_context_parallel.py"
click node_vram "https://github.com/meigen-ai/infinitetalk/blob/main/src/vram_management/layers.py"
click node_encode "https://github.com/meigen-ai/infinitetalk/blob/main/app.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_app,node_cli,node_user toneBlue
class node_audio_prep,node_audio_features,node_text_models,node_tokenizers,node_prompt_extend toneAmber
class node_pipeline,node_dit,node_attention,node_vae toneMint
class node_parallel,node_vram,node_encode toneRose
class node_media,node_dashscope,node_result toneIndigo
```

## Components

### Inputs and interface

- [Gradio demo](https://github.com/meigen-ai/infinitetalk/blob/main/app.py): `app.py`
- [Inference entry](https://github.com/meigen-ai/infinitetalk/blob/main/generate_infinitetalk.py): `generate_infinitetalk.py`

### Conditioning

- [Audio preparation](https://github.com/meigen-ai/infinitetalk/blob/main/app.py): `app.py`
- [Wav2Vec2 features](https://github.com/meigen-ai/infinitetalk/blob/main/src/audio_analysis/wav2vec2.py): `src/audio_analysis/wav2vec2.py`
- [Text encoders](https://github.com/meigen-ai/infinitetalk/blob/main/wan/modules/t5.py): `wan/modules/t5.py`
- [Tokenization](https://github.com/meigen-ai/infinitetalk/blob/main/wan/modules/tokenizers.py): `wan/modules/tokenizers.py`
- [Prompt extension](https://github.com/meigen-ai/infinitetalk/blob/main/wan/utils/prompt_extend.py): `wan/utils/prompt_extend.py`

### Video generation

- [InfiniteTalk pipeline](https://github.com/meigen-ai/infinitetalk/blob/main/wan/multitalk.py): `wan/multitalk.py`
- [Audio-conditioned DiT](https://github.com/meigen-ai/infinitetalk/blob/main/wan/modules/multitalk_model.py): `wan/modules/multitalk_model.py`
- [Attention operations](https://github.com/meigen-ai/infinitetalk/blob/main/wan/modules/attention.py): `wan/modules/attention.py`
- [Video VAE](https://github.com/meigen-ai/infinitetalk/blob/main/wan/modules/vae.py): `wan/modules/vae.py`

### Runtime and output

- [Distributed inference](https://github.com/meigen-ai/infinitetalk/blob/main/wan/distributed/xdit_context_parallel.py): `wan/distributed/xdit_context_parallel.py`
- [VRAM management](https://github.com/meigen-ai/infinitetalk/blob/main/src/vram_management/layers.py): `src/vram_management/layers.py`
- [Video and audio export](https://github.com/meigen-ai/infinitetalk/blob/main/app.py): `app.py`

### Other components

- **User**
- **Audio and reference media**
- **DashScope service**
- **Generated video**

## Connections

- User → Gradio demo: submits request
- Audio and reference media → Gradio demo: provides inputs
- Gradio demo → Audio preparation: prepares audio
- Audio preparation → Wav2Vec2 features: extracts embeddings
- Gradio demo → InfiniteTalk pipeline: requests generation
- Inference entry → InfiniteTalk pipeline: runs inference
- InfiniteTalk pipeline → Audio-conditioned DiT: denoises latents
- InfiniteTalk pipeline → Video VAE: decodes latents
- InfiniteTalk pipeline → Text encoders: encodes prompts
- InfiniteTalk pipeline → Tokenization: tokenizes prompts
- InfiniteTalk pipeline → Distributed inference: uses parallel inference
- InfiniteTalk pipeline → VRAM management: manages memory
- Gradio demo → Prompt extension: may extend prompt
- Prompt extension → DashScope service: calls service
- Gradio demo → Video and audio export: saves generated video
- Video and audio export → Generated video: produces