How meigen-ai/infinitetalk works
InfiniteTalk takes audio with a source video or image, extracts audio features, and generates an identity-conditioned talking video intended to support long durations. The graph follows the visible Gradio preprocessing and generation path and the documented image/audio and video/audio modes. Source excerpts are partial, so some model-internal wiring and the standalone CLI path are represented only where the tree or excerpts support them.
Main entry points: multitalk.py InfiniteTalk pipeline · app.py Gradio demo · prompt_extend.py Prompt extension · generate_infinitetalk.py Inference entry
Components 18
Inputs and interface
- Gradio demo
app.py - Inference entry
generate_infinitetalk.py
Conditioning
- Audio preparation
app.py - Wav2Vec2 features
src/audio_analysis/wav2vec2.py - Text encoders
wan/modules/t5.py - Tokenization
wan/modules/tokenizers.py - Prompt extension
wan/utils/prompt_extend.py
Video generation
- InfiniteTalk pipeline
wan/multitalk.py - Audio-conditioned DiT
wan/modules/multitalk_model.py - Attention operations
wan/modules/attention.py - Video VAE
wan/modules/vae.py
Runtime and output
- Distributed inference
wan/distributed/xdit_context_parallel.py - VRAM management
src/vram_management/layers.py - Video and audio export
app.py
Other components
- User
- Audio and reference media
- DashScope service
- Generated video
Connections 16
- User → Gradio demo: submits request
- Audio and reference media → Gradio demo: provides inputs
- Gradio demo → Audio preparation: prepares audio
- Audio preparation → Wav2Vec2 features: extracts embeddings
- Gradio demo → InfiniteTalk pipeline: requests generation
- Inference entry → InfiniteTalk pipeline: runs inference
- InfiniteTalk pipeline → Audio-conditioned DiT: denoises latents
- InfiniteTalk pipeline → Video VAE: decodes latents
- InfiniteTalk pipeline → Text encoders: encodes prompts
- InfiniteTalk pipeline → Tokenization: tokenizes prompts
- InfiniteTalk pipeline → Distributed inference: uses parallel inference
- InfiniteTalk pipeline → VRAM management: manages memory
- Gradio demo → Prompt extension: may extend prompt
- Prompt extension → DashScope service: calls service
- Gradio demo → Video and audio export: saves generated video
- Video and audio export → Generated video: produces