How dr-rehansaeed/infinite-talk works
This repository combines Wan video-generation capabilities with Kokoro speech synthesis and audio-analysis utilities; the visible product workflow is InfiniteTalk generation from prompts and media inputs to a rendered video. Wan also exposes text-to-video, image-to-video, first/last-frame, and VACE generation paths. The source index supports the module relationships below, but excerpts are incomplete and the README is empty, so some internal orchestration details are omitted.
Main entry points: multitalk.py InfiniteTalk pipeline · image2video.py Image-to-video · first_last_frame2video.py First/last-frame video · text2video.py Text-to-video
Components 24
Generation workflows
- InfiniteTalk runner
generate_infinitetalk.py - InfiniteTalk pipeline
wan/multitalk.py - Text-to-video
wan/text2video.py - Image-to-video
wan/image2video.py - First/last-frame video
wan/first_last_frame2video.py - VACE generation
wan/vace.py - Rendered video
wan/utils/multitalk_utils.py
Model components
- Wan diffusion model
wan/modules/model.py - Multitalk model
wan/modules/multitalk_model.py - Video VAE
wan/modules/vae.py - CLIP image encoder
wan/modules/clip.py - T5 text encoder
wan/modules/t5.py - Flow solvers
wan/utils/fm_solvers.py - UniPC solver
wan/utils/fm_solvers_unipc.py - VACE model
wan/modules/vace_model.py
Speech and audio
- Wav2Vec2 analysis
src/audio_analysis/wav2vec2.py - Kokoro speech pipeline
kokoro/pipeline.py - Kokoro synthesizer
kokoro/model.py - Vocoder and STFT
kokoro/istftnet.py
Runtime support
- Distributed execution
wan/distributed/xdit_context_parallel.py - Model sharding
wan/distributed/fsdp.py - Prompt extension
wan/utils/prompt_extend.py - Video segmentation
wan/utils/segvideo.py
Other components
- User
Connections 39
- User → InfiniteTalk runner: provides inputs
generate_infinitetalk.py - InfiniteTalk runner → InfiniteTalk pipeline: starts generation
generate_infinitetalk.py - InfiniteTalk runner → Wav2Vec2 analysis: analyzes audio
generate_infinitetalk.py - InfiniteTalk runner → Kokoro speech pipeline: synthesizes speech
generate_infinitetalk.py - InfiniteTalk runner → Video segmentation: uses segmentation
generate_infinitetalk.py - InfiniteTalk pipeline → Multitalk model: runs model
wan/multitalk.py - InfiniteTalk pipeline → Wan diffusion model: configures model
wan/multitalk.py - InfiniteTalk pipeline → Video VAE: encodes and decodes
wan/multitalk.py - InfiniteTalk pipeline → T5 text encoder: encodes prompts
wan/multitalk.py - InfiniteTalk pipeline → CLIP image encoder: encodes images
wan/multitalk.py - InfiniteTalk pipeline → Distributed execution: patches attention
wan/multitalk.py - InfiniteTalk pipeline → Model sharding: shards models
wan/multitalk.py - InfiniteTalk pipeline → Rendered video: renders video
wan/utils/multitalk_utils.py - Text-to-video → Wan diffusion model: runs model
wan/text2video.py - Text-to-video → T5 text encoder: encodes prompts
wan/text2video.py - Text-to-video → Video VAE: decodes frames
wan/text2video.py - Text-to-video → Flow solvers: samples steps
wan/text2video.py - Text-to-video → UniPC solver: samples steps
wan/text2video.py - Text-to-video → Distributed execution: uses parallelism
wan/text2video.py - Image-to-video → Wan diffusion model: runs model
wan/image2video.py - Image-to-video → CLIP image encoder: encodes images
wan/image2video.py - Image-to-video → T5 text encoder: encodes prompts
wan/image2video.py - Image-to-video → Video VAE: decodes frames
wan/image2video.py - Image-to-video → Flow solvers: samples steps
wan/image2video.py - Image-to-video → UniPC solver: samples steps
wan/image2video.py - Image-to-video → Distributed execution: uses parallelism
wan/image2video.py - First/last-frame video → Wan diffusion model: runs model
wan/first_last_frame2video.py - First/last-frame video → CLIP image encoder: encodes images
wan/first_last_frame2video.py - First/last-frame video → T5 text encoder: encodes prompts
wan/first_last_frame2video.py - First/last-frame video → Video VAE: decodes frames
wan/first_last_frame2video.py - First/last-frame video → Flow solvers: samples steps
wan/first_last_frame2video.py - First/last-frame video → UniPC solver: samples steps
wan/first_last_frame2video.py - First/last-frame video → Distributed execution: uses parallelism
wan/first_last_frame2video.py - VACE generation → VACE model: runs model
wan/vace.py - VACE generation → Distributed execution: uses parallelism
wan/vace.py - VACE generation → Text-to-video: reuses generation
wan/vace.py - Kokoro speech pipeline → Kokoro synthesizer: runs synthesis
kokoro/pipeline.py - Kokoro synthesizer → Vocoder and STFT: generates waveform
kokoro/model.py - InfiniteTalk runner → Prompt extension: extends prompts
generate_infinitetalk.py