How tencent-hunyuan/hunyuanvideo-avatar works
Project type and purpose
• This is an audio-driven human animation inference engine (HunyuanVideo-Avatar). It provides both CLI scripts and a lightweight web interface (Gradio/Flask) to generate multi-character, emotion-aware talking head videos from avatar images and audio.
• It’s not a full-stack web app in the traditional sense but a specialized ML inference service with optional web front-end.
File structure
• Top-level directories:
• hymm_sp: the core Python package implementing model loading, data preprocessing, diffusion pipelines, VAE, text encoder, modules, and inference routines.
• hymm_gradio: two server wrappers (flask_audio.py, gradio_audio.py) plus a tool_for_end2end integration script.
• scripts: shell wrappers to launch Gradio (run_gradio.sh), batch inference (run_sample_batch_sp.sh), single-GPU and low-VRAM runs.
• weights: pretrained model checkpoints.
• requirements.txt and typical deployment files (.gitignore, Docker/Conda instructions in README).
README for additional insights
• Core architecture is a multimodal diffusion transformer (MM-DiT).
• Three modules: Character Image Injection, Audio Emotion Module (AEM), and Face-Aware Audio Adapter (FAA).
• Supports parallel inference with torchrun and single-GPU/low-VRAM modes.
• Offers a Gradio-based UI for interactive demos.