# francis-rings/stableavatar architecture

> Architecture diagram of the GitHub repository francis-rings/stableavatar, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/francis-rings/stableavatar
- Repository: https://github.com/francis-rings/stableavatar

## Overview

StableAvatar generates identity-preserving, audio-driven avatar video from a reference image and speech audio, with long-duration output and training/data-preparation workflows. The main user path runs through the Gradio app into the inference pipeline, which conditions video diffusion on image, text and audio features, then decodes and muxes the result. The graph also captures documented preprocessing and training capabilities. Some source excerpts are partial, so unconfirmed internal calls and subsystem wiring are omitted.

Main entry points: [`wan/pipeline/wan_inference_long_pipeline.py`](https://github.com/francis-rings/stableavatar/blob/main/wan/pipeline/wan_inference_long_pipeline.py) (Long-video pipeline), [`app.py`](https://github.com/francis-rings/stableavatar/blob/main/app.py) (Gradio interface), [`wan/dataset/talking_video_dataset_fantasy.py`](https://github.com/francis-rings/stableavatar/blob/main/wan/dataset/talking_video_dataset_fantasy.py) (Talking-video dataset), [`wan/models/wan_fantasy_transformer3d_1B.py`](https://github.com/francis-rings/stableavatar/blob/main/wan/models/wan_fantasy_transformer3d_1B.py) (Video transformer)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_generation["Video generation"]
  node_app["Gradio interface<br/>[app.py]"]
  node_inference["Inference runner<br/>[inference.py]"]
  node_longpipe["Long-video pipeline"]
  node_fantasypipe["Fantasy pipeline"]
  node_text2video["Text-to-video runner<br/>[text2video.py]"]
  node_image2video["Image-to-video runner<br/>[image2video.py]"]
end

subgraph group_models["Model components"]
  node_transformer["Video transformer"]
  node_vocal["Vocal projector"]
  node_wav2vec["Audio encoder<br/>[wav2vec.py]"]
  node_textencoder["Text encoder"]
  node_imageencoder["Image encoder"]
  node_vae["Video VAE<br/>[wan_vae.py]"]
  node_motion["Motion controller"]
end

subgraph group_preparation["Audio preparation"]
  node_audio_extract["Audio extraction<br/>[audio_extractor.py]"]
  node_vocal_separation["Vocal separation<br/>[vocal_seperator.py]"]
  node_lip_masks["Lip-mask extraction"]
end

subgraph group_training["Training"]
  node_dataset["Talking-video dataset"]
  node_training_scripts["Training and LoRA<br/>[train_1B_square.py]"]
end

node_user(("User"))
node_video["Generated video"]
node_audio["Speech audio"]
node_reference["Reference image"]
node_prompt["Prompt"]

node_user -->|"provides inputs"| node_app
node_app -->|"runs generation"| node_longpipe
node_inference -->|"runs generation"| node_longpipe
node_longpipe -->|"encodes prompt"| node_textencoder
node_longpipe -->|"encodes image"| node_imageencoder
node_longpipe -->|"encodes audio"| node_wav2vec
node_longpipe -->|"denoises latents"| node_transformer
node_longpipe -->|"projects audio"| node_vocal
node_longpipe -->|"decodes frames"| node_vae
node_longpipe -->|"produces"| node_video
node_app -->|"reads audio"| node_audio
node_app -->|"reads image"| node_reference
node_app -->|"reads prompt"| node_prompt
node_app -->|"muxes audio"| node_video
node_text2video -->|"uses pipeline"| node_fantasypipe
node_image2video -->|"runs model"| node_transformer
node_dataset -->|"supplies samples"| node_training_scripts
node_training_scripts -->|"trains model"| node_transformer
node_audio_extract -->|"extracts tracks"| node_audio
node_vocal_separation -->|"separates vocals"| node_audio
node_lip_masks -->|"prepares masks"| node_dataset
node_dataset -->|"loads audio"| node_audio

click node_app "https://github.com/francis-rings/stableavatar/blob/main/app.py"
click node_inference "https://github.com/francis-rings/stableavatar/blob/main/inference.py"
click node_longpipe "https://github.com/francis-rings/stableavatar/blob/main/wan/pipeline/wan_inference_long_pipeline.py"
click node_fantasypipe "https://github.com/francis-rings/stableavatar/blob/main/wan/pipeline/wan_inference_pipeline_fantasy.py"
click node_text2video "https://github.com/francis-rings/stableavatar/blob/main/wan/text2video.py"
click node_image2video "https://github.com/francis-rings/stableavatar/blob/main/wan/image2video.py"
click node_transformer "https://github.com/francis-rings/stableavatar/blob/main/wan/models/wan_fantasy_transformer3d_1B.py"
click node_vocal "https://github.com/francis-rings/stableavatar/blob/main/wan/models/vocal_projector_fantasy_1B.py"
click node_wav2vec "https://github.com/francis-rings/stableavatar/blob/main/wan/models/wav2vec.py"
click node_textencoder "https://github.com/francis-rings/stableavatar/blob/main/wan/models/wan_text_encoder.py"
click node_imageencoder "https://github.com/francis-rings/stableavatar/blob/main/wan/models/wan_image_encoder.py"
click node_vae "https://github.com/francis-rings/stableavatar/blob/main/wan/models/wan_vae.py"
click node_motion "https://github.com/francis-rings/stableavatar/blob/main/wan/models/motion_controller.py"
click node_dataset "https://github.com/francis-rings/stableavatar/blob/main/wan/dataset/talking_video_dataset_fantasy.py"
click node_training_scripts "https://github.com/francis-rings/stableavatar/blob/main/train_1B_square.py"
click node_audio_extract "https://github.com/francis-rings/stableavatar/blob/main/audio_extractor.py"
click node_vocal_separation "https://github.com/francis-rings/stableavatar/blob/main/vocal_seperator.py"
click node_lip_masks "https://github.com/francis-rings/stableavatar/blob/main/lip_mask_extractor.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_app,node_inference,node_longpipe,node_fantasypipe,node_text2video,node_image2video,node_user toneBlue
class node_transformer,node_vocal,node_wav2vec,node_textencoder,node_imageencoder,node_vae,node_motion toneAmber
class node_audio_extract,node_vocal_separation,node_lip_masks toneMint
class node_dataset,node_training_scripts toneRose
class node_video,node_audio,node_reference,node_prompt toneIndigo
```

## Components

### Video generation

- [Gradio interface](https://github.com/francis-rings/stableavatar/blob/main/app.py): `app.py`
- [Inference runner](https://github.com/francis-rings/stableavatar/blob/main/inference.py): `inference.py`
- [Long-video pipeline](https://github.com/francis-rings/stableavatar/blob/main/wan/pipeline/wan_inference_long_pipeline.py): `wan/pipeline/wan_inference_long_pipeline.py`
- [Fantasy pipeline](https://github.com/francis-rings/stableavatar/blob/main/wan/pipeline/wan_inference_pipeline_fantasy.py): `wan/pipeline/wan_inference_pipeline_fantasy.py`
- [Text-to-video runner](https://github.com/francis-rings/stableavatar/blob/main/wan/text2video.py): `wan/text2video.py`
- [Image-to-video runner](https://github.com/francis-rings/stableavatar/blob/main/wan/image2video.py): `wan/image2video.py`

### Model components

- [Video transformer](https://github.com/francis-rings/stableavatar/blob/main/wan/models/wan_fantasy_transformer3d_1B.py): `wan/models/wan_fantasy_transformer3d_1B.py`
- [Vocal projector](https://github.com/francis-rings/stableavatar/blob/main/wan/models/vocal_projector_fantasy_1B.py): `wan/models/vocal_projector_fantasy_1B.py`
- [Audio encoder](https://github.com/francis-rings/stableavatar/blob/main/wan/models/wav2vec.py): `wan/models/wav2vec.py`
- [Text encoder](https://github.com/francis-rings/stableavatar/blob/main/wan/models/wan_text_encoder.py): `wan/models/wan_text_encoder.py`
- [Image encoder](https://github.com/francis-rings/stableavatar/blob/main/wan/models/wan_image_encoder.py): `wan/models/wan_image_encoder.py`
- [Video VAE](https://github.com/francis-rings/stableavatar/blob/main/wan/models/wan_vae.py): `wan/models/wan_vae.py`
- [Motion controller](https://github.com/francis-rings/stableavatar/blob/main/wan/models/motion_controller.py): `wan/models/motion_controller.py`

### Audio preparation

- [Audio extraction](https://github.com/francis-rings/stableavatar/blob/main/audio_extractor.py): `audio_extractor.py`
- [Vocal separation](https://github.com/francis-rings/stableavatar/blob/main/vocal_seperator.py): `vocal_seperator.py`
- [Lip-mask extraction](https://github.com/francis-rings/stableavatar/blob/main/lip_mask_extractor.py): `lip_mask_extractor.py`

### Training

- [Talking-video dataset](https://github.com/francis-rings/stableavatar/blob/main/wan/dataset/talking_video_dataset_fantasy.py): `wan/dataset/talking_video_dataset_fantasy.py`
- [Training and LoRA](https://github.com/francis-rings/stableavatar/blob/main/train_1B_square.py): `train_1B_square.py`

### Other components

- **User**
- **Generated video**
- **Speech audio**
- **Reference image**
- **Prompt**

## Connections

- User → Gradio interface: provides inputs
- Gradio interface → Long-video pipeline: runs generation
- Inference runner → Long-video pipeline: runs generation
- Long-video pipeline → Text encoder: encodes prompt
- Long-video pipeline → Image encoder: encodes image
- Long-video pipeline → Audio encoder: encodes audio
- Long-video pipeline → Video transformer: denoises latents
- Long-video pipeline → Vocal projector: projects audio
- Long-video pipeline → Video VAE: decodes frames
- Long-video pipeline → Generated video: produces
- Gradio interface → Speech audio: reads audio
- Gradio interface → Reference image: reads image
- Gradio interface → Prompt: reads prompt
- Gradio interface → Generated video: muxes audio
- Text-to-video runner → Fantasy pipeline: uses pipeline
- Image-to-video runner → Video transformer: runs model
- Talking-video dataset → Training and LoRA: supplies samples
- Training and LoRA → Video transformer: trains model
- Audio extraction → Speech audio: extracts tracks
- Vocal separation → Speech audio: separates vocals
- Lip-mask extraction → Talking-video dataset: prepares masks
- Talking-video dataset → Speech audio: loads audio