# opentalker/sadtalker architecture

> Architecture diagram of the GitHub repository opentalker/sadtalker, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/opentalker/sadtalker
- Repository: https://github.com/opentalker/sadtalker

## Overview

SadTalker turns a portrait and driving audio into a talking-head video, with reference motion, still/full-image preprocessing, pose controls, and optional enhancement. The main flow runs through the Gradio interface or inference entry point: prepare the portrait, derive audio-conditioned coefficients, assemble render data, animate, and save video. Model internals support 3D face reconstruction, audio-to-expression/pose prediction, and neural rendering. Some orchestration code is absent from the supplied tree or excerpts, so those links are omitted or marked unverified rather than inferred as direct calls.

Main entry points: [`src/facerender/animate.py`](https://github.com/opentalker/sadtalker/blob/main/src/facerender/animate.py) (Animation Orchestrator), [`src/gradio_demo.py`](https://github.com/opentalker/sadtalker/blob/main/src/gradio_demo.py) (Gradio Demo), [`src/generate_batch.py`](https://github.com/opentalker/sadtalker/blob/main/src/generate_batch.py) (Audio Features), [`src/utils/preprocess.py`](https://github.com/opentalker/sadtalker/blob/main/src/utils/preprocess.py) (Portrait Prep)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interface["User Interfaces"]
  node_gradio["Gradio Demo<br/>[gradio_demo.py]"]
  node_cli["Inference CLI<br/>[inference.py]"]
  node_webui_extension["WebUI Extension<br/>[extension.py]"]
end

subgraph group_conditioning["Input Conditioning"]
  node_portrait["Portrait Prep<br/>[preprocess.py]"]
  node_face_recon["3D Face Reconstruction<br/>[networks.py]"]
  node_audio_batch["Audio Features<br/>[generate_batch.py]"]
end

subgraph group_motion["Motion Models"]
  node_audio_model["Audio Expression Model<br/>[audio2exp.py]"]
  node_pose_model["Audio Pose Model<br/>[audio2pose.py]"]
end

subgraph group_rendering["Face Rendering"]
  node_render_batch["Render Data Prep"]
  node_animate["Animation Orchestrator<br/>[animate.py]"]
  node_keypoints["Keypoint Detector"]
  node_generator["Frame Generator<br/>[generator.py]"]
  node_enhancer["Face Enhancer<br/>[face_enhancer.py]"]
end

node_user(("User"))
node_output["Talking Video"]

node_user -->|"submits inputs"| node_gradio
node_user -->|"provides inputs"| node_cli
node_gradio -->|"prepares portrait"| node_portrait
node_cli -->|"prepares portrait"| node_portrait
node_portrait -->|"runs reconstruction"| node_face_recon
node_gradio -->|"builds audio batch"| node_audio_batch
node_audio_batch -.->|"conditions expression"| node_audio_model
node_audio_batch -.->|"conditions pose"| node_pose_model
node_gradio -->|"builds render data"| node_render_batch
node_render_batch -->|"supplies render data"| node_animate
node_animate -->|"constructs detector"| node_keypoints
node_animate -->|"constructs generator"| node_generator
node_animate -->|"optionally enhances"| node_enhancer
node_animate -->|"writes video"| node_output
node_user -.->|"uses extension"| node_webui_extension

click node_gradio "https://github.com/opentalker/sadtalker/blob/main/src/gradio_demo.py"
click node_cli "https://github.com/opentalker/sadtalker/blob/main/inference.py"
click node_portrait "https://github.com/opentalker/sadtalker/blob/main/src/utils/preprocess.py"
click node_face_recon "https://github.com/opentalker/sadtalker/blob/main/src/face3d/models/networks.py"
click node_audio_batch "https://github.com/opentalker/sadtalker/blob/main/src/generate_batch.py"
click node_audio_model "https://github.com/opentalker/sadtalker/blob/main/src/audio2exp_models/audio2exp.py"
click node_pose_model "https://github.com/opentalker/sadtalker/blob/main/src/audio2pose_models/audio2pose.py"
click node_render_batch "https://github.com/opentalker/sadtalker/blob/main/src/generate_facerender_batch.py"
click node_animate "https://github.com/opentalker/sadtalker/blob/main/src/facerender/animate.py"
click node_keypoints "https://github.com/opentalker/sadtalker/blob/main/src/facerender/modules/keypoint_detector.py"
click node_generator "https://github.com/opentalker/sadtalker/blob/main/src/facerender/modules/generator.py"
click node_enhancer "https://github.com/opentalker/sadtalker/blob/main/src/utils/face_enhancer.py"
click node_webui_extension "https://github.com/opentalker/sadtalker/blob/main/scripts/extension.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_gradio,node_cli,node_webui_extension,node_user toneBlue
class node_portrait,node_face_recon,node_audio_batch toneAmber
class node_audio_model,node_pose_model toneMint
class node_render_batch,node_animate,node_keypoints,node_generator,node_enhancer toneRose
class node_output toneIndigo
```

## Components

### User Interfaces

- [Gradio Demo](https://github.com/opentalker/sadtalker/blob/main/src/gradio_demo.py): `src/gradio_demo.py`
- [Inference CLI](https://github.com/opentalker/sadtalker/blob/main/inference.py): `inference.py`
- [WebUI Extension](https://github.com/opentalker/sadtalker/blob/main/scripts/extension.py): `scripts/extension.py`

### Input Conditioning

- [Portrait Prep](https://github.com/opentalker/sadtalker/blob/main/src/utils/preprocess.py): `src/utils/preprocess.py`
- [3D Face Reconstruction](https://github.com/opentalker/sadtalker/blob/main/src/face3d/models/networks.py): `src/face3d/models/networks.py`
- [Audio Features](https://github.com/opentalker/sadtalker/blob/main/src/generate_batch.py): `src/generate_batch.py`

### Motion Models

- [Audio Expression Model](https://github.com/opentalker/sadtalker/blob/main/src/audio2exp_models/audio2exp.py): `src/audio2exp_models/audio2exp.py`
- [Audio Pose Model](https://github.com/opentalker/sadtalker/blob/main/src/audio2pose_models/audio2pose.py): `src/audio2pose_models/audio2pose.py`

### Face Rendering

- [Render Data Prep](https://github.com/opentalker/sadtalker/blob/main/src/generate_facerender_batch.py): `src/generate_facerender_batch.py`
- [Animation Orchestrator](https://github.com/opentalker/sadtalker/blob/main/src/facerender/animate.py): `src/facerender/animate.py`
- [Keypoint Detector](https://github.com/opentalker/sadtalker/blob/main/src/facerender/modules/keypoint_detector.py): `src/facerender/modules/keypoint_detector.py`
- [Frame Generator](https://github.com/opentalker/sadtalker/blob/main/src/facerender/modules/generator.py): `src/facerender/modules/generator.py`
- [Face Enhancer](https://github.com/opentalker/sadtalker/blob/main/src/utils/face_enhancer.py): `src/utils/face_enhancer.py`

### Other components

- **User**
- **Talking Video**

## Connections

- User → Gradio Demo: submits inputs (evidence: `README.md`)
- User → Inference CLI: provides inputs (evidence: `inference.py`)
- Gradio Demo → Portrait Prep: prepares portrait (evidence: `src/gradio_demo.py`)
- Inference CLI → Portrait Prep: prepares portrait (evidence: `inference.py`)
- Portrait Prep → 3D Face Reconstruction: runs reconstruction (evidence: `src/utils/preprocess.py`)
- Gradio Demo → Audio Features: builds audio batch (evidence: `src/gradio_demo.py`)
- Audio Features → Audio Expression Model: conditions expression
- Audio Features → Audio Pose Model: conditions pose
- Gradio Demo → Render Data Prep: builds render data (evidence: `src/gradio_demo.py`)
- Render Data Prep → Animation Orchestrator: supplies render data (evidence: `src/gradio_demo.py`)
- Animation Orchestrator → Keypoint Detector: constructs detector (evidence: `src/facerender/animate.py`)
- Animation Orchestrator → Frame Generator: constructs generator (evidence: `src/facerender/animate.py`)
- Animation Orchestrator → Face Enhancer: optionally enhances (evidence: `src/facerender/animate.py`)
- Animation Orchestrator → Talking Video: writes video (evidence: `src/facerender/animate.py`)
- User → WebUI Extension: uses extension (evidence: `README.md`)
