# heygen-com/tavr architecture

> Architecture diagram of the GitHub repository heygen-com/tavr, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/heygen-com/tavr
- Repository: https://github.com/heygen-com/tavr

## Overview

TAVR is a video-reference talking-avatar inference system: a sample supplies a person’s reference clip, target-scene still and caption, and driving audio; inference conditions video generation on those inputs and writes an MP4. The graph preserves media preparation, reference identity/pose, audio and text conditioning, latent sampling, and decoding. Several pipeline and model internals are represented from the README and module boundaries; omitted source excerpts limit some wiring, so uncertain calls are not asserted.

Main entry points: [`tavr/pipeline.py`](https://github.com/heygen-com/tavr/blob/main/tavr/pipeline.py) (Inference pipeline), [`tavr/preprocess/media.py`](https://github.com/heygen-com/tavr/blob/main/tavr/preprocess/media.py) (Media handling), [`tavr/preprocess/reference.py`](https://github.com/heygen-com/tavr/blob/main/tavr/preprocess/reference.py) (Reference preparation), [`infer.py`](https://github.com/heygen-com/tavr/blob/main/infer.py) (CLI entry)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interface["Inference interface"]
  node_infer["CLI entry<br/>[infer.py]"]
  node_pipeline["Inference pipeline<br/>[pipeline.py]"]
  node_contract["Data contracts<br/>[contract.py]"]
end

subgraph group_prep["Input preparation"]
  node_media["Media handling<br/>[media.py]"]
  node_reference["Reference preparation<br/>[reference.py]"]
  node_masks["Person and pose<br/>[masks.py]"]
  node_audio["Audio features<br/>[audio.py]"]
end

subgraph group_generation["Video generation"]
  node_sampling["Latent sampling<br/>[sampling.py]"]
end

subgraph group_models["Model components"]
  node_dit["Video transformer<br/>[dit.py]"]
  node_text["Text encoder<br/>[text.py]"]
  node_vae["Wan VAE<br/>[vae.py]"]
end

node_user(("Inference user"))
node_inputs["Sample files"]
node_weights["Pretrained weights"]
node_detectors["YOLO and DWPose"]
node_wav2vec["Wav2Vec2"]
node_output["Generated video"]

node_user -->|"starts inference"| node_infer
node_inputs -->|"provides sample"| node_infer
node_weights -->|"provides checkpoints"| node_infer
node_infer -->|"runs sample"| node_pipeline
node_pipeline -->|"uses data types"| node_contract
node_pipeline -->|"reads still"| node_media
node_media -->|"returns still"| node_pipeline
node_pipeline -->|"prepares reference"| node_reference
node_reference -->|"processes video"| node_media
node_reference -->|"extracts pose"| node_masks
node_reference -->|"extracts reference audio"| node_audio
node_masks -->|"runs detection"| node_detectors
node_audio -->|"encodes speech"| node_wav2vec
node_pipeline -->|"builds driving features"| node_audio
node_pipeline -->|"encodes prompt"| node_text
node_pipeline -->|"requests latents"| node_sampling
node_sampling -->|"denoises latents"| node_dit
node_pipeline -->|"decodes latents"| node_vae
node_pipeline -->|"writes video"| node_media
node_media -->|"produces MP4"| node_output

click node_infer "https://github.com/heygen-com/tavr/blob/main/infer.py"
click node_pipeline "https://github.com/heygen-com/tavr/blob/main/tavr/pipeline.py"
click node_contract "https://github.com/heygen-com/tavr/blob/main/tavr/contract.py"
click node_media "https://github.com/heygen-com/tavr/blob/main/tavr/preprocess/media.py"
click node_reference "https://github.com/heygen-com/tavr/blob/main/tavr/preprocess/reference.py"
click node_masks "https://github.com/heygen-com/tavr/blob/main/tavr/preprocess/masks.py"
click node_audio "https://github.com/heygen-com/tavr/blob/main/tavr/preprocess/audio.py"
click node_sampling "https://github.com/heygen-com/tavr/blob/main/tavr/sampling.py"
click node_dit "https://github.com/heygen-com/tavr/blob/main/tavr/models/dit.py"
click node_text "https://github.com/heygen-com/tavr/blob/main/tavr/models/text.py"
click node_vae "https://github.com/heygen-com/tavr/blob/main/tavr/models/vae.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_infer,node_pipeline,node_contract,node_user toneBlue
class node_media,node_reference,node_masks,node_audio toneAmber
class node_sampling toneMint
class node_dit,node_text,node_vae toneRose
class node_inputs,node_weights,node_detectors,node_wav2vec,node_output toneIndigo
```

## Components

### Inference interface

- [CLI entry](https://github.com/heygen-com/tavr/blob/main/infer.py): `infer.py`
- [Inference pipeline](https://github.com/heygen-com/tavr/blob/main/tavr/pipeline.py): `tavr/pipeline.py`
- [Data contracts](https://github.com/heygen-com/tavr/blob/main/tavr/contract.py): `tavr/contract.py`

### Input preparation

- [Media handling](https://github.com/heygen-com/tavr/blob/main/tavr/preprocess/media.py): `tavr/preprocess/media.py`
- [Reference preparation](https://github.com/heygen-com/tavr/blob/main/tavr/preprocess/reference.py): `tavr/preprocess/reference.py`
- [Person and pose](https://github.com/heygen-com/tavr/blob/main/tavr/preprocess/masks.py): `tavr/preprocess/masks.py`
- [Audio features](https://github.com/heygen-com/tavr/blob/main/tavr/preprocess/audio.py): `tavr/preprocess/audio.py`

### Video generation

- [Latent sampling](https://github.com/heygen-com/tavr/blob/main/tavr/sampling.py): `tavr/sampling.py`

### Model components

- [Video transformer](https://github.com/heygen-com/tavr/blob/main/tavr/models/dit.py): `tavr/models/dit.py`
- [Text encoder](https://github.com/heygen-com/tavr/blob/main/tavr/models/text.py): `tavr/models/text.py`
- [Wan VAE](https://github.com/heygen-com/tavr/blob/main/tavr/models/vae.py): `tavr/models/vae.py`

### Other components

- **Inference user**
- **Sample files**
- **Pretrained weights**
- **YOLO and DWPose**
- **Wav2Vec2**
- **Generated video**

## Connections

- Inference user → CLI entry: starts inference
- Sample files → CLI entry: provides sample
- Pretrained weights → CLI entry: provides checkpoints
- CLI entry → Inference pipeline: runs sample
- Inference pipeline → Data contracts: uses data types
- Inference pipeline → Media handling: reads still
- Media handling → Inference pipeline: returns still
- Inference pipeline → Reference preparation: prepares reference
- Reference preparation → Media handling: processes video
- Reference preparation → Person and pose: extracts pose
- Reference preparation → Audio features: extracts reference audio
- Person and pose → YOLO and DWPose: runs detection
- Audio features → Wav2Vec2: encodes speech
- Inference pipeline → Audio features: builds driving features
- Inference pipeline → Text encoder: encodes prompt
- Inference pipeline → Latent sampling: requests latents
- Latent sampling → Video transformer: denoises latents
- Inference pipeline → Wan VAE: decodes latents
- Inference pipeline → Media handling: writes video
- Media handling → Generated video: produces MP4