How heygen-com/tavr works
TAVR is a video-reference talking-avatar inference system: a sample supplies a person’s reference clip, target-scene still and caption, and driving audio; inference conditions video generation on those inputs and writes an MP4. The graph preserves media preparation, reference identity/pose, audio and text conditioning, latent sampling, and decoding. Several pipeline and model internals are represented from the README and module boundaries; omitted source excerpts limit some wiring, so uncertain calls are not asserted.
Main entry points: pipeline.py Inference pipeline · media.py Media handling · reference.py Reference preparation · infer.py CLI entry
Components 17
Inference interface
- CLI entry
infer.py - Inference pipeline
tavr/pipeline.py - Data contracts
tavr/contract.py
Input preparation
- Media handling
tavr/preprocess/media.py - Reference preparation
tavr/preprocess/reference.py - Person and pose
tavr/preprocess/masks.py - Audio features
tavr/preprocess/audio.py
Video generation
- Latent sampling
tavr/sampling.py
Model components
- Video transformer
tavr/models/dit.py - Text encoder
tavr/models/text.py - Wan VAE
tavr/models/vae.py
Other components
- Inference user
- Sample files
- Pretrained weights
- YOLO and DWPose
- Wav2Vec2
- Generated video
Connections 20
- Inference user → CLI entry: starts inference
- Sample files → CLI entry: provides sample
- Pretrained weights → CLI entry: provides checkpoints
- CLI entry → Inference pipeline: runs sample
- Inference pipeline → Data contracts: uses data types
- Inference pipeline → Media handling: reads still
- Media handling → Inference pipeline: returns still
- Inference pipeline → Reference preparation: prepares reference
- Reference preparation → Media handling: processes video
- Reference preparation → Person and pose: extracts pose
- Reference preparation → Audio features: extracts reference audio
- Person and pose → YOLO and DWPose: runs detection
- Audio features → Wav2Vec2: encodes speech
- Inference pipeline → Audio features: builds driving features
- Inference pipeline → Text encoder: encodes prompt
- Inference pipeline → Latent sampling: requests latents
- Latent sampling → Video transformer: denoises latents
- Inference pipeline → Wan VAE: decodes latents
- Inference pipeline → Media handling: writes video
- Media handling → Generated video: produces MP4