How brasd99/heygenclone works
HeyGenClone translates a video’s speech and produces a dubbed video; it also offers a separate voice-driven lip-sync workflow. The documented translation stages cover scene and face analysis, speech cleanup, transcription and speaker attribution, translation, voice cloning, video lip-sync, face restoration, and audio/video assembly. The graph emphasizes the translation orchestrator and major runtime subsystems. Some stage links are based on README documentation because the relevant engine code was not fully excerpted.
Main entry points: engine.py Translation engine · scene_preprocessor.py Scene and face analysis · lipsync.py Lip-sync rendering · translate.py Translation CLI
Components 20
Entry and orchestration
- Translation CLI
translate.py - Translation engine
core/engine.py - Temporary files
core/temp_manager.py
Scene and speech analysis
- Scene and face analysis
core/scene_preprocessor.py - Face detector
core/face/detector.py - Face records
core/scene_preprocessor.py - Speech cleanup
core/dereverb.py - Speech recognition
core/whisperx/asr.py - Word alignment
core/whisperx/alignment.py - Speaker diarization
core/whisperx/diarize.py - Voice activity detection
core/whisperx/vad.py - Subtitle processing
core/whisperx/SubtitlesProcessor.py
Translated speech
- Text translation
core/translator.py - Language mapping
core/mapper.py - Voice cloning
core/voice_cloner.py
Video rendering
- Lip-sync rendering
core/face/lipsync.py - Wav2Lip model
core/face/models/wav2lip.py - Face restoration
core/gfpganer.py
Other components
- Video user
- Dubbed video
Connections 19
- Video user → Translation CLI: provides video
README.md - Translation CLI → Translation engine: starts translation
translate.py - Translation engine → Scene and face analysis: analyzes video
core/engine.py - Scene and face analysis → Face detector: detects faces
core/scene_preprocessor.py - Scene and face analysis → Face records: stores face records
core/scene_preprocessor.py - Translation engine → Speech cleanup: cleans speech
core/engine.py - Translation engine → Speech recognition: transcribes speech
core/engine.py - Speech recognition → Voice activity detection: uses voice detection
core/whisperx/asr.py - Translation engine → Word alignment: aligns transcript
core/engine.py - Translation engine → Speaker diarization: assigns speakers
core/engine.py - Translation engine → Subtitle processing: processes subtitles
README.md - Translation engine → Text translation: translates text
core/engine.py - Text translation → Language mapping: maps languages
core/translator.py - Translation engine → Voice cloning: clones voices
core/engine.py - Translation engine → Lip-sync rendering: renders lip sync
core/engine.py - Lip-sync rendering → Wav2Lip model: runs model
core/face/lipsync.py - Lip-sync rendering → Face restoration: restores faces
core/face/lipsync.py - Translation engine → Temporary files: creates temp files
core/engine.py - Translation engine → Dubbed video: assembles output
README.md