How tayyabsiddique77/ai-dubbing-studio works
AI Dubbing Studio accepts a video and target language, then returns a dubbed video. The documented workflow is implemented by a FastAPI entry point and a pipeline that extracts audio, transcribes and optionally diarizes speech, builds a speaker-labeled timeline, then translates, analyzes, synthesizes, synchronizes, and remuxes the result. The graph follows calls shown in the excerpts; optional diarization is explicit. The README documents the studio UI, while its implementation was not sampled.
Main entry points: pipeline.py Master Pipeline · audio_ops.py Audio Processing · main.py Upload API · index.html Studio UI
Components 16
Studio Interface
- Studio UI
index.html - Upload API
main.py
Pipeline Orchestration
- Master Pipeline
pipeline.py - Audio Processing
utils/audio_ops.py
Speech Analysis
- Transcription
models/asr.py - Speaker Diarization
models/diarization.py - Speaker Timeline
utils/segment_utils.py - Prosody Analysis
models/emotion_prosody.py
Dub Production
- Translation
models/translation.py - Voice Synthesis
models/emotion_transfer.py - Voice Cloning
models/voice_cloning.py - Standard TTS
models/tts.py - Audio Synchronization
models/sync_engine.py
Other components
- Video Producer
- FFmpeg
- Dubbed Video
Connections 17
- Video Producer → Studio UI: opens studio
README.md - Studio UI → Upload API: uploads video
- Upload API → Master Pipeline: processes video
main.py - Master Pipeline → Audio Processing: extracts audio
pipeline.py - Master Pipeline → Transcription: transcribes audio
pipeline.py - Master Pipeline → Speaker Diarization: optionally diarizes
pipeline.py - Master Pipeline → Speaker Timeline: merges segments
pipeline.py - Master Pipeline → Translation: translates dialogue
pipeline.py - Master Pipeline → Audio Processing: crops references
pipeline.py - Master Pipeline → Prosody Analysis: analyzes prosody
pipeline.py - Master Pipeline → Voice Synthesis: generates speech
pipeline.py - Master Pipeline → Audio Synchronization: fits segments
pipeline.py - Master Pipeline → Audio Synchronization: stitches timeline
pipeline.py - Master Pipeline → Audio Processing: merges dubbed audio
pipeline.py - Audio Processing → FFmpeg: runs media commands
utils/audio_ops.py - Master Pipeline → Dubbed Video: returns video path
pipeline.py - Upload API → Dubbed Video: serves video
main.py