# jianchang512/pyvideotrans architecture

> Architecture diagram of the GitHub repository jianchang512/pyvideotrans, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-03-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jianchang512/pyvideotrans
- Repository: https://github.com/jianchang512/pyvideotrans

## Overview

pyVideoTrans is a desktop-first media processing application, with a secondary command-line entry point for batch/headless use. Its architecture is organized around a translation/dubbing pipeline that turns input video or audio into subtitles, translated text, synthesized speech, and finally rebuilt media.

At a high level, the repo splits into five important layers:

##### Task orchestration layer: pipeline control and worker execution

The `videotrans/task/` package is the central orchestrator. It contains the operational workflow controllers for different use cases:

- `trans_create.py`: likely creates the translation workflow
- `_speech2text.py`: ASR / transcription tasks
- `_translate_srt.py`: subtitle translation tasks
- `_dubbing.py`: TTS and dubbing workflow
- `_mult_video.py`: batch or multi-video processing
- `_only_one.py`: single-job workflow
- `job.py`, `taskcfg.py`, `_base.py`: job model and shared task configuration
- `simple_runnable_qt.py`, `separate_worker.py`: background execution and worker isolation
- `vad.py`: voice activity detection stage
- `check_update.py`, `update_ffmpeg.py`: operational support tasks

This layer is the process manager for the whole system. It receives configuration from UI or CLI, then coordinates the pipeline stages, progress reporting, retries, and cancellation.

##### Media-processing / pipeline layer: audio, subtitles, and muxing

The core content pipeline is media-centric rather than web-centric.

Important modules include:

- `videotrans/process/prepare_audio.py`
- `videotrans/process/stt_fun.py`
- `videotrans/process/tts_fun.py`
- `videotrans/task/vad.py`

These modules handle the “data plane” of the app:

- Extract or prepare audio from video
- Segment audio using VAD or alignment logic
- Run speech recognition
- Generate or refine subtitle timing
- Translate subtitle text
- Synthesize speech for dubbing
- Recombine audio/video/subtitles

The README’s main workflow is accurate for the architecture:

Speech recognition -> subtitle translation -> speech synthesis -> video synthesis

The pipeline supports several variants:

- pure subtitle generation from audio/video
- subtitle translation without dubbing
- full dubbing / re-voicing
- multi-speaker dubbing with role assignment
- voice cloning scenarios

The repo also includes utility-style media tools under `videotrans/ui/` and `videotrans/winform/` for:
- vocal separation
- subtitle merging
- format conversion
- watermarking
- audio-video alignment
- subtitle-video embedding

Those utilities are not separate applications; they are alternate front doors into the same processing stack.

##### Provider adapter layer: ASR, translation, and TTS abstractions

A major architectural feature of this repository is its provider abstraction. The codebase normalizes many third-party APIs and local models behind common interfaces.

##### ASR / recognition
The `videotrans/recognition/` package contains the speech-to-text adapters, including local and cloud backends such as:

- Whisper / Faster-Whisper
- WhisperX
- Parakeet
- FunASR
- Google
- Deepgram
- Gemini
- Qwen ASR variants
- ByteDance / Volcano / Zijie-related models
- OpenAI-style recognition APIs
- HuggingFace-backed local models

There is a `recognition/_base.py` and `recognition/_overall.py`, which suggests a shared contract plus provider selection/routing.

##### Translation
The `videotrans/translator/` package provides translation adapters, including:

- ChatGPT / OpenAI-compatible APIs
- DeepSeek
- Gemini
- Azure / Microsoft
- Google
- Baidu
- Tencent
- LibreTranslate
- Ollama / local LLM paths
- MiniMax
- Qwen
- SiliconFlow / OpenRouter / other model gateways

The presence of `translator/_base.py`, `_transapi.py`, and many provider-specific modules indicates a pluggable translation interface. This is a key boundary: the rest of the system should not care whether translation comes from a cloud LLM, a local model, or a classic MT service.

##### TTS / dubbing
The `videotrans/tts/` package contains speech synthesis adapters, such as:

- Edge TTS
- Azure TTS
- OpenAI-compatible TTS
- ElevenLabs
- Google Cloud TTS
- MiniMax
- Qwen TTS
- Coqui/CosyVoice/F5-TTS/GPT-SoVITS-style voice cloning or local synthesis options
- Piper, VITS, ChatTTS, ChatterBox, and other local models

Like the other provider layers, this package has a common base and per-provider implementations.

##### Configuration and startup
The `videotrans/configure/` package contains application state and environment handling:

- `config.py`: core configuration
- `_base.py`, `_except.py`, `_guiexcept.py`: shared setup and exception handling
- `whispernet_config.py`: WhisperNet-specific settings

This is the likely source of runtime defaults, API keys, proxy settings, GPU preferences, and app-wide state.

##### Localization and UI assets
The repo contains:

- `videotrans/language/en.json`
- `videotrans/language/zh.json`

So the application is localized at least in English and Chinese.

It also contains:

- `videotrans/styles/style.qss`
- `videotrans/styles/no-remove.*`

This suggests a themed Qt UI with embedded style assets and bundled media resources.

##### Voice/model metadata
The `videotrans/voicejson/` directory contains provider voice catalogs and model metadata for services such as Azure, ElevenLabs, Edge TTS, MiniMax, Piper, Qwen, and others.

These JSON files are part of the provider UI/configuration layer and likely drive selectable voices and model lists.

##### External binaries and environment dependencies
FFmpeg is a first-class system dependency. The repository includes helper files like:

- `ffmpeg.txt`
- `update_ffmpeg.bat`
- `run-cuda.bat`

This indicates the app expects local FFmpeg availability and offers platform-specific setup helpers. FFmpeg is central to extracting audio, cutting clips, muxing output, and subtitle/video handling.

CUDA support is also explicitly part of the deployment story. The README and files like `testcuda.py` indicate that the app can run with GPU acceleration when supported models or PyTorch builds are installed.

##### End-to-end data flow

A typical full translation job moves through these boundaries:

##### UI/config layer captures
- input media path
- source/target language
- ASR provider
- translation provider
- TTS provider
- voice/role settings
- output and alignment preferences
##### Progress and results are surfaced back to the UI or CLI output.

That flow is the main architectural spine of the repository.

##### Important boundaries to capture

- UI boundary: `mainwin`, `ui`, `winform`, `component`
- Orchestration boundary: `task`
- Media pipeline boundary: `process`
- ASR boundary: `recognition`
- Translation boundary: `translator`
- TTS boundary: `tts`
- System/asset boundary: `configure`, `util`, `styles`, `language`, `voicejson`, FFmpeg, model files

The codebase is therefore best understood as a modular media-processing platform with pluggable AI providers, not as a traditional single-purpose application or web service.

Main entry points: [`videotrans/task`](https://github.com/jianchang512/pyvideotrans/tree/HEAD/videotrans/task) (Task Core), [`videotrans/configure`](https://github.com/jianchang512/pyvideotrans/tree/HEAD/videotrans/configure) (Config), [`videotrans/mainwin`](https://github.com/jianchang512/pyvideotrans/tree/HEAD/videotrans/mainwin) (Main Window), [`videotrans/recognition`](https://github.com/jianchang512/pyvideotrans/tree/HEAD/videotrans/recognition) (ASR Adapters)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph "Presentation"
  sp_py["GUI App<br/>entrypoint<br/>[sp.py]"]
  cli_py["CLI App<br/>entrypoint<br/>[cli.py]"]
  mainwin["Main Window<br/>ui shell"]
  ui_forms["Feature UI<br/>qt screens"]
  widgets["Shared Widgets<br/>ui components"]
end

subgraph "Task Orchestration"
  task_core["Task Core<br/>orchestrator"]
  workers["Workers<br/>execution"]
  job_model["Job Model<br/>job config<br/>[job.py]"]
end

subgraph "Media Pipeline"
  prepare_audio["Audio Prep<br/>media prep<br/>[prepare_audio.py]"]
  vad_stage["VAD Stage<br/>segmentation<br/>[vad.py]"]
  stt_stage["Speech-to-Text<br/>pipeline stage<br/>[stt_fun.py]"]
  translate_stage["Subtitle Translate<br/>pipeline stage<br/>[_translate_srt.py]"]
  tts_stage["Dubbing<br/>pipeline stage<br/>[tts_fun.py]"]
  dubbing_task["Dubbing Task<br/>workflow<br/>[_dubbing.py]"]
  speech_tasks["STT Tasks<br/>workflow<br/>[_speech2text.py]"]
  translation_tasks["Translation Tasks<br/>workflow<br/>[trans_create.py]"]
end

subgraph "Provider Adapters"
  recognition["ASR Adapters<br/>provider layer"]
  translator["MT Adapters<br/>provider layer"]
  tts["TTS Adapters<br/>provider layer"]
end

subgraph "Shared Runtime"
  config["Config<br/>runtime state"]
  util["Utilities<br/>helpers"]
  assets["Assets<br/>localization/resources"]
end

sp_py -->|"launches"| mainwin
cli_py -->|"dispatches"| task_core
mainwin -->|"opens"| ui_forms
mainwin -->|"uses"| widgets
ui_forms -->|"submits jobs"| task_core
widgets -->|"binds settings"| config
task_core -->|"runs via"| workers
task_core -->|"tracks"| job_model
task_core -->|"starts"| prepare_audio
task_core -->|"selects"| speech_tasks
task_core -->|"selects"| translation_tasks
task_core -->|"selects"| dubbing_task
prepare_audio -->|"feeds"| vad_stage
vad_stage -->|"chunks for"| recognition
recognition -->|"produces"| stt_stage
stt_stage -->|"hands off"| translator
translator -->|"returns text"| translate_stage
translate_stage -->|"requests audio"| tts
tts -->|"outputs"| tts_stage
tts_stage -->|"uses"| util
task_core -->|"reads"| config
task_core -->|"loads"| assets
recognition -.->|"configured by"| config
translator -.->|"configured by"| config
tts -.->|"configured by"| config

click sp_py "https://github.com/jianchang512/pyvideotrans/blob/main/sp.py"
click cli_py "https://github.com/jianchang512/pyvideotrans/blob/main/cli.py"
click mainwin "https://github.com/jianchang512/pyvideotrans/tree/main/videotrans/mainwin"
click ui_forms "https://github.com/jianchang512/pyvideotrans/tree/main/videotrans/ui"
click widgets "https://github.com/jianchang512/pyvideotrans/tree/main/videotrans/component"
click task_core "https://github.com/jianchang512/pyvideotrans/tree/main/videotrans/task"
click workers "https://github.com/jianchang512/pyvideotrans/blob/main/videotrans/task/simple_runnable_qt.py"
click job_model "https://github.com/jianchang512/pyvideotrans/blob/main/videotrans/task/job.py"
click prepare_audio "https://github.com/jianchang512/pyvideotrans/blob/main/videotrans/process/prepare_audio.py"
click vad_stage "https://github.com/jianchang512/pyvideotrans/blob/main/videotrans/task/vad.py"
click stt_stage "https://github.com/jianchang512/pyvideotrans/blob/main/videotrans/process/stt_fun.py"
click translate_stage "https://github.com/jianchang512/pyvideotrans/blob/main/videotrans/task/_translate_srt.py"
click tts_stage "https://github.com/jianchang512/pyvideotrans/blob/main/videotrans/process/tts_fun.py"
click dubbing_task "https://github.com/jianchang512/pyvideotrans/blob/main/videotrans/task/_dubbing.py"
click speech_tasks "https://github.com/jianchang512/pyvideotrans/blob/main/videotrans/task/_speech2text.py"
click translation_tasks "https://github.com/jianchang512/pyvideotrans/blob/main/videotrans/task/trans_create.py"
click recognition "https://github.com/jianchang512/pyvideotrans/tree/main/videotrans/recognition"
click translator "https://github.com/jianchang512/pyvideotrans/tree/main/videotrans/translator"
click tts "https://github.com/jianchang512/pyvideotrans/tree/main/videotrans/tts"
click config "https://github.com/jianchang512/pyvideotrans/tree/main/videotrans/configure"
click util "https://github.com/jianchang512/pyvideotrans/tree/main/videotrans/util"
click assets "https://github.com/jianchang512/pyvideotrans/tree/main/videotrans/language"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class sp_py,cli_py,mainwin,ui_forms,widgets toneBlue
class task_core,workers,job_model toneAmber
class prepare_audio,vad_stage,stt_stage,translate_stage,tts_stage,dubbing_task,speech_tasks,translation_tasks toneMint
class recognition,translator,tts toneRose
class config,util,assets toneIndigo
```

## Components

### Presentation

Desktop UI and command-line entry points

- [GUI App](https://github.com/jianchang512/pyvideotrans/blob/HEAD/sp.py): entrypoint · Qt desktop launcher · `sp.py`
- [CLI App](https://github.com/jianchang512/pyvideotrans/blob/HEAD/cli.py): entrypoint · Headless batch entrypoint · `cli.py`
- [Main Window](https://github.com/jianchang512/pyvideotrans/tree/HEAD/videotrans/mainwin): ui shell · Application shell, actions, and signals · `videotrans/mainwin`
- [Feature UI](https://github.com/jianchang512/pyvideotrans/tree/HEAD/videotrans/ui): qt screens · Provider and tool configuration screens · `videotrans/ui`
- [Shared Widgets](https://github.com/jianchang512/pyvideotrans/tree/HEAD/videotrans/component): ui components · Reusable dialogs, panels, and controls · `videotrans/component`

### Task Orchestration

Job control and background execution

- [Task Core](https://github.com/jianchang512/pyvideotrans/tree/HEAD/videotrans/task): orchestrator · Job creation, scheduling, and task control · `videotrans/task`
- [Workers](https://github.com/jianchang512/pyvideotrans/blob/HEAD/videotrans/task/simple_runnable_qt.py): execution · Background runnable and isolated workers · `videotrans/task/simple_runnable_qt.py`
- [Job Model](https://github.com/jianchang512/pyvideotrans/blob/HEAD/videotrans/task/job.py): job config · Shared job state and task configuration · `videotrans/task/job.py`

### Media Pipeline

Audio, subtitle, and recomposition stages

- [Audio Prep](https://github.com/jianchang512/pyvideotrans/blob/HEAD/videotrans/process/prepare_audio.py): media prep · Extracts and prepares source audio · `videotrans/process/prepare_audio.py`
- [VAD Stage](https://github.com/jianchang512/pyvideotrans/blob/HEAD/videotrans/task/vad.py): segmentation · Voice activity detection and chunking · `videotrans/task/vad.py`
- [Speech-to-Text](https://github.com/jianchang512/pyvideotrans/blob/HEAD/videotrans/process/stt_fun.py): pipeline stage · Recognition and subtitle generation · `videotrans/process/stt_fun.py`
- [Subtitle Translate](https://github.com/jianchang512/pyvideotrans/blob/HEAD/videotrans/task/_translate_srt.py): pipeline stage · Subtitle rewriting and translation flow · `videotrans/task/_translate_srt.py`
- [Dubbing](https://github.com/jianchang512/pyvideotrans/blob/HEAD/videotrans/process/tts_fun.py): pipeline stage · Speech synthesis and voice assignment · `videotrans/process/tts_fun.py`
- [Dubbing Task](https://github.com/jianchang512/pyvideotrans/blob/HEAD/videotrans/task/_dubbing.py): workflow · Full re-voicing and rebuild workflow · `videotrans/task/_dubbing.py`
- [STT Tasks](https://github.com/jianchang512/pyvideotrans/blob/HEAD/videotrans/task/_speech2text.py): workflow · Recognition-oriented task variants · `videotrans/task/_speech2text.py`
- [Translation Tasks](https://github.com/jianchang512/pyvideotrans/blob/HEAD/videotrans/task/trans_create.py): workflow · Translation-oriented task variants · `videotrans/task/trans_create.py`

### Provider Adapters

Pluggable ASR, translation, and TTS backends

- [ASR Adapters](https://github.com/jianchang512/pyvideotrans/tree/HEAD/videotrans/recognition): provider layer · Speech recognition backends · `videotrans/recognition`
- [MT Adapters](https://github.com/jianchang512/pyvideotrans/tree/HEAD/videotrans/translator): provider layer · Translation backends and model gateways · `videotrans/translator`
- [TTS Adapters](https://github.com/jianchang512/pyvideotrans/tree/HEAD/videotrans/tts): provider layer · Speech synthesis and voice cloning backends · `videotrans/tts`

### Shared Runtime

Configuration, utilities, localization, and assets

- [Config](https://github.com/jianchang512/pyvideotrans/tree/HEAD/videotrans/configure): runtime state · Defaults, exceptions, and environment setup · `videotrans/configure`
- [Utilities](https://github.com/jianchang512/pyvideotrans/tree/HEAD/videotrans/util): helpers · FFmpeg, GPU, download, and text helpers · `videotrans/util`
- [Assets](https://github.com/jianchang512/pyvideotrans/tree/HEAD/videotrans/language): localization/resources · Language packs, styles, prompts, and voice metadata · `videotrans/language`

## Connections

- GUI App → Main Window: launches · GUI starts the application shell
- CLI App → Task Core: dispatches · CLI sends work directly into orchestration
- Main Window → Feature UI: opens · Main window routes to feature screens
- Main Window → Shared Widgets: uses · Shared dialogs and controls support the shell
- Feature UI → Task Core: submits jobs · UI collects parameters and starts tasks
- Shared Widgets → Config: binds settings · Controls reflect app and provider configuration
- Task Core → Workers: runs via · Tasks execute in background workers
- Task Core → Job Model: tracks · Job metadata and runtime state
- Task Core → Audio Prep: starts · Orchestration invokes media preparation
- Task Core → STT Tasks: selects · Task variants for recognition workflows
- Task Core → Translation Tasks: selects · Task variants for translation workflows
- Task Core → Dubbing Task: selects · Task variants for dubbing workflows
- Audio Prep → VAD Stage: feeds · Prepared audio enters segmentation
- VAD Stage → ASR Adapters: chunks for · Speech chunks are sent to ASR backends
- ASR Adapters → Speech-to-Text: produces · Recognition results drive subtitle generation
- Speech-to-Text → MT Adapters: hands off · Subtitle text is translated through adapters
- MT Adapters → Subtitle Translate: returns text · Translated subtitles flow back to pipeline
- Subtitle Translate → TTS Adapters: requests audio · Translated lines are synthesized as speech
- TTS Adapters → Dubbing: outputs · Synthesis feeds dubbing and recomposition
- Dubbing → Utilities: uses · Media assembly relies on shared helpers and FFmpeg support
- Task Core → Config: reads · Global runtime settings and exception handling
- Task Core → Assets: loads · Localization, prompts, styles, and voice catalogs
- ASR Adapters → Config: configured by · ASR adapters consume runtime configuration
- MT Adapters → Config: configured by · Translation adapters consume runtime configuration
- TTS Adapters → Config: configured by · TTS adapters consume runtime configuration
