# aravindan20/voice_clone_usingindic architecture

> Architecture diagram of the GitHub repository aravindan20/voice_clone_usingindic, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-02-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/aravindan20/voice_clone_usingindic
- Repository: https://github.com/aravindan20/voice_clone_usingindic

## Overview

##### Project type & purpose (from README + tree)
**Project type:** Single-node, local-first **AI audio dubbing web app** (not a distributed backend/frontend deployment).
**Purpose:** Take an input audio clip → **transcribe** (Gemini 2.0 Flash) → **translate** to an Indian language (Gemini) → allow **manual edit** → **voice-clone TTS** (IndicF5 via PyTorch/Hugging Face) → output dubbed audio.
**UI:** **Gradio** served on `http://localhost:12000`.

##### What the file tree implies architecturally
Top-level files map cleanly to “one service” with external dependencies:

- `dubsync_complete.py` — **main application/orchestrator** (Gradio UI + pipeline)
- `app.py` — likely an alternate entrypoint/prototype (include as “optional/legacy runner” unless you confirm usage)
- `requirements.txt` — Python deps (Gradio, librosa, soundfile, google genai libs, etc.)
- `setup_environment.sh` — environment bootstrap (conda + torch cuda + IndicF5 install)
- `.env.example` — shows required **secrets**: `GEMINI_API_KEY`, `HUGGINGFACE_TOKEN`
- `.browser_screenshots/` — non-runtime artifacts (documentation/QA evidence)

##### User Browser
- uploads/records audio
- edits translation text
- downloads dubbed audio

##### DubSync Application (Python process)
- **Gradio Web Server/UI**
- **Pipeline Orchestrator** (the glue logic in `dubsync_complete.py`)
- **Audio Pre/Post-processing** (librosa/soundfile; resample to 24kHz mono)
- **ASR + Translation Client** (Gemini API calls)
- **Voice Cloning TTS Engine** (IndicF5 running locally via PyTorch)
- **Temp File Storage / Working Directory** (where uploaded audio, intermediate wav, final wav live)

##### External Services
- **Google Gemini API** (transcription + translation)
- **Hugging Face Hub** (model weights/token-auth download) *(even if the model is installed, weights are typically fetched/validated here)*

##### User → Gradio: Edit translated text (manual review)
8. **Gradio → IndicF5 Adapter**: Provide (a) normalized source audio (voice reference) + (b) edited translation text + (c) target language
##### Gradio → Browser: Return downloadable dubbed audio

- **App startup → Hugging Face Hub**: authenticate + download/load model artifacts (token)
- **App startup → Env Vars**: `GEMINI_API_KEY`, `HUGGINGFACE_TOKEN`
- **Error path**: IndicF5 load failure → “placeholder audio” → Browser

##### Important architectural patterns/design principles present
- **Orchestrator pipeline pattern**: one coordinator (in `dubsync_complete.py`) sequencing distinct AI steps.
- **Human-in-the-loop**: translation is not final until user edits; model inference depends on user-approved text.
- **Adapter pattern for external AI**: Gemini and IndicF5 are best drawn as adapters to keep the core pipeline clean.
- **Local inference + remote inference hybrid**: Gemini is remote API; IndicF5 is local GPU-heavy inference.

##### What to label explicitly (to make it “detailed” and accurate)
- **Technology**: “Gradio”, “PyTorch + IndicF5”, “Gemini 2.0 Flash API”, “librosa/soundfile”
- **Data contracts**:
- Input formats supported: WAV/MP3/M4A/FLAC
- Internal canonical format: **24kHz mono**
- Text: transcript + translated text + edited text
- **Constraints**:
- “<= 30s recommended”
- “GPU 4GB+ recommended”
- **Secrets**:
- `.env` provides `GEMINI_API_KEY`, `HUGGINGFACE_TOKEN`

##### Don’t overthink it: split into *more* components than fewer
Even if everything is implemented in one Python file, represent it as:
- UI layer
- preprocessing layer
- external API adapter
- human review step
- inference engine
- storage
- configuration
- setup-time provisioning

## Diagram (Mermaid)

```mermaid
flowchart TD

%% =======================
%% Diagram 1: System Context / Container (Runtime)
%% =======================
subgraph D1["Diagram 1 — System Context / Container (Runtime)"]
direction TB

subgraph LM["Local Machine"]
direction TB

U["User Browser\n- Upload/record audio\n- Edit translation text\n- Download dubbed audio\n\nUI: http://localhost:12000"]:::external

subgraph APP["DubSync Application (Single Python Process)"]
direction TB

G["Gradio Web Server / UI\n(handlers + web app)"]:::internal
P["Pipeline Orchestrator\n(transcribe→translate→edit→dub)"]:::internal
A["Audio Pre/Post-processing\nlibrosa + soundfile\n- decode WAV/MP3/M4A/FLAC\n- normalize to 24kHz mono"]:::internal
GA["Gemini Service Adapter\n(Gemini 2.0 Flash)\n- ASR\n- Translation (conversational)"]:::internal
E["Human-in-the-loop Editor\n(translation approval/edit)"]:::internal
TTS["IndicF5 Voice Cloning TTS\n(PyTorch local inference)"]:::internal
S[("Temp File Storage / Working Dir\n- uploads\n- intermediate wav\n- output wav")]:::storage
CFG["Secrets / Config (.env)\n- GEMINI_API_KEY\n- HUGGINGFACE_TOKEN"]:::config

end

GPU["Compute\nGPU (CUDA) recommended\nCPU fallback supported"]:::compute

end

subgraph NET["Internet Services"]
direction TB
GEM["Google Gemini API\n(remote ASR + translation)"]:::externalSvc
HF["Hugging Face Hub\n(model weights download/auth)"]:::externalSvc
end

%% Runtime connections
U -->|"upload/record_audio"| G
G -->|"invoke_pipeline"| P
P -->|"normalize_audio"| A
A -->|"canonical_wav(24kHz_mono)"| S
A -->|"audio_for_ASR"| GA
GA -->|"transcribe"| GEM
GA -->|"translate"| GEM
GA -->|"transcript+translated_text"| G
U -->|"edit_translation_text"| E
E -->|"final_translation_text"| G
G -->|"generate_dub(audio_ref+final_text+lang)"| TTS
TTS -->|"uses"| GPU
TTS -->|"write_output_audio"| S
G -->|"download_dubbed_audio"| U

%% Startup/init flows
CFG -.->|"read_env_vars"| P
TTS -.->|"auth+download_weights"| HF

%% Resilience
TTS -.->|"if_model_load_fails"| F["Placeholder Audio\n(resilience path)"]:::warn
F -.->|"return_placeholder"| G

end

%% =======================
%% Diagram 2: Component + Data Flow (Inside DubSync)
%% =======================
subgraph D2["Diagram 2 — Component + Data Flow (Inside DubSync)"]
direction TB

subgraph GH["Gradio Handlers"]
direction TB
H1["on_upload_or_record(audio)"]:::internal
H2["on_transcribe_translate(audio,target_lang)"]:::internal
H3["on_generate_dub(audio,edited_translation,target_lang)"]:::internal
end

subgraph CORE["Core Pipeline Modules"]
direction TB
N["Audio Normalizer\n- decode\n- resample 24kHz\n- mono"]:::internal
ASR["ASR Request Builder\n(audio→prompt/bytes)"]:::internal
TR["Translation Request Builder\n(transcript→target_lang)"]:::internal
AD["Gemini Adapter\n(handle responses/errors)"]:::internal
HITL["Manual Text Editor\n(user-approved translation)"]:::internal
IF5["IndicF5 Adapter\n- load checkpoints\n- voice clone synth"]:::internal
ERR["Fallback / Error Handling\n- placeholder audio on failure"]:::warn
FS[("Temp Storage\n(input/intermediate/output wav)")]:::storage
end

EXT1["Google Gemini API"]:::externalSvc
EXT2["Hugging Face Hub"]:::externalSvc
C["GPU/CPU\n(PyTorch inference target)"]:::compute

%% Flow (vertical, pipeline)
H1 -->|"save_raw_audio"| FS
H2 -->|"audio_in"| N
N -->|"write_canonical_wav"| FS
N -->|"normalized_audio"| ASR
ASR -->|"call_transcribe"| AD
AD -->|"transcribe"| EXT1
AD -->|"transcript"| TR
TR -->|"call_translate"| AD
AD -->|"translate"| EXT1
AD -->|"return_texts"| HITL
HITL -->|"edited_translation"| H3
H3 -->|"audio_ref+text+lang"| IF5
IF5 -.->|"startup_download_weights"| EXT2
IF5 -->|"infer"| C
IF5 -->|"write_dubbed_audio"| FS
IF5 -.->|"on_failure"| ERR
ERR -.->|"placeholder_audio"| FS

end

%% =======================
%% Diagram 3: Sequence (Transcribe/Translate then Dub) - Rendered as Flow
%% =======================
subgraph D3["Diagram 3 — Sequence (Transcribe/Translate then Dub)"]
direction TB

UB["User Browser"]:::external
GR["Gradio UI/Handlers"]:::internal
AN["Audio Normalizer"]:::internal
GM["Gemini Adapter"]:::internal
API["Google Gemini API"]:::externalSvc
ED["Text Editor"]:::internal
IF["IndicF5 Adapter"]:::internal
ST[("Temp Storage")]:::storage
HW["GPU/CPU"]:::compute

UB -->|"Upload/record audio"| GR
GR -->|"Normalize (24kHz mono)"| AN
AN -->|"Write canonical wav"| ST
GR -->|"Transcribe + Translate"| GM
GM -->|"Transcription request"| API
API -->|"Transcript"| GM
GM -->|"Translation request"| API
API -->|"Translated text"| GM
GM -->|"Transcript + translation"| GR
UB -->|"Edit translation text"| ED
ED -->|"Final translation"| GR
GR -->|"Generate dub (audio_ref + final_text + lang)"| IF
IF -->|"PyTorch inference"| HW
IF -->|"Write dubbed audio"| ST
GR -->|"Download dubbed audio"| UB

end

%% =======================
%% Repo / Setup Artifacts (Non-runtime)
%% =======================
subgraph R["Repo / Setup Artifacts (Non-runtime)"]
direction TB
Docs["Project Documentation\n(README)"]:::doc
Req["Python Dependencies\n(requirements)"]:::doc
Setup["Environment Provisioning\n(setup script)"]:::setup
Alt["Optional / Alternate Entrypoint\n(app.py)"]:::doc
Shots["Non-runtime Artifacts\n(.browser_screenshots/)"]:::doc
end

%% =======================
%% Click Events (from component_mapping)
%% =======================
click P "https://github.com/aravindan20/voice_clone_usingindic/blob/main/dubsync_complete.py"
click G "https://github.com/aravindan20/voice_clone_usingindic/blob/main/dubsync_complete.py"
click A "https://github.com/aravindan20/voice_clone_usingindic/blob/main/dubsync_complete.py"
click GA "https://github.com/aravindan20/voice_clone_usingindic/blob/main/dubsync_complete.py"
click TTS "https://github.com/aravindan20/voice_clone_usingindic/blob/main/dubsync_complete.py"
click S "https://github.com/aravindan20/voice_clone_usingindic/blob/main/dubsync_complete.py"
click H1 "https://github.com/aravindan20/voice_clone_usingindic/blob/main/dubsync_complete.py"
click H2 "https://github.com/aravindan20/voice_clone_usingindic/blob/main/dubsync_complete.py"
click H3 "https://github.com/aravindan20/voice_clone_usingindic/blob/main/dubsync_complete.py"
click N "https://github.com/aravindan20/voice_clone_usingindic/blob/main/dubsync_complete.py"
click ASR "https://github.com/aravindan20/voice_clone_usingindic/blob/main/dubsync_complete.py"
click TR "https://github.com/aravindan20/voice_clone_usingindic/blob/main/dubsync_complete.py"
click AD "https://github.com/aravindan20/voice_clone_usingindic/blob/main/dubsync_complete.py"
click HITL "https://github.com/aravindan20/voice_clone_usingindic/blob/main/dubsync_complete.py"
click IF5 "https://github.com/aravindan20/voice_clone_usingindic/blob/main/dubsync_complete.py"
click FS "https://github.com/aravindan20/voice_clone_usingindic/blob/main/dubsync_complete.py"
click CFG "https://github.com/aravindan20/voice_clone_usingindic/blob/main/.env.example"
click F "https://github.com/aravindan20/voice_clone_usingindic/blob/main/dubsync_complete.py"
click ERR "https://github.com/aravindan20/voice_clone_usingindic/blob/main/dubsync_complete.py"
click U "https://github.com/aravindan20/voice_clone_usingindic/blob/main/README.md"
click EXT1 "https://github.com/aravindan20/voice_clone_usingindic/blob/main/README.md"
click EXT2 "https://github.com/aravindan20/voice_clone_usingindic/blob/main/README.md"
click GPU "https://github.com/aravindan20/voice_clone_usingindic/blob/main/README.md"
click C "https://github.com/aravindan20/voice_clone_usingindic/blob/main/README.md"
click HF "https://github.com/aravindan20/voice_clone_usingindic/blob/main/README.md"
click GEM "https://github.com/aravindan20/voice_clone_usingindic/blob/main/README.md"
click ST "https://github.com/aravindan20/voice_clone_usingindic/blob/main/dubsync_complete.py"
click GR "https://github.com/aravindan20/voice_clone_usingindic/blob/main/dubsync_complete.py"
click AN "https://github.com/aravindan20/voice_clone_usingindic/blob/main/dubsync_complete.py"
click GM "https://github.com/aravindan20/voice_clone_usingindic/blob/main/dubsync_complete.py"
click IF "https://github.com/aravindan20/voice_clone_usingindic/blob/main/dubsync_complete.py"
click ED "https://github.com/aravindan20/voice_clone_usingindic/blob/main/dubsync_complete.py"
click UB "https://github.com/aravindan20/voice_clone_usingindic/blob/main/README.md"
click API "https://github.com/aravindan20/voice_clone_usingindic/blob/main/README.md"

click Req "https://github.com/aravindan20/voice_clone_usingindic/blob/main/requirements.txt"
click Setup "https://github.com/aravindan20/voice_clone_usingindic/blob/main/setup_environment.sh"
click Alt "https://github.com/aravindan20/voice_clone_usingindic/blob/main/app.py"
click Docs "https://github.com/aravindan20/voice_clone_usingindic/blob/main/README.md"
click Shots "https://github.com/aravindan20/voice_clone_usingindic/tree/main/.browser_screenshots/"

%% Styles (color is critical)
classDef internal fill:#d7ebff,stroke:#2b6cb0,color:#0b2545,stroke-width:1px
classDef external fill:#ffffff,stroke:#374151,color:#111827,stroke-width:1px
classDef externalSvc fill:#ffe6cc,stroke:#c2410c,color:#7c2d12,stroke-width:1px
classDef storage fill:#e5e7eb,stroke:#4b5563,color:#111827,stroke-width:1px
classDef compute fill:#efe1ff,stroke:#6d28d9,color:#2e1065,stroke-width:1px
classDef config fill:#fff7cc,stroke:#b45309,color:#7c2d12,stroke-width:1px
classDef warn fill:#ffe0e0,stroke:#b91c1c,color:#7f1d1d,stroke-width:1px
classDef doc fill:#ecfeff,stroke:#0891b2,color:#083344,stroke-width:1px
classDef setup fill:#e9ffe9,stroke:#15803d,color:#052e16,stroke-width:1px
```
