# yeyupiaoling/whisper-finetune architecture

> Architecture diagram of the GitHub repository yeyupiaoling/whisper-finetune, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/yeyupiaoling/whisper-finetune
- Repository: https://github.com/yeyupiaoling/whisper-finetune

## Overview

##### Project type and purpose
- This is a combined AI/ML toolkit and full-stack/demo system for fine-tuning OpenAI’s Whisper ASR model, running inference (CLI, server, GUI), and deploying to Android and Windows desktop. It includes:
- Data preparation & augmentation scripts (aishell.py, tools/*)
- Training (finetune.py), merging and evaluation (merge_lora.py, evaluation.py)
- Inference CLI and accelerated variants (infer.py, infer_ct2.py, infer_server.py, infer_gui.py)
- Web frontend (templates/, static/) served by Flask (infer_server.py)
- Mobile app (AndroidDemo/) and desktop app (WhisperDesktop/)

##### File structure
- Top-level ML scripts: `aishell.py`, `finetune.py`, `merge_lora.py`, `evaluation.py`, `infer*.py`
- Configuration: `configs/`, `requirements.txt`, `run.sh`
- Data: `dataset/`, `tools/` (data conversion, speed tests)
- Web UI: `templates/` (HTML), `static/` (CSS, JS)
- Android client: `AndroidDemo/` (Gradle project, JNI wrapper for whisper.cpp)
- Desktop client: `WhisperDesktop/` (Windows .exe)
- Helpers: `utils/`, `metrics/`

##### README for additional insights
- It describes training, conversion to CTranslate2 and GGML, evaluation, inference options, GUI and server deployment, Android and Windows demos.
- It lists dependency stack (Python, PyTorch, CTranslate2, faster-whisper, whisper.cpp).

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Data Layer
    subgraph "Data Layer"
        direction TB
        RawData[(“Raw Audio Datasets”)]:::storage
        DataPrep["Data Prep & Augmentation"]:::ml
    end

    %% ML Pipeline
    subgraph "ML Pipeline"
        direction TB
        Finetune["Fine-tuning (finetune.py)"]:::ml
        Merge["Merge LoRA (merge_lora.py)"]:::ml
        Eval["Evaluation (evaluation.py)"]:::ml
        Convert["Model Conversion (convert-ggml.py)"]:::ml
        ModelStore[(“Model Repository”)]:::storage
    end

    %% Inference Services
    subgraph "Inference Services"
        direction TB
        CLI["CLI Inference"]:::service
        Server["Flask API Server"]:::service
        GUI["Desktop GUI Service"]:::service
    end

    %% Frontends / Clients
    subgraph "Frontends"
        direction TB
        WebUI["Web UI (templates/static)"]:::frontend
        AndroidApp["Android App"]:::native
        DesktopApp["Windows Desktop App"]:::native
    end

    %% External Dependencies
    subgraph "External"
        HF["HuggingFace Hub"]:::external
        GPU["GPU / CPU"]:::external
    end

    %% Utilities & Metrics
    subgraph "Helpers"
        direction TB
        Utils["Utilities (utils/)"]:::helper
        Metrics["Metrics (metrics/)"]:::helper
    end

    %% Dataset Examples
    subgraph "Dataset Examples"
        direction TB
        Examples["dataset/"]:::storage
    end

    %% Flow Connections
    RawData -->|“ingested by”| DataPrep
    DataPrep -->|“trains on”| Finetune
    Finetune -->|“outputs”| Merge
    Merge -->|“yields”| ModelStore
    ModelStore -->|“evaluated by”| Eval
    ModelStore -->|“converted by”| Convert
    Convert -->|“accelerated models”| ModelStore
    ModelStore -->|“loaded by”| CLI
    ModelStore -->|“loaded by”| Server
    ModelStore -->|“loaded by”| GUI

    CLI -->|“CLI outputs”| GPU
    Server -->|“serves REST API”| WebUI
    Server -->|“runs inference on”| GPU
    GUI -->|“local inference on”| GPU

    WebUI -->|“HTTP REST”| Server
    AndroidApp -->|“JNI calls”| Convert
    AndroidApp -->|“uses”| GPU
    DesktopApp -->|“uses”| GPU

    Finetune -->|“imports”| Utils
    Merge -->|“imports”| Utils
    Eval -->|“imports”| Metrics
    DataPrep -->|“imports”| Utils
    CLI -->|“imports”| Utils

    HF -->|“download base model”| Finetune
    HF -->|“upload final model”| ModelStore

    Examples -->|“sample audio”| DataPrep

    %% Click Events
    click DataPrep "https://github.com/yeyupiaoling/whisper-finetune/blob/master/aishell.py"
    click DataPrep "https://github.com/yeyupiaoling/whisper-finetune/tree/master/tools/"
    click Finetune "https://github.com/yeyupiaoling/whisper-finetune/blob/master/finetune.py"
    click Merge "https://github.com/yeyupiaoling/whisper-finetune/blob/master/merge_lora.py"
    click Eval "https://github.com/yeyupiaoling/whisper-finetune/blob/master/evaluation.py"
    click Convert "https://github.com/yeyupiaoling/whisper-finetune/blob/master/convert-ggml.py"
    click CLI "https://github.com/yeyupiaoling/whisper-finetune/blob/master/infer.py"
    click CLI "https://github.com/yeyupiaoling/whisper-finetune/blob/master/infer_ct2.py"
    click CLI "https://github.com/yeyupiaoling/whisper-finetune/blob/master/infer_thirteen.py"
    click Server "https://github.com/yeyupiaoling/whisper-finetune/blob/master/infer_server.py"
    click Server "https://github.com/yeyupiaoling/whisper-finetune/tree/master/templates/"
    click Server "https://github.com/yeyupiaoling/whisper-finetune/tree/master/static/"
    click GUI "https://github.com/yeyupiaoling/whisper-finetune/blob/master/infer_gui.py"
    click configs "https://github.com/yeyupiaoling/whisper-finetune/blob/master/configs/augmentation.json"
    click configs "https://github.com/yeyupiaoling/whisper-finetune/blob/master/requirements.txt"
    click configs "https://github.com/yeyupiaoling/whisper-finetune/blob/master/run.sh"
    click Examples "https://github.com/yeyupiaoling/whisper-finetune/tree/master/dataset/"
    click Metrics "https://github.com/yeyupiaoling/whisper-finetune/blob/master/metrics/cer.py"
    click Metrics "https://github.com/yeyupiaoling/whisper-finetune/blob/master/metrics/wer.py"
    click Utils "https://github.com/yeyupiaoling/whisper-finetune/tree/master/utils/"
    click AndroidApp "https://github.com/yeyupiaoling/whisper-finetune/tree/master/AndroidDemo/"
    click DesktopApp "https://github.com/yeyupiaoling/whisper-finetune/blob/master/WhisperDesktop/WhisperDesktop.exe"

    %% Styles
    classDef ml fill:#e0f7e9,stroke:#2e7d32,stroke-width:2px
    classDef service fill:#e3f2fd,stroke:#1565c0,stroke-width:2px
    classDef frontend fill:#fff3e0,stroke:#ef6c00,stroke-width:2px
    classDef native fill:#f3e5f5,stroke:#6a1b9a,stroke-width:2px
    classDef storage fill:#eceff1,stroke:#607d8b,stroke-width:2px
    classDef external fill:#f5f5f5,stroke:#9e9e9e,stroke-width:2px
    classDef helper fill:#f1f8e9,stroke:#7cb342,stroke-width:2px
```
