# pr0mila/medibeng-whisper-tiny architecture

> Architecture diagram of the GitHub repository pr0mila/medibeng-whisper-tiny, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pr0mila/medibeng-whisper-tiny
- Repository: https://github.com/pr0mila/medibeng-whisper-tiny

## Overview

##### Project type and purpose
- This is an ML-centric application combining:
- A training/fine-tuning pipeline for a speech translation model (Whisper-Tiny → MediBeng-Whisper-Tiny).
- An inference service exposing the fine-tuned model via both a Gradio UI and a FastAPI REST endpoint.
- Purpose: ingest code-switched Bengali-English audio, translate to English, and return text (for clinical settings).

##### File structure
- Top-level scripts:
- main.py – orchestrates fine-tuning
- data_loader.py – loads and preprocesses dataset
- upload_model/upload_model.py – pushes model to Hugging Face Hub
- Config:
- config/config.py – hyperparameters, model names, output paths
- Inference app:
- app/api.py – FastAPI server
- app/gradio_interface.py – Gradio web UI
- app/audio_processing.py – audio feature extraction and post-processing
- Assets, tests, requirements define static resources, test cases, and dependencies.

##### Key README insights
- Two interaction modes: Gradio UI and FastAPI endpoint (`/transcribe`).
- Training uses Hugging Face Transformers, Accelerate, datasets → outputs WER metrics.
- Model and dataset live on Hugging Face Hub; upload script uses HF token.

##### Main components
a. Training Pipeline
1. Data Source:
- Local synthetic MediBeng dataset (data_loader.py)
- Hugging Face dataset (pr0mila-gh0sh/MediBeng)
2. Preprocessing:
- librosa, audio_feature_extraction (audio_processing.py or inside data_loader)
3. Model Fine-tuning:
- openai/whisper-tiny base model → huggingface/transformers Trainer via main.py
- Config parameters from config/config.py
4. Output:
- Saved model checkpoints in OUTPUT_DIR
5. Upload:
- Hugging Face Hub via upload_model/upload_model.py

b. Inference Stack
1. Client Layer:
- Gradio Web UI (app/gradio_interface.py)
- External REST client (cURL or browser hitting FastAPI endpoint)
2. API Layer:
- FastAPI server (app/api.py) exposes `/transcribe`
- Both UI and API import audio_processing.py
3. Preprocessing & Postprocessing:
- Audio file ingestion → waveform extraction → feature normalization
- Whisper model inference → raw logits → decode to English text
4. Model Serving:
- Load MediBeng-Whisper-Tiny weights from local or HF Hub
- Hugging Face Transformers/Accelerate runtime
5. Response:
- Return translated text payload in JSON (API) or render in the web UI (Gradio)

c. External Services & Storage
- Hugging Face Hub (model & dataset)
- CTranslate2 conversion (optional, per README)
- Local filesystem (assets/, tests/data/, OUTPUT_DIR)

##### Relationships and interactions
- Training → pushes model to HF Hub → inference layers pull model
- Clients → UI/API → Preprocessing → Model → Postprocessing → Client
- Config drives both training and inference parameters

##### Architectural patterns & principles
- Clear separation of concerns:
- Data ingestion (data_loader.py)
- Model logic (main.py, upload_model.py)
- Inference API (app/api.py)
- Presentation/UI (app/gradio_interface.py)
- Modular design to swap out base models or interfaces
- Use of external managed services (Hugging Face Hub) for model storage and retrieval

##### Elements
- A title indicating “MediBeng-Whisper-Tiny System Architecture”

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Training Pipeline
    subgraph "Training Pipeline"
        direction TB
        DL["Data Loader"]:::data
        Config["Config"]:::training
        Orch["Training Orchestrator"]:::training
        FineTuner["Whisper-Tiny Fine-Tuner"]:::training
        Storage[(Local OUTPUT_DIR)]:::external
        Upload["Model Upload"]:::training

        DL -->|"loads & preprocesses"| Orch
        Config -->|"provides params"| Orch
        Orch -->|"fine-tunes model"| FineTuner
        FineTuner -->|"saves checkpoints"| Storage
        FineTuner -->|"model files"| Upload
    end

    %% Inference Service
    subgraph "Inference Service"
        direction TB
        UI["Gradio UI"]:::inference
        API["FastAPI /transcribe"]:::inference
        AudioProc["Audio Processing"]:::inference
        ModelServe["Model Serving"]:::inference

        UI -->|"invoke transcribe"| API
        UI -->|"audio file"| AudioProc
        API -->|"audio file"| AudioProc
        AudioProc -->|"features"| ModelServe
        ModelServe -->|"text"| API
        ModelServe -->|"text"| UI
    end

    %% External Services & Storage
    subgraph "External Services"
        direction TB
        HFData[(Hugging Face Dataset)]:::external
        HFModel[(Hugging Face Model Repo)]:::external
        Assets["Static Assets"]:::external
        TestsData["Test Audio Files"]:::external

        HFData -->|"dataset pull"| DL
        HFModel -->|"model pull"| ModelServe
        Upload -->|"push model"| HFModel
    end

    %% Tests & Documentation
    subgraph "Tests & Docs"
        direction TB
        UT1["Unit Test: test_model_audio_list"]:::external
        UT2["Unit Test: test_model_single_audio"]:::external
        Req["requirements.txt"]:::external
        Readme["README.md"]:::external
        License["LICENSE"]:::external
    end

    %% Click Events
    click DL "https://github.com/pr0mila/medibeng-whisper-tiny/blob/main/data_loader.py"
    click Orch "https://github.com/pr0mila/medibeng-whisper-tiny/blob/main/main.py"
    click Config "https://github.com/pr0mila/medibeng-whisper-tiny/blob/main/config/config.py"
    click Upload "https://github.com/pr0mila/medibeng-whisper-tiny/blob/main/upload_model/upload_model.py"
    click API "https://github.com/pr0mila/medibeng-whisper-tiny/blob/main/app/api.py"
    click UI "https://github.com/pr0mila/medibeng-whisper-tiny/blob/main/app/gradio_interface.py"
    click AudioProc "https://github.com/pr0mila/medibeng-whisper-tiny/blob/main/app/audio_processing.py"
    click Assets "https://github.com/pr0mila/medibeng-whisper-tiny/tree/main/assets/"
    click TestsData "https://github.com/pr0mila/medibeng-whisper-tiny/tree/main/tests/data/"
    click UT1 "https://github.com/pr0mila/medibeng-whisper-tiny/blob/main/tests/test_model_audio_list.py"
    click UT2 "https://github.com/pr0mila/medibeng-whisper-tiny/blob/main/tests/test_model_single_audio.py"
    click Req "https://github.com/pr0mila/medibeng-whisper-tiny/blob/main/requirements.txt"
    click Readme "https://github.com/pr0mila/medibeng-whisper-tiny/blob/main/README.md"
    click License "https://github.com/pr0mila/medibeng-whisper-tiny/tree/main/LICENSE"

    %% Styles
    classDef data fill:#ADD8E6,stroke:#0000FF,color:#000
    classDef training fill:#98FB98,stroke:#008000,color:#000
    classDef inference fill:#FFD580,stroke:#FFA500,color:#000
    classDef external fill:#D3D3D3,stroke:#696969,color:#000
```
