# alexgichamba/whisper architecture

> Architecture diagram of the GitHub repository alexgichamba/whisper, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/alexgichamba/whisper
- Repository: https://github.com/alexgichamba/whisper

## Overview

##### Project Type and Purpose
- Type: Open-source CLI tool & Python library
- Purpose:
- Fine-tune OpenAI’s Whisper speech model on low-resource language datasets (ASR & speech-to-text)
- Provide data-preparation utilities, training, inference (transcription/translation), and evaluation scripts

##### High-Level Components
2.1 CLI Interface
- Entry point: `whisper/__main__.py`
- Commands: `whisper.transcribe`, `whisper.trainer`, data prep (`iwslt/utils`), evaluation (`iwslt/evaluate/run_eval.py`)
2.2 Core Whisper Library (`whisper/`)
- Audio processing: `audio.py`
- Feature extraction & mel-spectrograms (via Triton ops)
- Model definition: `model.py`
- Tokenizer & normalizers: `tokenizer.py`, `normalizers/`
- Decoding & timing: `decoding.py`, `timing.py`
- Training harness: `trainer.py`
- Transcription API: `transcribe.py`
2.3 Task-Specific Modules
- Data-prep scripts: `iwslt/utils/dataprep_*.py`
- Training configs: `iwslt/train_conf/*.yaml`
- Evaluation runner: `iwslt/evaluate/run_eval.py`
2.4 Data Storage
- Raw datasets directory: `data/`, plus external corpora mounted via `--base_path`
- Checkpoints & results: local filesystem (`results/` in instructions)
2.5 Notebooks
- Interactive demos: `notebooks/LibriSpeech.ipynb`, `notebooks/Multilingual_ASR.ipynb`
2.6 Tests
- Unit & CLI tests: `tests/`

##### External Dependencies
- ffmpeg (audio decoding)
- PyTorch (deep learning)
- tiktoken (Rust-based tokenizer)
- YAML parser (training configs)
- Python CLI libs (argparse)
- Conda/Pip environment

##### Relationships & Data Flow
- CLI command → invokes Python entrypoint → routes to one of:
- Data-prep: load raw audio/text → preprocess → write processed files
- Training: load config YAML → initialize model → load dataset → forward/backward → save checkpoints
- Inference/Transcribe: load checkpoint → audio→normalize→spectrogram→model→decode→output text
- Evaluation: load model checkpoint + predictions → compute BLEU/chrF/WER → report
- Notebooks import the core library for demos
- Tests import and validate individual modules

##### Architectural Patterns & Principles
- Modular package: clear separation of core library vs task-specific code
- Configuration-driven training via YAML
- CLI wrappers over library functions for reproducibility
- Pluggable data-prep scripts per dataset
- Single source of truth for model code under `whisper/`

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% CLI Interface
    subgraph "CLI Interface"
        CLI["CLI Interface\n(whisper/__main__.py)"]:::pythonlib
    end

    %% Core Whisper Library
    subgraph "Core Whisper Library"
        A["Audio Processing\n(whisper/audio.py)"]:::pythonlib
        F["Feature Extraction & Triton Ops\n(whisper/triton_ops.py)"]:::pythonlib
        M["Model Definition\n(whisper/model.py)"]:::pythonlib
        TKN["Tokenizer\n(whisper/tokenizer.py)"]:::pythonlib
        N["Normalizers\n(whisper/normalizers/)"]:::pythonlib
        D["Decoding\n(whisper/decoding.py)"]:::pythonlib
        TM["Timing\n(whisper/timing.py)"]:::pythonlib
        U["Utility Functions\n(whisper/utils.py)"]:::pythonlib
        TRN["Training Harness\n(whisper/trainer.py)"]:::pythonlib
        TRS["Transcription API\n(whisper/transcribe.py)"]:::pythonlib
    end

    %% Data Preparation
    subgraph "Data Preparation"
        DP["Data-Preparation Utilities\n(iwslt/utils)"]:::scripts
    end

    %% Training Configurations
    CONFIG["Training Configurations\n(iwslt/train_conf/)"]:::scripts

    %% Evaluation
    EVAL["Evaluation Runner\n(iwslt/evaluate/run_eval.py)"]:::scripts

    %% Notebooks
    subgraph "Interactive Notebooks"
        NB1["LibriSpeech Demo\n(notebooks/LibriSpeech.ipynb)"]:::scripts
        NB2["Multilingual ASR Demo\n(notebooks/Multilingual_ASR.ipynb)"]:::scripts
    end

    %% Tests
    subgraph "Tests"
        T1["CLI Tests\n(tests/cli_tests.py)"]:::tests
        T2["Audio Tests\n(tests/test_audio.py)"]:::tests
        T3["Normalizer Tests\n(tests/test_normalizer.py)"]:::tests
        T4["Timing Tests\n(tests/test_timing.py)"]:::tests
        T5["Tokenizer Tests\n(tests/test_tokenizer.py)"]:::tests
        T6["Transcribe Tests\n(tests/test_transcribe.py)"]:::tests
        T7["Test Fixtures\n(tests/conftest.py)"]:::tests
        T8["Sample Audio File\n(tests/jfk.flac)"]:::tests
    end

    %% Data Storage
    DS_RAW["Raw Datasets\n(data/)"]:::data
    DS_CHECK["Checkpoints & Results"]:::data

    %% External Dependencies
    subgraph "External Dependencies"
        E1["ffmpeg"]:::external
        E2["PyTorch"]:::external
        E3["tiktoken (Rust)"]:::external
        E4["YAML parser"]:::external
        E5["argparse"]:::external
    end

    %% Packaging & CI
    PC1["Project Config\n(pyproject.toml)"]:::external
    PC2["GitHub Actions\n(.github/workflows/)"]:::external

    %% Relationships
    CLI --> DP
    CLI --> CONFIG
    CLI --> TRN
    CLI --> TRS
    CLI --> EVAL

    DP --> DS_RAW
    DS_RAW --> DP

    TRN --> M
    TRN --> A
    TRN --> F
    TRN --> TKN
    TRN --> N
    TRN --> U
    TRN --> DS_CHECK
    DS_RAW --> TRN
    CONFIG --> TRN

    TRS --> A
    TRS --> F
    TRS --> M
    TRS --> D
    TRS --> TM
    TRS --> DS_CHECK

    EVAL --> DS_CHECK
    EVAL --> TRS
    EVAL --> TRN

    NB1 --> A
    NB1 --> M
    NB1 --> TRS
    NB2 --> A
    NB2 --> M
    NB2 --> TRS

    T1 --> CLI
    T2 --> A
    T3 --> N
    T4 --> TM
    T5 --> TKN
    T6 --> TRS
    T7 --> T1
    T8 --> T2

    E1 --> A
    E2 --> M
    E3 --> TKN
    E4 --> CONFIG
    E5 --> CLI

    PC1 --> CLI
    PC2 --> T1
    PC2 --> PC1

    %% Click Events
    click CLI "https://github.com/alexgichamba/whisper/blob/main/whisper/__main__.py"
    click TRS "https://github.com/alexgichamba/whisper/blob/main/whisper/transcribe.py"
    click TRN "https://github.com/alexgichamba/whisper/blob/main/whisper/trainer.py"
    click DP "https://github.com/alexgichamba/whisper/tree/main/iwslt/utils"
    click EVAL "https://github.com/alexgichamba/whisper/blob/main/iwslt/evaluate/run_eval.py"
    click A "https://github.com/alexgichamba/whisper/blob/main/whisper/audio.py"
    click F "https://github.com/alexgichamba/whisper/blob/main/whisper/triton_ops.py"
    click M "https://github.com/alexgichamba/whisper/blob/main/whisper/model.py"
    click TKN "https://github.com/alexgichamba/whisper/blob/main/whisper/tokenizer.py"
    click N "https://github.com/alexgichamba/whisper/tree/main/whisper/normalizers/"
    click D "https://github.com/alexgichamba/whisper/blob/main/whisper/decoding.py"
    click TM "https://github.com/alexgichamba/whisper/blob/main/whisper/timing.py"
    click U "https://github.com/alexgichamba/whisper/blob/main/whisper/utils.py"
    click CONFIG "https://github.com/alexgichamba/whisper/tree/main/iwslt/train_conf/"
    click DS_RAW "https://github.com/alexgichamba/whisper/tree/main/data/"
    click NB1 "https://github.com/alexgichamba/whisper/blob/main/notebooks/LibriSpeech.ipynb"
    click NB2 "https://github.com/alexgichamba/whisper/blob/main/notebooks/Multilingual_ASR.ipynb"
    click T1 "https://github.com/alexgichamba/whisper/blob/main/tests/cli_tests.py"
    click T2 "https://github.com/alexgichamba/whisper/blob/main/tests/test_audio.py"
    click T3 "https://github.com/alexgichamba/whisper/blob/main/tests/test_normalizer.py"
    click T4 "https://github.com/alexgichamba/whisper/blob/main/tests/test_timing.py"
    click T5 "https://github.com/alexgichamba/whisper/blob/main/tests/test_tokenizer.py"
    click T6 "https://github.com/alexgichamba/whisper/blob/main/tests/test_transcribe.py"
    click T7 "https://github.com/alexgichamba/whisper/blob/main/tests/conftest.py"
    click T8 "https://github.com/alexgichamba/whisper/blob/main/tests/jfk.flac"
    click PC1 "https://github.com/alexgichamba/whisper/blob/main/pyproject.toml"
    click PC2 "https://github.com/alexgichamba/whisper/tree/main/.github/workflows/"

    %% Styles
    classDef pythonlib fill:#d0e1f9,stroke:#2b3a67,color:#2b3a67;
    classDef scripts fill:#d4f1cd,stroke:#2a5d34,color:#2a5d34;
    classDef external fill:#ffe4b5,stroke:#b36b00,color:#b36b00;
    classDef data fill:#f0f0f0,stroke:#7a7a7a,color:#7a7a7a;
    classDef tests fill:#f4d4e3,stroke:#8e2a48,color:#8e2a48;
```
