How alexgichamba/whisper works
Project Type and Purpose
• Type: Open-source CLI tool & Python library
• Purpose:
• Fine-tune OpenAI’s Whisper speech model on low-resource language datasets (ASR & speech-to-text)
• Provide data-preparation utilities, training, inference (transcription/translation), and evaluation scripts
High-Level Components
2.1 CLI Interface
• Entry point: whisper/__main__.py
• Commands: whisper.transcribe, whisper.trainer, data prep (iwslt/utils), evaluation (iwslt/evaluate/run_eval.py)
2.2 Core Whisper Library (whisper/)
• Audio processing: audio.py
• Feature extraction & mel-spectrograms (via Triton ops)
• Model definition: model.py
• Tokenizer & normalizers: tokenizer.py, normalizers/
• Decoding & timing: decoding.py, timing.py
• Training harness: trainer.py
• Transcription API: transcribe.py
2.3 Task-Specific Modules
• Data-prep scripts: iwslt/utils/dataprep_*.py
• Training configs: iwslt/train_conf/*.yaml
• Evaluation runner: iwslt/evaluate/run_eval.py
2.4 Data Storage
• Raw datasets directory: data/, plus external corpora mounted via --base_path
• Checkpoints & results: local filesystem (results/ in instructions)
2.5 Notebooks
• Interactive demos: notebooks/LibriSpeech.ipynb, notebooks/Multilingual_ASR.ipynb
2.6 Tests
• Unit & CLI tests: tests/
External Dependencies
• ffmpeg (audio decoding)
• PyTorch (deep learning)
• tiktoken (Rust-based tokenizer)
• YAML parser (training configs)
• Python CLI libs (argparse)
• Conda/Pip environment
Relationships & Data Flow
• CLI command → invokes Python entrypoint → routes to one of:
• Data-prep: load raw audio/text → preprocess → write processed files
• Training: load config YAML → initialize model → load dataset → forward/backward → save checkpoints
• Inference/Transcribe: load checkpoint → audio→normalize→spectrogram→model→decode→output text
• Evaluation: load model checkpoint + predictions → compute BLEU/chrF/WER → report
• Notebooks import the core library for demos
• Tests import and validate individual modules
Architectural Patterns & Principles
• Modular package: clear separation of core library vs task-specific code
• Configuration-driven training via YAML
• CLI wrappers over library functions for reproducibility
• Pluggable data-prep scripts per dataset
• Single source of truth for model code under whisper/