How openai/whisper works
Whisper is a speech-recognition and speech-translation package. Users provide audio through the Python API or CLI; the transcription workflow prepares audio, runs the model and decoding, and returns text segments, optionally with word timings and subtitle files. The README also documents language identification and multilingual translation. Edges reflect README statements and the supplied source excerpts or SOURCE INDEX; normalizer and some optional implementation details are shown without inferred workflow links.
Main entry points: transcribe.py API and CLI · timing.py Word timing · audio.py Audio decoding · decoding.py Task decoding
Components 16
User entry
- API and CLI
whisper/transcribe.py - Model loader
whisper/__init__.py
Audio preparation
- Audio decoding
whisper/audio.py - Log-Mel features
whisper/audio.py - Audio utilities
whisper/utils.py
Model inference
- Transformer model
whisper/model.py - Task decoding
whisper/decoding.py - Task tokenizer
whisper/tokenizer.py
Results and text
- Word timing
whisper/timing.py - Triton kernels
whisper/triton_ops.py - Basic normalization
whisper/normalizers/basic.py - English normalization
whisper/normalizers/english.py - Transcript outputs
whisper/utils.py
Other components
- User
- Model download host
- Audio file
Connections 17
- User → API and CLI: submits audio
README.md - API and CLI → Audio decoding: loads audio
whisper/transcribe.py - API and CLI → Log-Mel features: builds features
whisper/transcribe.py - Audio decoding → Audio file: reads
whisper/audio.py - Audio decoding → Audio utilities: uses
whisper/audio.py - API and CLI → Task decoding: decodes tasks
whisper/transcribe.py - Task decoding → Transformer model: runs inference
whisper/decoding.py - Task decoding → Task tokenizer: uses tokens
whisper/decoding.py - API and CLI → Task tokenizer: selects task tokens
whisper/transcribe.py - API and CLI → Word timing: aligns words
whisper/transcribe.py - Word timing → Transformer model: reads attention
whisper/timing.py - Word timing → Task tokenizer: splits word tokens
whisper/timing.py - Word timing → Triton kernels: uses kernels
whisper/timing.py - API and CLI → Transcript outputs: writes results
whisper/transcribe.py - Model loader → Transformer model: constructs model
whisper/__init__.py - Model loader → Model download host: downloads weights
whisper/__init__.py - English normalization → Basic normalization: extends
whisper/normalizers/english.py