How nyrahealth/crisperwhisper works
Project Type and Purpose
• CrisperWhisper is primarily a standalone Python-based speech‐recognition tool that offers:
• A CLI interface (transcribe.py) for batch/command-line transcription
• A web interface (app.py via Streamlit) for interactive transcription and visualization
• Core transcription logic (loading models, feature extraction, inference, post‐processing)
File Structure
• Top‐level scripts:
• app.py – Streamlit web app
• transcribe.py – CLI entry point
• utils.py – shared helper functions (e.g., timestamp adjustments)
• requirements.txt – dependencies (PyTorch, Transformers, Streamlit, ffmpeg, etc.)
• No separate “backend” or “frontend” folders – it’s a layered monolith
Main Components
1. User Interfaces
• CLI Client (console)
• Web Client (browser ⇄ Streamlit UI)
2. Application Layer
• Transcription Service
• orchestrates feature extraction, model inference, post‐processing
• Utility Module
• timestamp adjustment, filler detection logic in utils.py
3. Model & Inference Engine
• HuggingFace Transformers Pipeline
• Faster-Whisper / CTranslate2 alternative
• PyTorch runtime (GPU support via cuBLAS/cuDNN)
4. Data & Storage
• Local Model Cache (downloaded from Hugging Face Model Hub)
• Temporary Audio Buffer (in‐memory or temp files via ffmpeg)
5. External Services & Dependencies
• Hugging Face Model Hub (model artifact downloader)
• ffmpeg (audio decoding)
• Optional: Internet for initial model fetch
Relationships and Data Flow
• Both CLI and Web UI send an audio payload to the Transcription Service.
• The Service:
a. invokes ffmpeg (if needed) to decode audio → raw waveform
b. passes waveform to HuggingFace feature extractor → input tensors
c. calls model inference (PyTorch or CTranslate2) → logits / cross‐attention
d. applies DTW alignment loss logic → word‐level timestamps
e. runs post‐processing in utils → final transcript + metadata
• The Service returns a JSON‐like result: transcripts + timestamps → UI for display or CLI stdout
Architectural Patterns & Principles
• Layered decomposition:
• Presentation Layer (CLI & Streamlit UI)
• Application Layer (Transcription Service)
• Infrastructure Layer (Model runtime, external tools)
• Single Responsibility: separate scripts for different interfaces
• Modularity: shared logic in utils.py so both interfaces reuse core functions
Technologies & Frameworks
• Python 3.10
• PyTorch 2.0 (GPU support)
• HuggingFace Transformers (custom fork for crisp timestamps)
• Faster-Whisper / CTranslate2 (optional path)
• Streamlit (web UI)
• ffmpeg (audio pre-processing)
• rust (for whisper prerequisites)
[ ] Components: CLI, Streamlit UI, Transcription Service, Model Engine, Utils, Model Cache, ffmpeg, HF Hub
[ ] Labels: component names + data payload types
[ ] Notes: key technologies/framework versions where relevant