# nyrahealth/crisperwhisper architecture

> Architecture diagram of the GitHub repository nyrahealth/crisperwhisper, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nyrahealth/crisperwhisper
- Repository: https://github.com/nyrahealth/crisperwhisper

## Overview

##### Project Type and Purpose
- CrisperWhisper is primarily a standalone Python-based speech‐recognition tool that offers:
- A CLI interface (`transcribe.py`) for batch/command-line transcription
- A web interface (`app.py` via Streamlit) for interactive transcription and visualization
- Core transcription logic (loading models, feature extraction, inference, post‐processing)

##### File Structure
- Top‐level scripts:
- app.py – Streamlit web app
- transcribe.py – CLI entry point
- utils.py – shared helper functions (e.g., timestamp adjustments)
- requirements.txt – dependencies (PyTorch, Transformers, Streamlit, ffmpeg, etc.)
- No separate “backend” or “frontend” folders – it’s a layered monolith

##### Main Components
1. **User Interfaces**
- CLI Client (console)
- Web Client (browser ⇄ Streamlit UI)
2. **Application Layer**
- Transcription Service
- orchestrates feature extraction, model inference, post‐processing
- Utility Module
- timestamp adjustment, filler detection logic in utils.py
3. **Model & Inference Engine**
- HuggingFace Transformers Pipeline
- Faster-Whisper / CTranslate2 alternative
- PyTorch runtime (GPU support via cuBLAS/cuDNN)
4. **Data & Storage**
- Local Model Cache (downloaded from Hugging Face Model Hub)
- Temporary Audio Buffer (in‐memory or temp files via ffmpeg)
5. **External Services & Dependencies**
- Hugging Face Model Hub (model artifact downloader)
- ffmpeg (audio decoding)
- Optional: Internet for initial model fetch

##### Relationships and Data Flow
- Both CLI and Web UI send an audio payload to the Transcription Service.
- The Service:
a. invokes ffmpeg (if needed) to decode audio → raw waveform
b. passes waveform to HuggingFace feature extractor → input tensors
c. calls model inference (PyTorch or CTranslate2) → logits / cross‐attention
d. applies DTW alignment loss logic → word‐level timestamps
e. runs post‐processing in utils → final transcript + metadata
- The Service returns a JSON‐like result: transcripts + timestamps → UI for display or CLI stdout

##### Architectural Patterns & Principles
- Layered decomposition:
- Presentation Layer (CLI & Streamlit UI)
- Application Layer (Transcription Service)
- Infrastructure Layer (Model runtime, external tools)
- Single Responsibility: separate scripts for different interfaces
- Modularity: shared logic in utils.py so both interfaces reuse core functions

##### Technologies & Frameworks
- Python 3.10
- PyTorch 2.0 (GPU support)
- HuggingFace Transformers (custom fork for crisp timestamps)
- Faster-Whisper / CTranslate2 (optional path)
- Streamlit (web UI)
- ffmpeg (audio pre-processing)
- rust (for whisper prerequisites)

[ ] Components: CLI, Streamlit UI, Transcription Service, Model Engine, Utils, Model Cache, ffmpeg, HF Hub
[ ] Labels: component names + data payload types
[ ] Notes: key technologies/framework versions where relevant

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Presentation Layer
    subgraph "Presentation Layer"
        CLI["CLI Client"]:::ui
        WEB["Web Client (Streamlit UI)"]:::ui
    end

    %% Application Layer
    subgraph "Application Layer"
        TS["Transcription Service"]:::service
        UT["Utils Module"]:::service
    end

    %% Inference & Modeling Engine
    subgraph "Inference & Modeling Engine"
        FE["Feature Extractor (HuggingFace Transformers)"]:::infra
        MI["Model Inference Engine\n(PyTorch / CTranslate2)"]:::infra
    end

    %% Data & Storage
    subgraph "Data & Storage"
        MC["Local Model Cache"]:::infra
        AB["Temporary Audio Buffer"]:::infra
    end

    %% External Services & Dependencies
    subgraph "External Services"
        HF["Hugging Face Model Hub"]:::external
        FFMPEG["ffmpeg"]:::external
        INTERNET["Internet (initial fetch)"]:::external
    end

    %% Project Files (clickable)
    subgraph "Project Files"
        REQ["Dependencies (requirements.txt)"]:::external
        DOC["Project Documentation (README.md)"]:::external
    end

    %% Data Flows
    CLI -->|"audio payload"| TS
    WEB -->|"audio payload"| TS

    TS -->|"invoke ffmpeg"| FFMPEG
    FFMPEG -->|"raw waveform"| AB
    AB -->|"raw waveform"| FE
    FE -->|"input tensors"| MI
    MI -->|"logits / attention"| TS

    TS -->|"post-processing input"| UT
    UT -->|"final transcript + metadata"| TS

    TS -->|"JSON result"| CLI
    TS -->|"JSON result"| WEB

    TS -->|"load model"| MC
    HF -->|"download models"| MC

    %% Click Events
    click CLI "https://github.com/nyrahealth/crisperwhisper/blob/main/transcribe.py"
    click WEB "https://github.com/nyrahealth/crisperwhisper/blob/main/app.py"
    click UT "https://github.com/nyrahealth/crisperwhisper/blob/main/utils.py"
    click REQ "https://github.com/nyrahealth/crisperwhisper/blob/main/requirements.txt"
    click DOC "https://github.com/nyrahealth/crisperwhisper/blob/main/README.md"

    %% Styling
    classDef ui fill:#D0E8FF,stroke:#0077CC,stroke-width:2px;
    classDef service fill:#D0FFD6,stroke:#2E8B57,stroke-width:2px;
    classDef infra fill:#FFE4C4,stroke:#D2691E,stroke-width:2px;
    classDef external fill:#F2F2F2,stroke:#666666,stroke-width:2px;
```
