# siddarthaa/whisperx architecture

> Architecture diagram of the GitHub repository siddarthaa/whisperx, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/siddarthaa/whisperx
- Repository: https://github.com/siddarthaa/whisperx

## Overview

##### Project Type and Purpose
- WhisperX is a full-stack, single-page Python application (using Streamlit via App.py) for media processing. It ingests audio/video, transcribes to text, creates embeddings, indexes them, and supports three key features:
- Summarization
- RAG-based Question Answering
- Quiz Generation

##### High-Level File Structure Analysis
- Top‐level
- App.py – the Streamlit frontend and main orchestrator
- QuizGen.py – standalone quiz generation module
- Setup.py / requirements.txt – environment and dependency setup
- Prompts/ – contains prompt templates (instructions, question types, summarization, quizzes)
- No separately deployed microservices; the design is a modular, pipeline-style monolith.

##### Main Architectural Components
a. User Interface (Streamlit)
- Upload widget, parameter settings, result display
b. Transcription Service
- OpenAI Whisper or Tiny.en invoked via Python bindings
c. Embedding Service
- All-MPNet-Base-V2 model generates vector embeddings
d. Vector Store / Retrieval (FAISS)
- In‐memory or on‐disk index for semantic search
e. Summarization Module
- Llama3 / Gemini via Ollama for condensing transcripts
f. RAG Question Answering Module
- Query FAISS, retrieve relevant chunks, feed to LLM for answer generation
g. Quiz Generation Module
- QuizGen.py + Prompts/Quiz templates to build MCQs, fill-in, true/false, short answer
h. Prompt Repository
- Prompts/ folder containing all prompt templates and instructions

##### Component Relationships & Data Flow
1. User uploads media in the UI
→ App.py sends file to Transcription Service
→ Transcript data flows to Embedding Service
→ Embeddings are stored/indexed in FAISS
2. Summarization Request
→ Transcript → LLM Summarization (Llama3/Gemini) → UI
3. Q&A Request
→ User question in UI → FAISS retrieval of top-k chunks → LLM RAG pipeline → UI
4. Quiz Generation
→ Summarized text + quiz templates → QuizGen.py → Generated quizzes → UI

##### Architectural Patterns & Principles
- Modular Pipeline: clear separation of concerns (transcription, embedding, retrieval, generation)
- RAG (Retrieval-Augmented Generation) pattern for Q&A
- Template-driven prompt management for extensibility
- Single-process orchestration (no heavy microservices)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% UI Layer
    subgraph "UI Layer"
        App["Streamlit UI"]:::ui
    end

    %% Processing Pipeline
    subgraph "Processing Pipeline"
        Transcribe("Transcription Service\n(Whisper)"):::service
        Embed("Embedding Service\n(All-MPNet)"):::service
        FAISS(["Vector Store\n(FAISS)"]):::storage
    end

    %% Data Artifacts
    Transcript[/Transcript/]:::data
    Embeddings[/Embeddings/]:::data
    Summary[/Summary/]:::data
    Answer[/Answer/]:::data
    QuizData[/Quiz/]:::data

    %% Modules
    Summ("Summarization Module\n(Llama3/Gemini)"):::external
    QA("RAG QA Module"):::external
    Quiz("Quiz Generation Module"):::service
    PromptRepo["Prompt Repository"]:::prompt
    subgraph "Setup"
        Setup["Setup.py & requirements.txt"]:::env
    end

    %% Relationships
    App -->|"upload media"| Transcribe
    Transcribe -->|"media → transcript"| Transcript
    Transcript --> Embed
    Embed -->|"transcript → embeddings"| Embeddings
    Embeddings --> FAISS

    FAISS -->|"for summarization"| Summ
    FAISS -->|"for retrieval"| QA

    Summ -->|"generate summary"| Summary
    Summary --> App

    App -->|"ask question"| QA
    QA -->|"generate answer"| Answer
    Answer --> App

    Summary -->|"summarized text"| Quiz
    Quiz -->|"generate quiz"| QuizData
    QuizData --> App

    PromptRepo --> Summ
    PromptRepo --> QA
    PromptRepo --> Quiz

    Setup --> App

    %% Click Events
    click App "https://github.com/siddarthaa/whisperx/blob/main/App.py"
    click Transcribe "https://github.com/siddarthaa/whisperx/blob/main/App.py"
    click Embed "https://github.com/siddarthaa/whisperx/blob/main/App.py"
    click FAISS "https://github.com/siddarthaa/whisperx/blob/main/App.py"
    click Summ "https://github.com/siddarthaa/whisperx/blob/main/App.py"
    click Summ "https://github.com/siddarthaa/whisperx/blob/main/Prompts/Summarization.txt"
    click QA "https://github.com/siddarthaa/whisperx/blob/main/App.py"
    click QA "https://github.com/siddarthaa/whisperx/blob/main/Prompts/Question.txt"
    click Quiz "https://github.com/siddarthaa/whisperx/blob/main/QuizGen.py"
    click PromptRepo "https://github.com/siddarthaa/whisperx/tree/main/Prompts/"
    click Setup "https://github.com/siddarthaa/whisperx/blob/main/Setup.py"
    click Setup "https://github.com/siddarthaa/whisperx/blob/main/requirements.txt"

    %% Styles
    classDef ui fill:#89CFF0,stroke:#333,stroke-width:1px
    classDef service fill:#8FBC8F,stroke:#333,stroke-width:1px
    classDef storage fill:#FFA500,stroke:#333,stroke-width:1px
    classDef external fill:#D3D3D3,stroke:#333,stroke-width:1px
    classDef data fill:#FFE4B5,stroke:#333,stroke-width:1px
    classDef prompt fill:#E0E0E0,stroke:#333,stroke-width:1px
    classDef env fill:#E0E0E0,stroke:#333,stroke-width:1px
```
