# devsp1312/serendipity architecture

> Architecture diagram of the GitHub repository devsp1312/serendipity, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-03-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/devsp1312/serendipity
- Repository: https://github.com/devsp1312/serendipity

## Overview

**Serendipity is a single-machine, fully local, two-stage ML pipeline** that:
- Ingests **audio files** from a local folder
- Runs **Stage 1 transcription** (Faster-Whisper) to produce timestamped text
- Runs **Stage 2 analysis** (multiple NLP/audio models) to generate:
- audio emotion, text emotion (27 labels), likes/dislikes, “person type”
- **HTML report**
- optional longitudinal **speaker profiles** (19 metrics across sessions)
- Exposes a **local Flask Web UI** that orchestrates the pipeline and streams progress via **SSE** (Server-Sent Events)

So the “system” is not a distributed architecture; it’s a **local app composed of CLI + local web server + ML inference modules + on-disk artifacts + cached models**.

- The **two-stage pipeline boundary**
- The **file-based handoffs/caching**
- The **orchestration paths** (Web UI vs CLI)
- The **model caches** and “download models” bootstrapping steps
- The **outputs** (JSON, reports, profiles) and how they feed back into later sessions

##### Derive the architecture from the file tree + README (what to include)
From the README structure:

##### Core runtime components
- **Stage 1 (transcription)**: `stage1/transcribe.py`, `stage1/download_model.py`, `stage1/model/` cache
- **Stage 2 (analysis + UI)**:
- Web server: `stage2/app.py` (Flask + SSE streaming)
- CLI analysis: `stage2/analyse.py`
- Analysis modules:
- `audio_emotion.py` (wav2vec2 emotion)
- `text_emotion.py` (GoEmotions BERT)
- `likes_dislikes.py` (sentiment + topic extraction)
- `person_type.py` (zero-shot NLI)
- `profiles.py` (19-metric aggregation across sessions)
- Model bootstrap: `download_models.py`, `stage2/models/` cache

##### Data / artifact directories (important: these are “storage subsystems”)
- `Audio files/` (input)
- `Text files/` (transcripts + segments)
- `JSON files/` (analysis outputs)
- `Reports/` (HTML output)
- `Profiles/` (long-lived aggregated speaker profiles)

##### External dependencies (represent as “libraries/models,” not network services)
- HuggingFace/transformers model artifacts (downloaded once, then offline)
- Faster-Whisper model (downloaded once, then offline)
- Hardware acceleration optional: CPU default; GPU if available (MPS/CUDA)

##### Optional: add to profile → update `Profiles/<speaker>.json`

**SSE progress stream** from Flask back to browser during steps.

##### Architectural patterns worth explicitly highlighting (because they’re key here)
- **Two-stage pipeline** with a hard boundary:
- Stage 1: speech-to-text
- Stage 2: multi-model analysis + reporting + profiles
- **File-based caching and idempotency**:
- transcript cache means re-analysis skips Stage 1
- **Local-first / offline-first**:
- downloads occur once; runtime is offline
- **Orchestration via UI**:
- Flask app orchestrates and provides a progress/event channel (SSE)
- **Pluggable step modules**:

##### User & Interfaces
- User
- Browser (index.html)
- CLI terminal (optional branch)

##### Orchestrator
- Flask Web Server `stage2/app.py`

##### Pipeline (boxed as ‘Analysis Pipeline’)
- Step 1: Transcription `stage1/transcribe.py` (with “skip if cached” note)
- Step 2: `audio_emotion.py`
- Step 3: `text_emotion.py`
- Step 4: `likes_dislikes.py`
- Step 5: `person_type.py`
- Step 6: `profiles.py` (conditional on “Add to Profile” or profile mode)

- Audio files/
- Text files/
- JSON files/
- Reports/
- Profiles/

##### Model caches (side row)
- stage1/model (Whisper)
- stage2/models (wav2vec2, goemotions, sentiment, zero-shot)

This layout matches how the repo is mentally operated: UI/CLI triggers → pipeline steps → filesystem artifacts → UI displays.

##### Don’t overthink it: maximize components (as requested)
- Keep each folder as its own storage node
- Distinguish “model download” scripts from “runtime inference” modules
- SSE as an explicit channel, not an implementation detail

## Diagram (Mermaid)

```mermaid
flowchart TD
  %% =========================
  %% System Context (Local App)
  %% =========================
  subgraph "System Context (Single Machine, Local-First)"
    U["User"]:::external
    B["Browser (Local)"]:::ui
    T["CLI Terminal (Local)"]:::ui
    EXT["Model Hosting (HuggingFace / Whisper)\n(setup only)"]:::external
  end

  %% =========================
  %% Web UI + Orchestration
  %% =========================
  subgraph "Stage 2 Orchestration (Web UI)"
    APP["Flask Web Server\n(stage2/app.py)"]:::orchestrator
    SSE["SSE Progress Stream\n(step start/end, elapsed,\nheartbeat 15s)"]:::sse
  end

  %% =========================
  %% Pipeline (Two Stages)
  %% =========================
  subgraph "Two-Stage ML Pipeline"
    direction TB

    subgraph "Stage 1: Transcription (Faster-Whisper)"
      direction TB
      S1_DL["Download Whisper Model\n(stage1/download_model.py)"]:::bootstrap
      S1_TX["Audio Transcription\n(stage1/transcribe.py)"]:::compute
      S1_REQ["Stage 1 Python Deps\n(stage1/requirements.txt)"]:::deps
      S1_CACHE(("stage1/model/\n(Whisper model cache)")):::modelcache
      DEC{"Transcript\ncache hit?"}:::decision
    end

    subgraph "Stage 2: Analysis + Reporting"
      direction TB
      S2_ROOT["Stage 2 Module Root\n(stage2/)"]:::module
      S2_DL["Download Analysis Models\n(stage2/download_models.py)"]:::bootstrap
      S2_CACHE(("stage2/models/\n(wav2vec2, GoEmotions,\nsentiment, zero-shot)")):::modelcache

      AE["Audio Emotion\n(stage2/audio_emotion.py)\nCPU default; GPU if available\n(MPS/CUDA)"]:::compute
      TE["Text Emotion (27 labels)\n(stage2/text_emotion.py)"]:::compute
      LD["Likes/Dislikes\n(stage2/likes_dislikes.py)"]:::compute
      PT["Person Type\n(stage2/person_type.py)"]:::compute
      PR["Profiles Aggregator (optional)\n(stage2/profiles.py)\n(19 metrics across sessions)"]:::compute

      VIS["HTML Report Generation\n(stage2/visualise.py)"]:::compute
      CLI_A["CLI Analysis Runner\n(stage2/analyse.py)"]:::orchestrator
    end
  end

  %% =========================
  %% Storage / Artifacts (On Disk)
  %% =========================
  subgraph "On-Disk Artifacts (File-Based Handoffs + Caching)"
    direction TB
    AUD(("Audio files/\n(.mp3/.wav input)")):::storage
    TXT(("Text files/\n(.txt + .segments.json cache)")):::storage
    JSO(("JSON files/\n(analysis outputs .json)")):::storage
    REP(("Reports/\n(.html output)")):::storage
    PRO(("Profiles/\n(<speaker>.json longitudinal state)")):::storage
  end

  %% =========================
  %% Documentation
  %% =========================
  DOC["README / Architecture Notes\n(README.md)"]:::doc

  %% =========================
  %% Control Flow: UI + CLI
  %% =========================
  U -->|"uses"| B
  U -->|"runs"| T

  B -->|"loads UI / selects audio / run analysis"| APP
  APP -->|"streams progress"| SSE
  SSE -->|"SSE events"| B

  T -->|"run Stage 1 directly"| S1_TX
  T -->|"run analysis pipeline"| CLI_A

  APP -->|"invokes pipeline"| CLI_A

  %% =========================
  %% Dataflow + Cache Decision
  %% =========================
  AUD -->|"read .mp3/.wav"| S1_TX
  TXT -->|"check existing transcript"| DEC
  DEC -->|"hit: skip Stage 1"| CLI_A
  DEC -->|"miss: transcribe"| S1_TX
  S1_TX -->|"write .txt + .segments.json"| TXT
  S1_TX -->|"hand off transcript path"| CLI_A

  %% Stage 2 uses audio + transcript
  AUD -->|"read audio"| AE
  TXT -->|"read transcript/segments"| TE
  TXT -->|"read transcript"| LD
  TXT -->|"read transcript"| PT

  %% Inference outputs
  AE -->|"write .json"| JSO
  TE -->|"write .json"| JSO
  LD -->|"write .json"| JSO
  PT -->|"write .json"| JSO

  %% Reporting
  JSO -->|"render"| VIS
  VIS -->|"write .html"| REP
  APP -->|"serve report view"| B

  %% Profiles (optional)
  CLI_A -->|"optional: add to profile"| PR
  TXT -->|"session transcript features"| PR
  JSO -->|"session metrics"| PR
  PR -->|"update <speaker>.json"| PRO

  %% =========================
  %% Model Bootstrap (Setup-only / Offline after)
  %% =========================
  EXT -.->|"setup only / offline after"| S1_DL
  EXT -.->|"setup only / offline after"| S2_DL

  S1_DL -.->|"download weights"| S1_CACHE
  S2_DL -.->|"download weights"| S2_CACHE

  %% Runtime model usage
  S1_TX -->|"load model"| S1_CACHE
  AE -->|"load wav2vec2"| S2_CACHE
  TE -->|"load GoEmotions BERT"| S2_CACHE
  LD -->|"load sentiment/topic models"| S2_CACHE
  PT -->|"load zero-shot MNLI"| S2_CACHE

  %% Stage 2 root (module boundary)
  S2_ROOT -->|"contains"| APP
  S2_ROOT -->|"contains"| CLI_A
  S2_ROOT -->|"contains"| AE
  S2_ROOT -->|"contains"| TE
  S2_ROOT -->|"contains"| LD
  S2_ROOT -->|"contains"| PT
  S2_ROOT -->|"contains"| PR
  S2_ROOT -->|"contains"| VIS
  S2_ROOT -->|"contains"| S2_DL

  %% =========================
  %% Docs reference
  %% =========================
  U -->|"reads"| DOC

  %% =========================
  %% Click Events (from component_mapping)
  %% =========================
  click S1_TX "https://github.com/devsp1312/serendipity/blob/main/stage1/transcribe.py"
  click S1_DL "https://github.com/devsp1312/serendipity/blob/main/stage1/download_model.py"
  click S1_REQ "https://github.com/devsp1312/serendipity/blob/main/stage1/requirements.txt"
  click S2_ROOT "https://github.com/devsp1312/serendipity/tree/main/stage2/"
  click TXT "https://github.com/devsp1312/serendipity/tree/main/Text files/"
  click JSO "https://github.com/devsp1312/serendipity/tree/main/JSON files/"
  click REP "https://github.com/devsp1312/serendipity/tree/main/Reports/"
  click PRO "https://github.com/devsp1312/serendipity/tree/main/Profiles/"
  click DOC "https://github.com/devsp1312/serendipity/blob/main/README.md"

  %% =========================
  %% Styles
  %% =========================
  classDef external fill:#eef2f7,stroke:#64748b,color:#0f172a,stroke-width:1px
  classDef ui fill:#dbeafe,stroke:#2563eb,color:#0b1b3a,stroke-width:1px
  classDef orchestrator fill:#bfdbfe,stroke:#1d4ed8,color:#0b1b3a,stroke-width:1px
  classDef sse fill:#93c5fd,stroke:#1e40af,color:#0b1b3a,stroke-width:1.5px
  classDef compute fill:#dcfce7,stroke:#16a34a,color:#052e16,stroke-width:1px
  classDef storage fill:#ffedd5,stroke:#ea580c,color:#3b1d0a,stroke-width:1px
  classDef modelcache fill:#e5e7eb,stroke:#6b7280,color:#111827,stroke-width:1px
  classDef bootstrap fill:#e5e7eb,stroke:#4b5563,color:#111827,stroke-width:1px,stroke-dasharray: 4 3
  classDef decision fill:#fde68a,stroke:#b45309,color:#3b1d0a,stroke-width:1px
  classDef deps fill:#f3f4f6,stroke:#9ca3af,color:#111827,stroke-width:1px
  classDef module fill:#e0f2fe,stroke:#0284c7,color:#082f49,stroke-width:1px
  classDef doc fill:#fef9c3,stroke:#ca8a04,color:#3b1d0a,stroke-width:1px
```
