# rmusser01/tldw architecture

> Architecture diagram of the GitHub repository rmusser01/tldw, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rmusser01/tldw
- Repository: https://github.com/rmusser01/tldw

## Overview

##### Main Components
- User Interface Layer:
- Temporary Gradio UI (runs as the GUI when using “python summarize.py -gui”)
- Planned FastAPI backend with intended JavaScript frontend (as noted in the latest update)
- Backend/API Layer:
- “Server_API” module that contains the API endpoints (look under Server_API/app/api/v1) and core business logic (Server_API/app/core) for media ingestion, transcription, chat, and summarization.
- Application Logic/Processing Libraries:
- “App_Function_Libraries” folder holds modular libraries for key to functions such as:
▪ Audio (processing, transcription, diarization)
▪ Chat (chat functions, pipelines, multi-chat styles)
▪ Summarization and RAG (retrieval-enhanced generation – using pipelines in the RAG folder)
▪ Books, PDF ingestion, and plain text conversion (“Plaintext”, “PDF”, “html_to_markdown”, etc.)
▪ LLM API calls (both remote and local interfaces, “LLM_API_Calls” and “Local_LLM”)
▪ Database interactions (via “DB” folder modules that manage SQLite, backups, and prompts)
▪ Metrics and workflow management (see “Metrics”, “Workflows”, and even “Prompt_Engineering” folders)
- Data Storage and Backup:
- “Databases” folder containing SQLite databases, plus backup management modules (also see “Config_Files” for backup configurations)
- Deployment and Infrastructure:
- Dockerfiles (in root, in the “Helper_Scripts/Dockerfiles”, and “HF/”) and CI workflows (in .github/workflows)
- Installer scripts (under Helper_Scripts/Installer_Scripts) and litestream.yml for automated DB backups
- Testing:
- “Tests” directory that covers integration–unit tests for each separate area (audio, chat, DB, etc.)
- External Integrations:
- Third-party API wrappers for commercial LLMs (OpenAI, Cohere, etc.) and local inference engine wrappers (llama.cpp, oobabooga, etc.)
- Tools like yt-dlp for ingesting content from URLs and external libraries for transcription (faster_whisper, pyannote)

##### Relationships and Interactions
- The UI/Frontend (Gradio/Upcoming JS frontend) interacts with the backend (currently mainly via the Server_API application) by sending REST or CLI commands.
- The backend is responsible for coordinating various processing modules:
- When a media item is ingested (either from a URL or local upload) the ingestion module (found in many of the App_Function_Libraries subfolders) downloads and processes content.
- Media files are then passed to transcription modules (using faster_whisper and other audio libraries) and/or summarization modules (which call LLM services via external APIs or local local LLM wrappers).
- Processed/transcribed text is stored in the SQLite databases with additional metadata and then is accessible via search (SQL FTS, RAG pipelines) and chat modules.
- The chat functionality can access both pre-generated summaries and transcripts as well as directly interact with LLM APIs.
- The “DB” modules, along with backup and migration helpers in “Helper_Scripts/DB-Related”, ensure that all data is locally stored and versioned.
- The modular design allows pieces to be reused across functionalities—the same ingestion module might be used whether the user wishes to “just ingest” content or even run evaluations/benchmarks (as seen in the Benchmarks_Evaluations folder).

##### Important Architectural Patterns and Design Principles
- Modular and layered architecture – separation of concerns is apparent from the division between UI, API/backend, processing libraries, and data storage.
- Clean integration points – components such as audio transcription or chat pipelines work as independent libraries that plug into the overall workflow.
- Extensibility – Integration points for multiple LLM APIs and local inference options show that the design anticipates adding more models and capabilities with minimal changes in the core.
- Test-driven design – with separate testing directories for integration, unit tests, and even benchmarking assays.

##### Relevant Technologies and Frameworks
- Python is the primary language; all modules follow a Python package structure.
- Gradio is used for the GUI (with a planned migration to FastAPI and JavaScript).
- SQLite (with FTS5 support) for database storage, with optional litestream integration.
- Tools like yt-dlp, ffmpeg, and others are tied into media ingestion.
- ML libraries such as PyTorch, faster_whisper, and huggingface transformers for transcription, summarization, and inference.
- Docker for deployment, CI/CD via GitHub workflows, and installer scripts across platforms.

##### Instructing the Principal Software Engineer
- UI Layer (e.g., “Gradio UI / Upcoming JS Frontend”)
- API/Backend Layer (e.g., “Server_API Endpoints”, “Core Business Logic”)
- Application Function Libraries (group them by functionality: Audio, Chat, Summarization, Database Management, etc.)
- Data Storage (SQLite DB, Backup mechanisms)
- External Services (LLM APIs, Third-party media download/transcription tools)
- Labels are clear and add brief notes if necessary (e.g., “Media ingestion via yt-dlp”, “Transcription via faster_whisper”, “LLM API calls for summarization”).

##### Final Advice
- Be as detailed as possible; do not hesitate to break down high-level components further if they have multiple submodules (for example, splitting “App_Function_Libraries” into separate areas for Audio, Chat, DB, etc.).
- Dependencies and configuration files (such as Dockerfile, litestream configuration, and installer scripts) to include notes on deployment and maintenance.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% UI Layer
    subgraph "UI Layer"
        A["Gradio UI / Upcoming JS Frontend"]:::ui
    end

    %% API/Backend Layer
    subgraph "API/Backend Layer"
        B["Server_API Endpoints"]:::backend
        C["Core Business Logic"]:::backend
    end

    %% Processing Modules
    subgraph "Processing Modules"
        %% Media Ingestion & Processing
        D1["Upload Module"]:::module
        D2["Local File Processing"]:::module
        
        %% Audio Processing and Transcription
        E1["Audio Processing (Audio)"]:::module
        E2["STT Module"]:::module
        
        %% Summarization
        F["Summarization Modules"]:::module
        
        %% RAG Pipelines
        G["RAG Pipelines"]:::module
        
        %% Chat Functionality
        H1["Chat Module"]:::module
        H2["Character Chat"]:::module
        
        %% LLM API Calls & Local Inference
        I1["LLM API Calls"]:::module
        I2["Local LLM"]:::module
        
        %% External Integrations
        J1["MediaWiki Integration"]:::external
        J2["Third Party API"]:::external
        J3["Web Scraping"]:::external
        
        %% Metrics and Workflow Management
        L1["Metrics"]:::module
        L2["Workflows"]:::module
    end

    %% Data Storage and Backup
    subgraph "Data Storage & Backup"
        K["Database Management & Backup"]:::database
        DB2["DB Backup & Migration"]:::database
    end

    %% Deployment & Infrastructure
    subgraph "Deployment & Infrastructure"
        M1["Root Dockerfile"]:::deployment
        M2["Helper Dockerfiles"]:::deployment
        M3["HF Folder"]:::deployment
        M4["Litestream Config"]:::deployment
        M5["Config Files"]:::deployment
    end

    %% Testing Suite
    subgraph "Testing Suite"
        N["Testing Suite"]:::test
    end

    %% Connections between Layers
    A -->|"sendsRequest"| B
    B -->|"routesTo"| C

    %% From Core Business Logic to Processing Modules
    C --> D1
    C --> D2
    C --> E1
    C --> E2
    C --> F
    C --> G
    C --> H1
    C --> H2
    C --> I1
    C --> I2
    C --> J1
    C --> J2
    C --> J3
    C --> L1
    C --> L2

    %% Processing Modules storing data into Database
    D1 -->|"store"| K
    D2 -->|"store"| K
    E1 -->|"store"| K
    E2 -->|"store"| K
    F  -->|"store"| K
    G  -->|"store"| K
    I1 -->|"store"| K
    I2 -->|"store"| K

    %% Chat Modules retrieving data from Database
    K -->|"retrieve"| H1
    K -->|"retrieve"| H2

    %% DB Backup and Migration sync to Main Database
    DB2 -->|"syncBackup"| K

    %% Deployment links (affecting multiple layers)
    M1 --- A
    M2 --- B
    M3 --- C
    M4 --- K
    M5 --- K

    %% Testing Suite connected to UI and Backend for validation
    N --- A
    N --- C

    %% Styles
    classDef ui fill:#ffccff,stroke:#cc00cc,stroke-width:2px;
    classDef backend fill:#cce5ff,stroke:#0066cc,stroke-width:2px;
    classDef module fill:#d4f7d4,stroke:#008000,stroke-width:2px;
    classDef database fill:#fff2cc,stroke:#ff9900,stroke-width:2px;
    classDef deployment fill:#e0e0e0,stroke:#808080,stroke-width:2px;
    classDef test fill:#ffffcc,stroke:#cccc00,stroke-width:2px;
    classDef external fill:#ffd6d6,stroke:#d00,stroke-width:2px;

    %% Click Events
    click A "https://github.com/rmusser01/tldw/tree/main/App_Function_Libraries/Gradio_UI"
    click B "https://github.com/rmusser01/tldw/tree/main/Server_API/app/api/v1"
    click C "https://github.com/rmusser01/tldw/tree/main/Server_API/app/core"
    click D1 "https://github.com/rmusser01/tldw/tree/main/App_Function_Libraries/Upload"
    click D2 "https://github.com/rmusser01/tldw/blob/main/App_Function_Libraries/Local_File_Processing_Lib.py"
    click E1 "https://github.com/rmusser01/tldw/tree/main/App_Function_Libraries/Audio"
    click E2 "https://github.com/rmusser01/tldw/tree/main/App_Function_Libraries/STT"
    click F "https://github.com/rmusser01/tldw/tree/main/App_Function_Libraries/Summarization"
    click G "https://github.com/rmusser01/tldw/tree/main/App_Function_Libraries/RAG"
    click H1 "https://github.com/rmusser01/tldw/tree/main/App_Function_Libraries/Chat"
    click H2 "https://github.com/rmusser01/tldw/tree/main/App_Function_Libraries/Character_Chat"
    click I1 "https://github.com/rmusser01/tldw/blob/main/App_Function_Libraries/LLM_API_Calls.py"
    click I2 "https://github.com/rmusser01/tldw/tree/main/App_Function_Libraries/Local_LLM"
    click J1 "https://github.com/rmusser01/tldw/tree/main/App_Function_Libraries/MediaWiki"
    click J2 "https://github.com/rmusser01/tldw/tree/main/App_Function_Libraries/Third_Party"
    click J3 "https://github.com/rmusser01/tldw/tree/main/App_Function_Libraries/Web_Scraping"
    click L1 "https://github.com/rmusser01/tldw/tree/main/App_Function_Libraries/Metrics"
    click L2 "https://github.com/rmusser01/tldw/tree/main/App_Function_Libraries/Workflows"
    click K "https://github.com/rmusser01/tldw/tree/main/App_Function_Libraries/DB"
    click DB2 "https://github.com/rmusser01/tldw/tree/main/Helper_Scripts/DB-Related"
    click M1 "https://github.com/rmusser01/tldw/tree/main/Dockerfile"
    click M2 "https://github.com/rmusser01/tldw/tree/main/Helper_Scripts/Dockerfiles"
    click M3 "https://github.com/rmusser01/tldw/tree/main/HF/"
    click M4 "https://github.com/rmusser01/tldw/blob/main/litestream.yml"
    click M5 "https://github.com/rmusser01/tldw/tree/main/Config_Files"
    click N "https://github.com/rmusser01/tldw/tree/main/Tests directory"
```
