# theaiautomators/insights-lm-public architecture

> Architecture diagram of the GitHub repository theaiautomators/insights-lm-public, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/theaiautomators/insights-lm-public
- Repository: https://github.com/theaiautomators/insights-lm-public

## Overview

##### Project Type and Purpose
- InsightsLM is a full-stack, self-hostable, retriev­al-augmented generation (RAG) web application.
- Frontend: a React + TypeScript single-page app (SPA) built with Vite, Tailwind CSS, and shadcn-ui.
- Backend: Supabase (PostgreSQL database, authentication, storage) plus Supabase Edge Functions for custom logic, and n8n for workflow automation and integration with AI services (OpenAI, Gemini).

##### File Structure
- Top-level directories:
- src/ — the entire frontend code (components, hooks, pages, contexts, services).
- supabase/ — database migrations and Edge Functions (TypeScript handlers).
- n8n/ — JSON exports of workflow definitions that run on the n8n server.
- Configuration files:
- supabase/config.toml — Supabase project config.
- tailwind.config.ts, postcss.config.js — CSS build setup.
- vite.config.ts — frontend bundler config.
- .github/workflows/deploy.yml — CI/CD pipeline for deployment.
- Scripts:
- patch-supabase.js — one-time DB patch tool.

##### Key Architectural Components
a. Client Browser (User)
b. React SPA (frontend)
c. Supabase:
– Database (PostgreSQL + vector extension)
– Auth & Storage
– Edge Functions (HTTP endpoints in supabase/functions/)
d. n8n Workflow Engine
– Hosted separately or via cloud
– Workflows imported from n8n/*.json
e. External AI Services
– OpenAI, Gemini (for embeddings, chat, transcription, TTS)
f. CI/CD
– GitHub Actions deploy pipeline

##### Relationships and Data Flows
1. User interacts with the React SPA in their browser.
2. SPA uses the Supabase client (src/integrations/supabase/client.ts) to authenticate and read/write data.
3. For complex operations (document processing, chat, podcast generation):
- SPA calls a Supabase Edge Function (e.g., send-chat-message).
- Edge Function issues an authenticated webhook request to n8n.
- n8n workflow orchestrates calls to AI services and Supabase (vector upserts, row inserts).
- n8n returns results, Edge Function persists final data in Supabase, or triggers real-time updates.
4. SPA subscribes or polls database tables for new chat messages, notes, audio URLs, etc.

##### Architectural Patterns and Design Principles
- Microservices Style:
- Each Edge Function is its own HTTP micro-endpoint.
- n8n represents a separate service for business-logic workflows.
- RAG Pattern:
- Documents are processed into embeddings stored in the Supabase vector extension.
- Chat queries perform vector searches, then run LLM calls.
- Real-time / Event-Driven:
- Supabase Realtime for pushing updates to the UI when new data arrives.
- Separation of Concerns:
- UI (React) vs. Business Logic (Edge Functions + n8n) vs. Data Storage (PostgreSQL).

##### Detailed Capture
- N8n/Import_Insights_LM_Workflows.json as the master workflow that deploys sub-workflows.
- Database table for vectors and notes, referencing the migration SQL.
- Supabase Secrets used for secure webhook authentication.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Client Browser and SPA
    subgraph "Client Browser / React SPA"
        direction TB
        Browser["User Browser"]:::frontend
        Main["Entry: main.tsx"]:::frontend
        App["Root: App.tsx"]:::frontend
        Pages["SPA Pages"]:::frontend
        UI["UI Components"]:::frontend
        AuthComp["Auth Components"]:::frontend
        ChatComp["Chat Components"]:::frontend
        DashComp["Dashboard Components"]:::frontend
        NoteComp["Notebook Components"]:::frontend
        Hooks["Custom Hooks"]:::frontend
        AuthCtx["Auth Context"]:::frontend
        SupClient["Supabase Client"]:::frontend
        SupTypes["Supabase Types"]:::frontend
    end

    %% Supabase Services
    subgraph "Supabase" 
        direction TB
        DB["Database (PostgreSQL + vector)"]:::backend
        AuthStorage["Auth & Storage"]:::backend
        subgraph "Edge Functions"
            direction TB
            EF1["send-chat-message"]:::backend
            EF2["process-document"]:::backend
            EF3["process-additional-sources"]:::backend
            EF4["process-document-callback"]:::backend
            EF5["generate-notebook-content"]:::backend
            EF6["generate-note-title"]:::backend
            EF7["generate-audio-overview"]:::backend
            EF8["audio-generation-callback"]:::backend
            EF9["refresh-audio-url"]:::backend
            EF10["webhook-handler"]:::backend
        end
        Migrations["Migrations"]:::backend
        Config["Config"]:::backend
        Realtime["Realtime Updates"]:::backend
    end

    %% n8n Workflow Engine
    subgraph "n8n Workflow Engine"
        direction TB
        Importer["Master Importer"]:::workflow
        ChatWF["Chat Workflow"]:::workflow
        ExtractWF["Text Extraction Workflow"]:::workflow
        NotebookWF["Notebook Detail Generation"]:::workflow
        PodcastWF["Podcast Generation"]:::workflow
        AdditionalWF["Additional Sources Workflow"]:::workflow
        UpsertWF["Vector Upsert Workflow"]:::workflow
    end

    %% External AI Services
    subgraph "External AI Services"
        direction TB
        AI["OpenAI / Gemini"]:::external
    end

    %% CI/CD
    CI["CI/CD: GitHub Actions"]:::ci

    %% Connections
    Browser --> Main
    Main --> App
    App --> Pages
    App --> UI
    App --> AuthComp
    App --> ChatComp
    App --> DashComp
    App --> NoteComp
    App --> Hooks
    App --> AuthCtx
    App --> SupClient
    App --> SupTypes

    App -->|"Auth request"| AuthStorage
    App -->|"HTTP call"| EF1
    App -->|"HTTP call"| EF2

    EF1 -->|"Webhook"| Importer
    EF2 -->|"Webhook"| Importer

    Importer --> ChatWF
    Importer --> ExtractWF
    Importer --> NotebookWF
    Importer --> PodcastWF
    Importer --> AdditionalWF
    Importer --> UpsertWF

    ChatWF --> AI
    ExtractWF --> AI
    NotebookWF --> AI
    PodcastWF --> AI
    AdditionalWF --> AI
    UpsertWF --> DB

    ChatWF -->|"Upsert / Insert"| DB
    NotebookWF -->|"Upsert / Insert"| DB
    PodcastWF -->|"Upsert / Insert"| DB

    DB --> Realtime
    Realtime --> Browser

    CI -->|"Deploy migrations"| Migrations
    CI -->|"Deploy functions"| EF1
    CI -->|"Import workflows"| Importer

    %% Click Events for SPA
    click Main "https://github.com/theaiautomators/insights-lm-public/blob/main/src/main.tsx"
    click App "https://github.com/theaiautomators/insights-lm-public/blob/main/src/App.tsx"
    click Pages "https://github.com/theaiautomators/insights-lm-public/tree/main/src/pages/"
    click UI "https://github.com/theaiautomators/insights-lm-public/tree/main/src/components/ui/"
    click AuthComp "https://github.com/theaiautomators/insights-lm-public/tree/main/src/components/auth/"
    click ChatComp "https://github.com/theaiautomators/insights-lm-public/tree/main/src/components/chat/"
    click DashComp "https://github.com/theaiautomators/insights-lm-public/tree/main/src/components/dashboard/"
    click NoteComp "https://github.com/theaiautomators/insights-lm-public/tree/main/src/components/notebook/"
    click Hooks "https://github.com/theaiautomators/insights-lm-public/tree/main/src/hooks/"
    click AuthCtx "https://github.com/theaiautomators/insights-lm-public/blob/main/src/contexts/AuthContext.tsx"
    click SupClient "https://github.com/theaiautomators/insights-lm-public/blob/main/src/integrations/supabase/client.ts"
    click SupTypes "https://github.com/theaiautomators/insights-lm-public/blob/main/src/integrations/supabase/types.ts"

    %% Click Events for Supabase
    click EF1 "https://github.com/theaiautomators/insights-lm-public/blob/main/supabase/functions/send-chat-message/index.ts"
    click EF2 "https://github.com/theaiautomators/insights-lm-public/blob/main/supabase/functions/process-document/index.ts"
    click EF3 "https://github.com/theaiautomators/insights-lm-public/blob/main/supabase/functions/process-additional-sources/index.ts"
    click EF4 "https://github.com/theaiautomators/insights-lm-public/blob/main/supabase/functions/process-document-callback/index.ts"
    click EF5 "https://github.com/theaiautomators/insights-lm-public/blob/main/supabase/functions/generate-notebook-content/index.ts"
    click EF6 "https://github.com/theaiautomators/insights-lm-public/blob/main/supabase/functions/generate-note-title/index.ts"
    click EF7 "https://github.com/theaiautomators/insights-lm-public/blob/main/supabase/functions/generate-audio-overview/index.ts"
    click EF8 "https://github.com/theaiautomators/insights-lm-public/blob/main/supabase/functions/audio-generation-callback/index.ts"
    click EF9 "https://github.com/theaiautomators/insights-lm-public/blob/main/supabase/functions/refresh-audio-url/index.ts"
    click EF10 "https://github.com/theaiautomators/insights-lm-public/blob/main/supabase/functions/webhook-handler/index.ts"
    click Migrations "https://github.com/theaiautomators/insights-lm-public/blob/main/supabase/migrations/20250606152423_v0.1.sql"
    click Config "https://github.com/theaiautomators/insights-lm-public/blob/main/supabase/config.toml"

    %% Click Events for n8n
    click Importer "https://github.com/theaiautomators/insights-lm-public/blob/main/n8n/Import_Insights_LM_Workflows.json"
    click ChatWF "https://github.com/theaiautomators/insights-lm-public/blob/main/n8n/InsightsLM___Chat.json"
    click ExtractWF "https://github.com/theaiautomators/insights-lm-public/blob/main/n8n/InsightsLM___Extract_Text.json"
    click NotebookWF "https://github.com/theaiautomators/insights-lm-public/blob/main/n8n/InsightsLM___Generate_Notebook_Details.json"
    click PodcastWF "https://github.com/theaiautomators/insights-lm-public/blob/main/n8n/InsightsLM___Podcast_Generation.json"
    click AdditionalWF "https://github.com/theaiautomators/insights-lm-public/blob/main/n8n/InsightsLM___Process_Additional_Sources.json"
    click UpsertWF "https://github.com/theaiautomators/insights-lm-public/blob/main/n8n/InsightsLM___Upsert_to_Vector_Store.json"

    %% Click Events for CI/CD
    click CI "https://github.com/theaiautomators/insights-lm-public/blob/main/.github/workflows/deploy.yml"

    %% Styles
    classDef frontend fill:#E0F2FE,stroke:#0284C7,color:#0369A1;
    classDef backend fill:#DCFCE7,stroke:#22C55E,color:#166534;
    classDef workflow fill:#FEF9C3,stroke:#EAB308,color:#92400E;
    classDef external fill:#F3E8FF,stroke:#A78BFA,color:#5B21B6,stroke-dasharray: 5 5;
    classDef ci fill:#F3F4F6,stroke:#9CA3AF,color:#374151;
```
