# rootsongjc/rag-chatbot architecture

> Architecture diagram of the GitHub repository rootsongjc/rag-chatbot, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rootsongjc/rag-chatbot
- Repository: https://github.com/rootsongjc/rag-chatbot

## Overview

##### Project type and purpose
- This is a full-stack “Retrieval-Augmented Generation (RAG) Chatbot” project.
- Runtime lives in a Cloudflare Worker (“worker.ts”), exposing a `/chat` endpoint for RAG queries and admin endpoints for index management.
- There is also a small embeddable JavaScript widget (`widget.js` generated on deploy) that your Hugo site includes.
- Offline ingestion/reindexing scripts read local Hugo Markdown content, chunk it, embed it, and write vectors to a Cloudflare Vectorize index.

##### File structure
Top-level:
- src/ – Worker entrypoint + RAG pipeline code
– providers/ – wrappers for embedding and LLM calls (Gemini vs. Qwen)
– rag/ – chunking (`chunk.ts`), retrieval (`retriever.ts`), prompt assembly (`prompt.ts`), title dictionary
– utils/ – markdown parsing, schema definitions
– worker.ts – binds HTTP routes, marshals requests to RAG components
- scripts/ – CLI ingestion utilities
– ingest.ts, manual-ingest.ts, full-reindex.ts, generate-title-dictionary.ts
- tests/ – unit, integration, e2e
- config/build – tsconfig, vitest, playwright, wrangler.toml

##### README for additional insights
- Architecture described in narrative: Markdown → Plain→Chunks→Embeddings→Vector Store; `/chat` retrieves top-K chunks + calls LLM; returns answers + source refs.
- Uses Cloudflare Vectorize binding for persistent vector index.
- Widget is a simple JavaScript client that calls the `/chat` Worker endpoint.
- Supports Gemini (via Google API) or Qwen provider.
- Admin endpoints clear and upsert vectors for full-reindex or manual ingest.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Content Source
    ContentSource["Markdown Files"]:::external

    %% Offline Ingestion Pipeline
    subgraph "Offline Ingestion Pipeline"
        direction TB
        FullReindex["full-reindex.ts"]:::offline
        IngestScript["ingest.ts"]:::offline
        ManualIngest["manual-ingest.ts"]:::offline
        GenerateTitleDict["generate-title-dictionary.ts"]:::offline
        Chunker["chunk.ts"]:::offline
        TitleDictionary["title-dictionary.ts"]:::offline
        EmbeddingsProvider["embeddings.ts"]:::utils
        VectorStoreClient["Vector Store Client"]:::offline
        VectorizeService["Cloudflare Vectorize"]:::external
        FullReindex -->|reads Markdown| Chunker
        IngestScript -->|reads Markdown| Chunker
        ManualIngest -->|reads Markdown| Chunker
        Chunker -->|chunks content| EmbeddingsProvider
        EmbeddingsProvider -->|embeds chunks| VectorStoreClient
        VectorStoreClient -->|writes vectors| VectorizeService
        GenerateTitleDict -->|builds titles| TitleDictionary
    end

    %% Runtime Pipeline
    subgraph "Online Chat Runtime" 
        direction TB
        Widget["widget.js"]:::frontend
        WorkerEntrypoint["worker.ts"]:::runtime
        Retriever["retriever.ts"]:::runtime
        PromptBuilder["prompt.ts"]:::runtime
        LLMProvider["llm.ts"]:::utils
        KVStore["Optional KV Namespace"]:::runtime
        VectorStoreClientRuntime["Vector Store Client"]:::runtime
        VectorizeServiceRuntime["Cloudflare Vectorize"]:::external
        LLMAPI["LLM API (Gemini/Qwen)"]:::external
        Widget -->|POST /chat| WorkerEntrypoint
        WorkerEntrypoint -->|retrieve vectors| Retriever
        Retriever -->|fetch top-K| VectorStoreClientRuntime
        VectorStoreClientRuntime -->|reads vectors| VectorizeServiceRuntime
        Retriever -->|pass chunks| PromptBuilder
        PromptBuilder -->|assemble prompt| LLMProvider
        LLMProvider -->|call API| LLMAPI
        LLMAPI -->|response| WorkerEntrypoint
        WorkerEntrypoint -->|returns JSON| Widget
        WorkerEntrypoint -->|optional| KVStore
    end

    %% Tests
    subgraph Tests
        direction TB
        E2E["tests/e2e/multilingual.spec.ts"]:::test
        Integration["tests/integration/ingest-retriever.test.ts"]:::test
        UnitIngest["tests/unit/ingest.test.ts"]:::test
        UnitRetriever["tests/unit/retriever.test.ts"]:::test
    end

    %% Config
    subgraph Config
        Wrangler["wrangler.toml"]:::config
    end

    %% Click Events
    click FullReindex "https://github.com/rootsongjc/rag-chatbot/blob/main/scripts/full-reindex.ts"
    click IngestScript "https://github.com/rootsongjc/rag-chatbot/blob/main/scripts/ingest.ts"
    click ManualIngest "https://github.com/rootsongjc/rag-chatbot/blob/main/scripts/manual-ingest.ts"
    click GenerateTitleDict "https://github.com/rootsongjc/rag-chatbot/blob/main/scripts/generate-title-dictionary.ts"
    click Chunker "https://github.com/rootsongjc/rag-chatbot/blob/main/src/rag/chunk.ts"
    click Retriever "https://github.com/rootsongjc/rag-chatbot/blob/main/src/rag/retriever.ts"
    click PromptBuilder "https://github.com/rootsongjc/rag-chatbot/blob/main/src/rag/prompt.ts"
    click TitleDictionary "https://github.com/rootsongjc/rag-chatbot/blob/main/src/rag/title-dictionary.ts"
    click EmbeddingsProvider "https://github.com/rootsongjc/rag-chatbot/blob/main/src/providers/embeddings.ts"
    click LLMProvider "https://github.com/rootsongjc/rag-chatbot/blob/main/src/providers/llm.ts"
    click Widget "https://github.com/rootsongjc/rag-chatbot/blob/main/src/widget.js"
    click WorkerEntrypoint "https://github.com/rootsongjc/rag-chatbot/blob/main/src/worker.ts"
    click Wrangler "https://github.com/rootsongjc/rag-chatbot/blob/main/wrangler.toml"
    click E2E "https://github.com/rootsongjc/rag-chatbot/blob/main/tests/e2e/multilingual.spec.ts"
    click Integration "https://github.com/rootsongjc/rag-chatbot/blob/main/tests/integration/ingest-retriever.test.ts"
    click UnitIngest "https://github.com/rootsongjc/rag-chatbot/blob/main/tests/unit/ingest.test.ts"
    click UnitRetriever "https://github.com/rootsongjc/rag-chatbot/blob/main/tests/unit/retriever.test.ts"

    %% Styles
    classDef offline fill:#ADD8E6,stroke:#000,stroke-width:1px
    classDef runtime fill:#90EE90,stroke:#000,stroke-width:1px
    classDef external fill:#FFA500,stroke:#000,stroke-width:1px
    classDef frontend fill:#D3D3D3,stroke:#000,stroke-width:1px
    classDef utils fill:#EEEEEE,stroke:#000,stroke-width:1px
    classDef test fill:#FFD700,stroke:#000,stroke-width:1px
    classDef config fill:#DDA0DD,stroke:#000,stroke-width:1px
```
