# umbertogriffo/rag-chatbot architecture

> Architecture diagram of the GitHub repository umbertogriffo/rag-chatbot, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/umbertogriffo/rag-chatbot
- Repository: https://github.com/umbertogriffo/rag-chatbot

## Overview

##### Project type and purpose
- This is a local, full-stack Python application that implements two variants of a chatbot:
a) A “Conversation-aware” Chatbot (stateful, ChatGPT-like)
b) A RAG (Retrieval-Augmented Generation) Chatbot
- It uses:
- llama-cpp-python as the local LLM backend
- Chroma as the vector store for embeddings
- Streamlit (and a CLI) for the user interface
- A custom “Memory Builder” to index Markdown docs

##### File structure
- Top-level scripts & apps:
- chatbot/chatbot_app.py (Streamlit GUI for conversation-only)
- chatbot/rag_chatbot_app.py (Streamlit GUI for RAG)
- chatbot/cli/{chatbot.py, rag_chatbot.py} (command-line entrypoints)
- chatbot/memory_builder.py (builds embedding index from docs/)
- Core modules under chatbot/bot/:
- client/ (llama_cpp_client.py: interfaces with llama-cpp-python; prompt templates)
- model/ (BaseModel abstraction, model registry, per-model settings)
- memory/ (embedder, vector_database wrapper for Chroma, distance metrics)
- conversation/ (chat_history, conversation_handler, context strategies)
- Data loaders: chatbot/document_loader/ (text splitting, format parsing, loader)
- Entities: chatbot/entities/document.py
- External persistence: docs/ (source Markdown), vector_store/ (Chroma persistence)

##### README for operational flow
- Offline “Memory Builder” loads docs/ → chunks via text_splitter → computes embeddings → stores in Chroma DB
- At runtime, user asks a question via GUI or CLI → the correct “chat app” module invokes the Bot
a) Conversation mode: uses conversation_handler + chat_history → prompts LLM via llama_cpp_client
b) RAG mode:
1. (optional) rewrite user query via LLM
2. retrieve top-k chunks from Chroma
3. synthesis strategy (“create-and-refine” or “(async) tree summarization”) → orchestrates multiple LLM calls
4. returns final answer

##### Main components
A. User Interface Layer
- CLI (chatbot.py, rag_chatbot.py)
- Streamlit UI (chatbot_app.py, rag_chatbot_app.py)
B. Application Layer
1. ChatRouter (entry point that picks Conversation vs. RAG)
2. Conversation Handler
- ChatHistory store (in-memory/session)
- Context Strategy (ctx_strategy.py)
3. RAG Pipeline
- Query Rewriter (LLM call)
- Retriever (Chroma wrapper)
- Response Synthesizer (create-and-refine or async-tree)
4. Memory Builder (offline batch)
- Document Loader → Text Splitter → Embedder → Chroma VectorDB
C. Model & Client Layer
- Model Registry (maps model names to BaseModel instances)
- BaseModel wrapper (loads quantized GGUF models via llama-cpp-python)
- Prompt templates (prompt.py)
D. Data & Persistence Layer
- docs/ folder (Markdown source)
- Chroma Vector Database (persistent embedding store)
- Optional: config files (pyproject.toml, Makefile)

##### Relationships & data flows
- User → UI → ChatRouter
- ChatRouter → Conversation Handler → Model Registry → LLM Client → Response → UI
- ChatRouter → RAG Pipeline steps in sequence (and parallel for async)
- Retriever ↔ Chroma VectorDB
- Model Registry → BaseModel → llama_cpp_python (external library)

##### Architectural patterns & design principles
- Layered architecture: UI / Application / Model / Data
- Modular design: separate packages for client, model, conversation, memory, loader
- Strategy pattern for context strategies and synthesis strategies
- Registry pattern for pluggable model settings

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% User Interface Layer
    subgraph "User Interface Layer"
        direction TB
        CLI_Conv["CLI Chatbot"]:::ui
        CLI_RAG["CLI RAG Chatbot"]:::ui
        UI_Conv["Streamlit Chatbot UI"]:::ui
        UI_RAG["Streamlit RAG Chatbot UI"]:::ui
    end

    %% Application Layer
    subgraph "Application Layer"
        direction TB
        ChatRouter["ChatRouter"]:::app

        subgraph "Conversation Handler"
            direction TB
            ConvHandler["ConversationHandler"]:::app
            ChatHistory["ChatHistory"]:::app
            ContextStrategy["ContextStrategy"]:::app
        end

        subgraph "RAG Pipeline"
            direction TB
            QueryRewriter["QueryRewriter"]:::app
            Retriever["Retriever"]:::app
            DistanceMetric["DistanceMetric"]:::app
            ResponseSynthesizer["ResponseSynthesizer"]:::app
        end

        subgraph "Memory Builder (offline)"
            direction TB
            MemoryBuilder["MemoryBuilder"]:::app
            DocumentLoader["DocumentLoader"]:::app
            TextSplitter["TextSplitter"]:::app
            Embedder["Embedder"]:::app
        end
    end

    %% Model & Client Layer
    subgraph "Model & Client Layer"
        direction TB
        ModelRegistry["ModelRegistry"]:::model
        BaseModel["BaseModel"]:::model
        LLMClient["LLMClient"]:::model
        PromptTemplates["PromptTemplates"]:::model
        subgraph "Model Settings"
            direction TB
            ModelLlama["llama.py"]:::model
            ModelOpenchat["openchat.py"]:::model
            ModelPhi["phi.py"]:::model
            ModelStablelm["stablelm_zephyr.py"]:::model
            ModelStarling["starling.py"]:::model
        end
    end

    %% Data & Persistence Layer
    subgraph "Data & Persistence Layer"
        direction TB
        Docs["docs/"]:::data
        ChromaDB["Chroma VectorDB"]:::data
        VectorStore["vector_store/"]:::data
    end

    %% Connections
    CLI_Conv -->|uses| ChatRouter
    CLI_RAG -->|uses| ChatRouter
    UI_Conv -->|uses| ChatRouter
    UI_RAG -->|uses| ChatRouter

    ChatRouter -->|route to conversation| ConvHandler
    ChatRouter -->|route to RAG| QueryRewriter

    ConvHandler -->|uses| ChatHistory
    ConvHandler -->|uses| ContextStrategy
    ConvHandler -->|fetch model| ModelRegistry
    ModelRegistry -->|provides| BaseModel
    BaseModel -->|invokes| LLMClient
    LLMClient -->|response| ConvHandler
    ConvHandler -->|reply| ChatRouter

    QueryRewriter -->|rewritten query| Retriever
    Retriever -->|retrieve| ChromaDB
    ChromaDB -->|results| Retriever
    Retriever -->|pass chunks| ResponseSynthesizer
    ResponseSynthesizer -->|final answer| ChatRouter

    MemoryBuilder -->|load docs| DocumentLoader
    DocumentLoader -->|split| TextSplitter
    TextSplitter -->|embed| Embedder
    Embedder -->|persist| ChromaDB

    ChatRouter -->|send answer| CLI_Conv
    ChatRouter -->|send answer| CLI_RAG
    ChatRouter -->|send answer| UI_Conv
    ChatRouter -->|send answer| UI_RAG

    %% Click Events
    click UI_Conv "https://github.com/umbertogriffo/rag-chatbot/blob/main/chatbot/chatbot_app.py"
    click UI_RAG "https://github.com/umbertogriffo/rag-chatbot/blob/main/chatbot/rag_chatbot_app.py"
    click CLI_Conv "https://github.com/umbertogriffo/rag-chatbot/blob/main/chatbot/cli/chatbot.py"
    click CLI_RAG "https://github.com/umbertogriffo/rag-chatbot/blob/main/chatbot/cli/rag_chatbot.py"
    click ConvHandler "https://github.com/umbertogriffo/rag-chatbot/blob/main/chatbot/bot/conversation/conversation_handler.py"
    click ChatHistory "https://github.com/umbertogriffo/rag-chatbot/blob/main/chatbot/bot/conversation/chat_history.py"
    click ContextStrategy "https://github.com/umbertogriffo/rag-chatbot/blob/main/chatbot/bot/conversation/ctx_strategy.py"
    click Retriever "https://github.com/umbertogriffo/rag-chatbot/blob/main/chatbot/bot/memory/vector_database/chroma.py"
    click DistanceMetric "https://github.com/umbertogriffo/rag-chatbot/blob/main/chatbot/bot/memory/vector_database/distance_metric.py"
    click MemoryBuilder "https://github.com/umbertogriffo/rag-chatbot/blob/main/chatbot/memory_builder.py"
    click DocumentLoader "https://github.com/umbertogriffo/rag-chatbot/blob/main/chatbot/document_loader/loader.py"
    click TextSplitter "https://github.com/umbertogriffo/rag-chatbot/blob/main/chatbot/document_loader/text_splitter.py"
    click Embedder "https://github.com/umbertogriffo/rag-chatbot/blob/main/chatbot/bot/memory/embedder.py"
    click LLMClient "https://github.com/umbertogriffo/rag-chatbot/blob/main/chatbot/bot/client/llama_cpp_client.py"
    click PromptTemplates "https://github.com/umbertogriffo/rag-chatbot/blob/main/chatbot/bot/client/prompt.py"
    click BaseModel "https://github.com/umbertogriffo/rag-chatbot/blob/main/chatbot/bot/model/base_model.py"
    click ModelRegistry "https://github.com/umbertogriffo/rag-chatbot/blob/main/chatbot/bot/model/model_registry.py"
    click ModelLlama "https://github.com/umbertogriffo/rag-chatbot/blob/main/chatbot/bot/model/settings/llama.py"
    click ModelOpenchat "https://github.com/umbertogriffo/rag-chatbot/blob/main/chatbot/bot/model/settings/openchat.py"
    click ModelPhi "https://github.com/umbertogriffo/rag-chatbot/blob/main/chatbot/bot/model/settings/phi.py"
    click ModelStablelm "https://github.com/umbertogriffo/rag-chatbot/blob/main/chatbot/bot/model/settings/stablelm_zephyr.py"
    click ModelStarling "https://github.com/umbertogriffo/rag-chatbot/blob/main/chatbot/bot/model/settings/starling.py"

    %% Styles
    classDef ui fill:#b3cde3,stroke:#333,stroke-width:1px
    classDef app fill:#ccebc5,stroke:#333,stroke-width:1px
    classDef model fill:#fbb4ae,stroke:#333,stroke-width:1px
    classDef data fill:#cccccc,stroke:#333,stroke-width:1px
```
