# aliaftab0786/langchain-chatbot architecture

> Architecture diagram of the GitHub repository aliaftab0786/langchain-chatbot, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/aliaftab0786/langchain-chatbot
- Repository: https://github.com/aliaftab0786/langchain-chatbot

## Overview

##### Main Components and Their Roles
- Frontend: The Streamlit-powered UI found in the frontend directory (app.py) that serves as the user interface where queries are entered and responses are displayed.
- Chatbot Backend: The core logic of the RAG chatbot contained in chatbot/rag_chatbot.py. This component interacts with LangChain, handles API interactions with Anthropic (configured in chatbot/.env) and orchestrates retrieval augmented generation.
- Web Scraper: Located in the scraper directory, the Scrapy spiders (ecommerce_spider.py, quotes_spider.py) and related modules gather product data (information such as title, price, rating) from various e-commerce and content sources.
- Data Storage & Retrieval: The data folder includes persistent storage of ChromaDB data (binary and metadata files) and a SQLite database (chroma.sqlite3). Also, products.json serves as a structured file for product data. These are used to support the retrieval functionality in the chatbot’s process.
- External Services: Integration with Anthropic’s API for language generation and sites/sources that are scraped.

##### Relationships and Data Flow
- The Frontend sends user queries (such as "What’s the price of Storage Carry Case?") to the Chatbot Backend.
- The Chatbot Backend processes the query using a Retrieval-Augmented Generation approach. It accesses the persistent ChromaDB/SQLite storage to retrieve relevant product or scraped data.
- The backend might in real-time or periodically rely on data fed by the Scraper, which collects updates from various e-commerce sites. Data is then stored in the data layer.
- External API interactions occur when the backend needs to leverage Anthropic’s Claude (as configured via the .env file).
- Data flows are bidirectional between the Frontend and Backend as well as between the Backend and Data Storage. The Scraper independently feeds data into the storage, which the Chatbot Backend accesses.

##### Architectural Patterns and Design Principles
- Modular Design: Each component (frontend, chatbot logic, scraper, data storage) has its own directory, reflecting separation of concerns.
- Service-Oriented Data Flow: The front-end interacts with the backend via API calls (or script execution in the case of Streamlit), and the backend acts as an orchestrator between data retrieval (from storage) and external language generation.
- Data Persistence: Use of persistent storage (ChromaDB and SQLite) ensures data consistency and enables retrieval augmentation.
- Integration Pattern: Scraped data is used as a knowledge source for answering queries, integrating web scraping and chatbot processing in a retrieval-augmented manner.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% UI Layer
    subgraph "UI Layer"
        FE["Frontend (Streamlit UI)"]
    end

    %% Application Layer
    subgraph "Application Layer"
        subgraph "Chatbot Service (LangChain+Anthropic)"
            CB["Backend Logic"]
            CC["Config (.env)"]
        end
        subgraph "Web Scraping Service"
            ES["E-commerce Spider"]
            QS["Quotes Spider"]
        end
    end

    %% Data Layer
    subgraph "Data Layer"
        DB1["ChromaDB"]
        DB2["SQLite Database"]
        DB3["Products Data (JSON)"]
    end

    %% External Services
    subgraph "External Services"
        EX["External API (Anthropic)"]
    end

    %% Connections
    FE -->|"sendsQuery"| CB
    CB -->|"retrievesData"| DB1
    CB -->|"retrievesData"| DB2
    CB -->|"retrievesData"| DB3
    DB1 -->|"returnsData"| CB
    DB2 -->|"returnsData"| CB
    DB3 -->|"returnsData"| CB
    CB -->|"callsAPI"| EX
    CB -->|"responds"| FE
    ES -->|"feedsData"| DB1
    ES -->|"feedsData"| DB2
    ES -->|"feedsData"| DB3
    QS -->|"feedsData"| DB1
    QS -->|"feedsData"| DB2
    QS -->|"feedsData"| DB3

    %% Click Events
    click FE "https://github.com/aliaftab0786/langchain-chatbot/blob/main/frontend/app.py"
    click CB "https://github.com/aliaftab0786/langchain-chatbot/blob/main/chatbot/rag_chatbot.py"
    click CC "https://github.com/aliaftab0786/langchain-chatbot/blob/main/chatbot/.env"
    click ES "https://github.com/aliaftab0786/langchain-chatbot/blob/main/scraper/scraper/spiders/ecommerce_spider.py"
    click QS "https://github.com/aliaftab0786/langchain-chatbot/blob/main/scraper/scraper/spiders/quotes_spider.py"
    click DB1 "https://github.com/aliaftab0786/langchain-chatbot/tree/main/data/chroma_db"
    click DB2 "https://github.com/aliaftab0786/langchain-chatbot/blob/main/data/chroma.sqlite3"
    click DB3 "https://github.com/aliaftab0786/langchain-chatbot/blob/main/data/products.json"

    %% Styles
    classDef ui fill:#add8e6,stroke:#333,stroke-width:2px;
    classDef backend fill:#90ee90,stroke:#333,stroke-width:2px;
    classDef data fill:#ffcccb,stroke:#333,stroke-width:2px;
    classDef external fill:#fdd,stroke:#933,stroke-width:2px;
    
    class FE ui;
    class CB,CC,ES,QS backend;
    class DB1,DB2,DB3 data;
    class EX external;
```
