# akshaykumarbedre/chatwithanywebsite architecture

> Architecture diagram of the GitHub repository akshaykumarbedre/chatwithanywebsite, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/akshaykumarbedre/chatwithanywebsite
- Repository: https://github.com/akshaykumarbedre/chatwithanywebsite

## Overview

##### Project Type and Purpose
- This is a full-stack application designed for URL extraction/classification, description and product management, and interactive chatbot functionality. It uses AI-powered services to automate operations, helping businesses streamline content and product management while enhancing customer support.

##### Main Components
a. Frontend:
- Built with Next.js (as seen from package.json, next.config.mjs, tailwind, and component structure). It includes various pages (chatbot, admin, order placement, description, products, text-processor, url-processor, and an iframe variant) and shared UI components (Header, Footer).
b. Backend:
- A Flask-based server (app.py) that powers the API endpoints.
- Services/submodules that include:
- llm_service (for handling language model interactions),
- main_service (for core business logic),
- vector_store (handling vector databases with FAISS indices for description and product information),
- web_scraper (to extract and classify URLs).
- Streamlit application support (streamlit_app.py) indicating an alternative or additional UI for data visualization or quick demos.
c. Data Components:
- Raw, processed, and database folders under backend/data.
- Vector store indices are persisted in multiple places (inside backend/vectors, and within notebooks) using FAISS and Pickle files.
d. Configuration/Environment:
- Multiple config files like .env, configuration YAML, and Python configuration in src/config.
- Dependency files (requirements.txt, requirements_streamlit.txt, package.json) indicating the essential libraries.
e. Testing and Experimentation:
- Notebooks for experimentation in backend/notebooks, including specific examples for creating vector indices.
- Tests directory (backend/tests) with sample tests to verify functionality.

##### Relationships and Data Flow
- Users interact with the system via the Next.js frontend; their actions trigger API calls.
- The frontend makes calls (using utilities like the API file in frontend/src/utils/api.js) to the backend Flask server.
- The Flask backend receives requests, and based on the endpoint’s purpose, delegates tasks to the appropriate service:
- Web scraping service extracts URLs.
- Main service orchestrates operations like classification and CRUD operations for descriptions/products.
- Vector store service processes and retrieves vector embeddings stored as FAISS indices.
- LLM service generates context-aware responses for the chatbot.
- There’s also support via a Streamlit-based UI for interactions or monitoring.
- Data storage occurs via the vector indices and raw/processed data directories, allowing efficient retrieval for features like “View All Descriptions” or “View All Products.”

##### Architectural Patterns/Design Principles
- Separation of concerns: Frontend and backend are strictly separated (with dedicated folders).
- Modular service architecture within the backend: Each core feature (web scraping, vector storage, language model interactions) is isolated into its own service module.
- Use of external libraries and vector databases (FAISS) for efficient high-dimensional searches.
- Configuration and Environment management ensuring deployment consistency.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Frontend Layer
    subgraph "Frontend Layer"
        subgraph "Pages"
            Chatbot["Chatbot Page"]:::frontend
            Admin["Admin Page"]:::frontend
            Description["Description Page"]:::frontend
            Products["Products Page"]:::frontend
            OrderPlace["Order Placement Page"]:::frontend
            TextProcessor["Text-Processor Page"]:::frontend
            URLProcessor["URL-Processor Page"]:::frontend
            ChatbotIframe["Chatbot (Iframe Variant) Page"]:::frontend
        end
        subgraph "UI Components"
            Header["Header"]:::ui
            Footer["Footer"]:::ui
        end
    end

    %% Backend Layer
    subgraph "Backend Layer"
        API["API Server (Flask App)"]:::backend

        subgraph "Services"
            LLM["LLM Service"]:::service
            MainS["Main Service"]:::service
            VectorStore["Vector Store Service"]:::service
            WebScraper["Web Scraper Service"]:::service
        end

        Streamlit["Streamlit App"]:::backend

        subgraph "Data Storage"
            Raw["Raw Data"]:::data
            Processed["Processed Data"]:::data
            VectorDesc["Vector Indices (Description)"]:::data
            VectorProd["Vector Indices (Product Info)"]:::data
        end

        subgraph "Configuration & Environment"
            Env["Environment Variables"]:::config
            YAML["YAML Configurations"]:::config
            PyConfig["Python Configurations"]:::config
        end

        subgraph "Testing & Experimentation"
            Tests["Test Suite"]:::config
            Notebooks["Notebooks"]:::config
        end
    end

    %% Data Flow: Frontend to API Server (multiple pages)
    Chatbot -->|"API_call"| API
    Admin -->|"API_call"| API
    Description -->|"API_call"| API
    Products -->|"API_call"| API
    OrderPlace -->|"API_call"| API
    TextProcessor -->|"API_call"| API
    URLProcessor -->|"API_call"| API
    ChatbotIframe -->|"API_call"| API

    %% API Server delegating requests to services
    API -->|"LLM_request"| LLM
    API -->|"Business_logic"| MainS
    API -->|"Vector_query"| VectorStore
    API -->|"Scraping_request"| WebScraper

    %% LLM Service response back to API Server for chat responses
    LLM -->|"Chat_response"| API

    %% Main Service interactions with data storage
    MainS -->|"Store_processed"| Processed
    MainS -->|"Access_raw"| Raw

    %% Vector Store Service interactions with vector indices
    VectorStore -->|"Manage_indices"| VectorDesc
    VectorStore -->|"Manage_indices"| VectorProd

    %% Notebooks accessing vector indices for experimentation
    Notebooks -->|"Access_VectorData"| VectorDesc
    Notebooks -->|"Access_VectorData"| VectorProd

    %% Streamlit App interactions for monitoring or demos
    Streamlit -->|"Data_monitoring"| API

    %% API Server configuration dependencies
    Env -->|"configs"| API
    YAML -->|"configs"| API
    PyConfig -->|"configs"| API

    %% Testing nodes validating the services and API
    Tests -->|"Validate_services"| API
    Tests -->|"Validate_services"| MainS
    Tests -->|"Validate_services"| VectorStore
    Tests -->|"Validate_services"| WebScraper

    %% Click Events for Frontend
    click Chatbot "https://github.com/akshaykumarbedre/chatwithanywebsite/blob/resto/frontend/src/app/chatbot/page.js"
    click Admin "https://github.com/akshaykumarbedre/chatwithanywebsite/blob/resto/frontend/src/app/admin/page.js"
    click Description "https://github.com/akshaykumarbedre/chatwithanywebsite/blob/resto/frontend/src/app/desciption/page.js"
    click Products "https://github.com/akshaykumarbedre/chatwithanywebsite/blob/resto/frontend/src/app/products/page.js"
    click OrderPlace "https://github.com/akshaykumarbedre/chatwithanywebsite/blob/resto/frontend/src/app/order_place/page.js"
    click TextProcessor "https://github.com/akshaykumarbedre/chatwithanywebsite/blob/resto/frontend/src/app/text-processor/page.js"
    click URLProcessor "https://github.com/akshaykumarbedre/chatwithanywebsite/blob/resto/frontend/src/app/url-processor/page.js"
    click ChatbotIframe "https://github.com/akshaykumarbedre/chatwithanywebsite/blob/resto/frontend/src/app/chatbot_iframe/page.js"
    click Header "https://github.com/akshaykumarbedre/chatwithanywebsite/blob/resto/frontend/src/app/components/Header.js"
    click Footer "https://github.com/akshaykumarbedre/chatwithanywebsite/blob/resto/frontend/src/app/components/Footer.js"

    %% Click Events for Backend
    click API "https://github.com/akshaykumarbedre/chatwithanywebsite/blob/resto/backend/app.py"
    click LLM "https://github.com/akshaykumarbedre/chatwithanywebsite/blob/resto/backend/src/services/llm_service.py"
    click MainS "https://github.com/akshaykumarbedre/chatwithanywebsite/blob/resto/backend/src/services/main_service.py"
    click VectorStore "https://github.com/akshaykumarbedre/chatwithanywebsite/blob/resto/backend/src/services/vector_store.py"
    click WebScraper "https://github.com/akshaykumarbedre/chatwithanywebsite/blob/resto/backend/src/services/web_scraper.py"
    click Streamlit "https://github.com/akshaykumarbedre/chatwithanywebsite/blob/resto/backend/streamlit_app.py"

    %% Click Events for Data Storage
    click Raw "https://github.com/akshaykumarbedre/chatwithanywebsite/tree/resto/backend/data/raw"
    click Processed "https://github.com/akshaykumarbedre/chatwithanywebsite/tree/resto/backend/data/processed"
    click VectorDesc "https://github.com/akshaykumarbedre/chatwithanywebsite/tree/resto/backend/vectors/description_index"
    click VectorProd "https://github.com/akshaykumarbedre/chatwithanywebsite/tree/resto/backend/vectors/product_info_index"

    %% Click Events for Configuration & Environment
    click Env "https://github.com/akshaykumarbedre/chatwithanywebsite/blob/resto/backend/.env"
    click YAML "https://github.com/akshaykumarbedre/chatwithanywebsite/blob/resto/backend/config/config.yaml"
    click PyConfig "https://github.com/akshaykumarbedre/chatwithanywebsite/blob/resto/backend/src/config/config.py"

    %% Click Events for Testing & Experimentation
    click Tests "https://github.com/akshaykumarbedre/chatwithanywebsite/tree/resto/backend/tests"
    click Notebooks "https://github.com/akshaykumarbedre/chatwithanywebsite/tree/resto/backend/notebooks"

    %% Styling Classes
    classDef frontend fill:#bbdefb,stroke:#0d47a1,stroke-width:2px;
    classDef ui fill:#ffccbc,stroke:#d84315,stroke-width:2px;
    classDef backend fill:#d1c4e9,stroke:#5e35b1,stroke-width:2px;
    classDef service fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px;
    classDef data fill:#ffe0b2,stroke:#e65100,stroke-width:2px;
    classDef config fill:#f0f4c3,stroke:#8bc34a,stroke-width:2px;
```
