# maneesha1618/e_commercesupport_chatbot architecture

> Architecture diagram of the GitHub repository maneesha1618/e_commercesupport_chatbot, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-03-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/maneesha1618/e_commercesupport_chatbot
- Repository: https://github.com/maneesha1618/e_commercesupport_chatbot

## Overview

Summary / project type and purpose
- This is a small RAG (Retrieval-Augmented Generation) application: a Streamlit-based frontend + Python backend that serves an E‑commerce FAQ chatbot. Its purpose is to answer user questions by retrieving relevant FAQ passages and generating LLM responses with LangChain + Hugging Face models.

How I reached that conclusion (quick pointers to the repo)
- Frontend: app/streamlit_app.py — Streamlit UI and user interaction.
- Document ingestion: loaders/document_loader.py — loads/splits ecommerce_faq.md (data/).
- Embeddings: embeddings/embedder.py — computes vector embeddings.
- Retriever / vector DB: retriever/vector_retriever.py and vector_store/faiss_index/* — FAISS-based retrieval.
- LLM / RAG chain: llm/rag_chain.py — constructs LangChain QA chain with Hugging Face LLMs.
- Config/env: config.py and optional .env (README mentions Hugging Face token).
- Persistent index files: vector_store/faiss_index/index.faiss and index.pkl.

##### Frontend (User Interface)
- Files: app/streamlit_app.py
- Responsibilities: accept queries, display results, handle user controls (choose model, start ingestion).

##### Ingestion / Offline Pipeline (build-time)
- Document Loader
- Files: loaders/document_loader.py
- Input: data/ecommerce_faq.md, data/ecomm_faq.json
- Output: document chunks
- Embedder
- Files: embeddings/embedder.py
- Responsibility: turn chunks into vectors (embedding model selection)
- Vector Store Persister
- Files: retriever/vector_retriever.py (indexing parts), vector_store/faiss_index/* (index files)
- Output: index.faiss, index.pkl

##### Runtime Retrieval + Generation (request-time)
- Query Embedding (often part of retriever or embedder)
- Vector Retriever
- Files: retriever/vector_retriever.py
- Responsibility: nearest-neighbor search to fetch top-k chunks
- RAG / QA Chain
- Files: llm/rag_chain.py
- Responsibility: assemble prompt + retrieved docs, call model pipeline, produce answer
- LLM Provider (external)
- External dependency: Hugging Face Hub / Transformers, possibly gated models requiring token
- Config: HUGGINGFACEHUB_API_TOKEN (.env) and config.py

##### Storage & Config
- Persistent Data: data/ (original docs), vector_store/faiss_index/* (index files)
- Config and secrets: config.py, .env (token)
- Requirements/deployment: requirements.txt, README.md

1. Offline / Build pipeline (left/top):
- data/* -> Document Loader -> Embedder -> FAISS Index Builder -> vector_store/faiss_index (index files)
- This as "one-time or on-update" pipeline.
2. Runtime / Query pipeline (center/right):
- User (Streamlit UI) -> Streamlit App -> Query Encoder (embedder) -> FAISS Retriever -> RAG Chain -> Hugging Face LLM -> RAG Chain -> Streamlit App -> User
- That the retriever reads the persisted index files.
- If caching is used, show a cache layer between retriever and FAISS.

Important architectural patterns and principles used
- RAG pattern (separate retrieval and generation steps).
- Separation of concerns: UI (Streamlit) vs retrieval service vs LLM chain vs storage.
- Persistence of vector index enables fast retrieval and decouples ingestion vs query-time.
- Configurable model layer: design allows swapping models (google/flan-t5-small vs gated LLaMA) — treat model provider as an interchangeable external service abstraction.

- External services: Hugging Face Hub (models), optionally model-hosting infra (Hugging Face Spaces / cloud)

- Use clear labels for each component (use the names above).
- Frontend/UI: green rectangle
- Runtime services: orange rectangles (real-time)
- Distinguish synchronous call flows (user -> app -> retriever -> LLM) vs asynchronous/batch flows (data ingestion -> index build).

- That retriever reads the persisted FAISS index (vector_store).
- Query embedding step explicitly (sometimes embedder is reused for both ingestion and queries).
- Where model selection occurs (UI or config) and that LLM calls may be gated (token .env).
- Possibility of storing chat history / metrics (not present now — show as future extension).

Guidelines tailored to this project type (full-stack / RAG app)
- Separation of offline ingestion vs request-time retrieval/generation.
- Make it explicit that the system is currently a monolith but has clearly separable components that can be converted to microservices:
- Extract retriever/embedder as a service with an API in the future.
- Treat the LLM provider as another service (remote hosting or local GPU).
- Persistency and reusability: indexing only needed when knowledge base changes.
- Points where you would add monitoring, auth, caching, and async queuing for scale.

- Index files location: vector_store/faiss_index/index.faiss, index.pkl.
- Mention token management: HUGGINGFACEHUB_API_TOKEN in .env or huggingface-cli login; model access gating.
- Expected latency hotspots: embedding call, FAISS kNN, LLM generation.

- Provide two views:

- Both offline ingestion and runtime flow as separate, clearly-labeled sections.
- Where secrets/config are used, and where persistence occurs.
- Scalability and extensibility notes (e.g., which components are safe to convert to microservices).

- UI (Streamlit) — app/streamlit_app.py
- Loader / Chunker — loaders/document_loader.py
- Embeddings (train & query) — embeddings/embedder.py
- Retriever (FAISS) — retriever/vector_retriever.py; vector_store/faiss_index/*
- RAG / LangChain chain — llm/rag_chain.py
- Raw KB — data/ecommerce_faq.md
- External LLM provider — Hugging Face Hub (requires token)

Concluding note

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Top-down orientation (vertical)
    %% OFFLINE / BUILD PIPELINE (one-time or on-update)
    subgraph "Offline Ingestion / Build Pipeline" 
        direction TB
        DataFiles["Raw KB\ndata/ecommerce_faq.md\n(data/...)"]:::storage
        AltData["Alternate Raw KB\ndata/ecomm_faq.json"]:::storage
        Loader["Document Loader / Chunker\nloaders/document_loader.py"]:::ingestion
        EmbedderBuild["Embeddings Generator (ingest)\nembeddings/embedder.py"]:::ingestion
        IndexBuilder["FAISS Index Builder / Persister\nretriever/vector_retriever.py (indexing)"]:::ingestion
        FaissFiles["Persisted Vector Index\nvector_store/faiss_index/index.faiss\nindex.pkl"]:::storage
    end

    %% OFFLINE FLOW (dashed indicates batch/control)
    DataFiles -.->|"source docs (md/json)"| Loader
    AltData -.->|"source docs (json)"| Loader
    Loader -.->|"document chunks (split)"| EmbedderBuild
    EmbedderBuild -.->|"chunk embeddings (vectors)"| IndexBuilder
    IndexBuilder -.->|"persist index files"| FaissFiles

    %% RUNTIME / REQUEST-TIME PIPELINE (real-time)
    subgraph "Runtime / Query Pipeline" 
        direction TB
        User["User (end-user)"]:::user
        StreamlitApp["Streamlit App\napp/streamlit_app.py"]:::frontend
        QueryEncoder["Query Encoder (embeddings/embedder.py)\nembeddings/embedder.py"]:::runtime
        Cache["Optional Cache (top-k cache)"]:::cache
        Retriever["FAISS Retriever\nretriever/vector_retriever.py"]:::runtime
        RAGChain["LangChain RAG Chain\nllm/rag_chain.py"]:::runtime
        HFModel["Hugging Face Models\n(External)"]:::external
        Response["LLM Response / Answer"]:::runtime
    end

    %% RUNTIME FLOW (solid for data)
    User -->|"query text"| StreamlitApp
    StreamlitApp -->|"query text / model selection"| QueryEncoder
    QueryEncoder -->|"query embedding (vector)"| Retriever
    Retriever -->|"top-k docs (chunks + scores)"| RAGChain
    RAGChain -->|"prompt + context"| HFModel
    HFModel -->|"generated text (answer)"| RAGChain
    RAGChain -->|"final answer"| StreamlitApp
    StreamlitApp -->|"display answer"| User

    %% Retriever reads persisted index (explicit)
    FaissFiles -->|"reads persisted index files"| Retriever
    Cache -.->|"optional: cached top-k"| Retriever
    Cache -->|"may read/write"| FaissFiles

    %% CONFIG / SECRETS (dashed control lines)
    Config["Config / Secrets\nconfig.py + .env\n(HUGGINGFACEHUB_API_TOKEN)"]:::config
    Config -.->|"model selection / token"| StreamlitApp
    Config -.->|"token / settings"| RAGChain
    Config -.->|"token (auth)"| HFModel

    %% EXTERNAL DEPENDENCIES
    HFModel -.->|"Hugging Face Hub / Transformers"| HuggingFaceHub["Hugging Face Hub\n(External service)"]:::cloud

    %% FUTURE / EXTENSIONS (annotations)
    Extensions["Future: Monitoring / Auth / Async Queue / Separate Services"]:::note
    Extensions -.-> StreamlitApp
    Extensions -.-> Retriever
    Extensions -.-> RAGChain

    %% LEGEND / Artifacts (stacked vertically)
    subgraph "Repository Mappings & Artifacts" 
        direction TB
        Map_UI["UI — app/streamlit_app.py"]:::map
        Map_Loader["Loader — loaders/document_loader.py"]:::map
        Map_Embedder["Embeddings — embeddings/embedder.py"]:::map
        Map_Retriever["Retriever — retriever/vector_retriever.py"]:::map
        Map_RAG["RAG Chain — llm/rag_chain.py"]:::map
        Map_Data["Raw KB — data/ecommerce_faq.md\nAlternate: data/ecomm_faq.json"]:::map
        Map_IndexFiles["Vector Index — vector_store/faiss_index/\nindex.faiss, index.pkl"]:::map
        Map_Config["Config — config.py + .env"]:::map
        Map_Readme["Docs/Deps — README.md, requirements.txt"]:::map
    end

    %% Connect mapping to components with dashed lines (control/annotation)
    Map_UI -.-> StreamlitApp
    Map_Loader -.-> Loader
    Map_Embedder -.-> EmbedderBuild
    Map_Retriever -.-> Retriever
    Map_RAG -.-> RAGChain
    Map_Data -.-> DataFiles
    Map_IndexFiles -.-> FaissFiles
    Map_Config -.-> Config
    Map_Readme -.-> Extensions

    %% Click Events for mapped components (paths from component_mapping)
    click StreamlitApp "https://github.com/maneesha1618/e_commercesupport_chatbot/blob/main/app/streamlit_app.py"
    click Loader "https://github.com/maneesha1618/e_commercesupport_chatbot/blob/main/loaders/document_loader.py"
    click EmbedderBuild "https://github.com/maneesha1618/e_commercesupport_chatbot/blob/main/embeddings/embedder.py"
    click QueryEncoder "https://github.com/maneesha1618/e_commercesupport_chatbot/blob/main/embeddings/embedder.py"
    click Retriever "https://github.com/maneesha1618/e_commercesupport_chatbot/blob/main/retriever/vector_retriever.py"
    click RAGChain "https://github.com/maneesha1618/e_commercesupport_chatbot/blob/main/llm/rag_chain.py"
    click DataFiles "https://github.com/maneesha1618/e_commercesupport_chatbot/blob/main/data/ecommerce_faq.md"
    click AltData "https://github.com/maneesha1618/e_commercesupport_chatbot/blob/main/data/ecomm_faq.json"
    click FaissFiles "https://github.com/maneesha1618/e_commercesupport_chatbot/blob/main/vector_store/faiss_index/index.faiss"
    click Map_IndexFiles "https://github.com/maneesha1618/e_commercesupport_chatbot/blob/main/vector_store/faiss_index/index.pkl"
    click Config "https://github.com/maneesha1618/e_commercesupport_chatbot/blob/main/config.py"
    click Map_Readme "https://github.com/maneesha1618/e_commercesupport_chatbot/blob/main/README.md"
    click Map_Readme "https://github.com/maneesha1618/e_commercesupport_chatbot/blob/main/requirements.txt"

    %% Styles / Colors
    classDef frontend fill:#e6ffea,stroke:#2e8b57,stroke-width:1px,color:#064e3b;
    classDef ingestion fill:#e8f1ff,stroke:#1e40af,stroke-width:1px,color:#1e3a8a;
    classDef runtime fill:#fff4e6,stroke:#c2410c,stroke-width:1px,color:#7c2d12;
    classDef storage fill:#f0f9ff,stroke:#0369a1,stroke-width:1px,color:#064e3b;
    classDef external fill:#f3f4f6,stroke:#6b7280,stroke-width:1px,color:#374151;
    classDef cloud fill:#f3f4f6,stroke:#374151,stroke-width:1px,color:#374151,stroke-dasharray: 5 5;
    classDef config fill:#fff7ed,stroke:#b45309,stroke-width:1px,color:#92400e;
    classDef user fill:#ecfdf5,stroke:#059669,stroke-width:1px,color:#065f46;
    classDef note fill:#fff1f2,stroke:#be123c,stroke-width:1px,color:#831843;
    classDef cache fill:#f8fafc,stroke:#0f172a,stroke-width:1px,color:#0f172a;
    classDef map fill:#ffffff,stroke:#111827,stroke-width:1px,color:#111827;

    class StreamlitApp frontend
    class Loader,EmbedderBuild,IndexBuilder ingestion
    class QueryEncoder,Retriever,RAGChain,Response runtime
    class DataFiles,AltData,FaissFiles storage
    class HFModel,HuggingFaceHub external,cloud
    class Config config
    class User user
    class Extensions note
    class Cache cache
    class Map_UI,Map_Loader,Map_Embedder,Map_Retriever,Map_RAG,Map_Data,Map_IndexFiles,Map_Config,Map_Readme map

    %% Notes on arrow styles
    linkStyle default stroke:#444,stroke-width:1px
    linkStyle 0 stroke-dasharray: 5 5
    linkStyle 1 stroke-dasharray: 5 5
    linkStyle 2 stroke-dasharray: 5 5
    linkStyle 3 stroke-dasharray: 5 5
    linkStyle 4 stroke-dasharray: 5 5
    linkStyle 5 stroke-dasharray: 5 5
    linkStyle 6 stroke-width:2px
    linkStyle 7 stroke-width:2px
    linkStyle 8 stroke-width:2px
    linkStyle 9 stroke-width:2px
    linkStyle 10 stroke-dasharray: 2 2
    linkStyle 11 stroke-dasharray: 2 2
    linkStyle 12 stroke-dasharray: 2 2
    linkStyle 13 stroke-dasharray: 2 2
```
