# skarthi369/raglangchain architecture

> Architecture diagram of the GitHub repository skarthi369/raglangchain, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/skarthi369/raglangchain
- Repository: https://github.com/skarthi369/raglangchain

## Overview

##### Project Type and Purpose
- This is a full-stack RAG (Retrieval-Augmented Generation) chatbot.
- Purpose: Let users upload documents (PDF, DOCX, TXT, CSV, etc.), automatically chunk and embed them, store embeddings in a vector database (Chroma), and answer user queries via a conversational UI powered by Grok’s Llama model and LangChain.

##### File Structure
- Top-level files:
- app.py + streamlit_app.py → Streamlit UI entry points
- rag_chatbot.py → Core RAG pipeline (document ingestion, chunking, embedding, retrieval, LLM query)
- .env → Grok API key
- requirements.txt / packages.txt → Python & system dependencies
- data/ → Local knowledge-base folder for uploaded documents
- No separate “backend” folder; UI and pipeline live in the same codebase.
- Deployment scripts: streamlit_app.py for Streamlit Cloud.

##### Key Architecture Details from the README
- Document processing stages (load → split → embed → store)
- Vector storage using Chroma
- Embedding generation via HuggingFace models
- Querying Grok Llama via API
- Conversation memory to maintain context across turns
- Streamlit for UI, deployed locally or on Streamlit Cloud

##### Main Components
a. Client Layer
- Web browser / user interacts with Streamlit front end (app.py / streamlit_app.py)
b. Application Layer
- Streamlit App: handles file uploads, user chat interface, renders conversation
- RAG Pipeline (rag_chatbot.py):
- Document Loader
- Text Splitter / Chunker
- Embedder (HuggingFace)
- Vector Store (Chroma)
- Retriever (semantic search)
- LLM Interface (Grok API)
- Conversation Memory Manager
c. Data Layer
- Local File System (data/ directory) for raw docs
- Vector Database (Chroma) persisted locally/in memory
d. External Services
- Grok Llama API
- HuggingFace model repo (for embeddings)
- (Optional) Streamlit Cloud for hosting

##### Relationships and Data Flow
1. User → Streamlit Frontend
2. Frontend → RAG Pipeline
a. On file upload → Document Loader → Splitter → Embedder → Vector Store
b. On chat message → Retriever queries Vector Store → Context returned
c. Pipeline sends context + user prompt → Grok API → Model response
3. Response → Conversation Memory → Back to Frontend → User
4. External calls: Grok API & HuggingFace model downloads

##### Architectural Patterns and Principles
- Modular design: separate concerns (UI vs. ingestion vs. search vs. LLM)
- Pipeline pattern for document processing
- Layered architecture: Presentation → Application → Data
- Stateless UI with stateful pipeline (conversation memory)
- Config-driven: environment variables (.env) for secrets

##### Essential Elements to Capture
- All code modules and their responsibilities
- Data flow from ingestion through retrieval to response
- Interaction with external models and services
- Persistence layers vs. in-memory operations
- Configuration and deployment touches (.env, Streamlit Cloud settings)

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Client Layer"
        UI["Streamlit UI<br>(app.py, streamlit_app.py)"]:::frontend
    end

    subgraph "Application Layer"
        subgraph "RAG Pipeline"
            Loader["Document Loader"]:::processing
            Chunker["Text Splitter / Chunker"]:::processing
            Embedder["Embedder<br>(HuggingFace)"]:::processing
            VectorStore["Vector Store<br>(Chroma)"]:::datastore
            Retriever["Retriever"]:::processing
            LLMInterface["LLM Interface<br>(Grok API)"]:::processing
            Memory["Conversation Memory Manager"]:::processing
        end
    end

    subgraph "Data Layer"
        RawDocs["Local FS<br>(data/)"]:::datastore
    end

    subgraph "External Services"
        Grok["Grok Llama API"]:::external
        HF["HuggingFace Models"]:::external
        StreamlitCloud["Streamlit Cloud"]:::external
    end

    subgraph "Config"
        Env[".env"]:::config
        Req["requirements.txt"]:::config
        Pkg["packages.txt"]:::config
    end

    UI -->|"upload documents"| Loader
    Loader --> Chunker
    Chunker --> Embedder
    Embedder -->|"fetch embeddings"| HF
    Embedder --> VectorStore
    UI -->|"user query"| Retriever
    Retriever --> VectorStore
    Retriever --> Memory
    Memory --> Retriever
    Retriever -->|"context + prompt"| LLMInterface
    LLMInterface -->|"HTTP API"| Grok
    LLMInterface --> Memory
    Memory --> UI
    RawDocs -->|"read files"| Loader
    UI -->|"deployed on"| StreamlitCloud
    Env -.-> UI
    Env -.-> LLMInterface
    Req -.-> UI
    Pkg -.-> UI

    click UI "https://github.com/skarthi369/raglangchain/blob/main/app.py"
    click UI "https://github.com/skarthi369/raglangchain/blob/main/streamlit_app.py"
    click Loader "https://github.com/skarthi369/raglangchain/blob/main/rag_chatbot.py"
    click Chunker "https://github.com/skarthi369/raglangchain/blob/main/rag_chatbot.py"
    click Embedder "https://github.com/skarthi369/raglangchain/blob/main/rag_chatbot.py"
    click VectorStore "https://github.com/skarthi369/raglangchain/blob/main/rag_chatbot.py"
    click Retriever "https://github.com/skarthi369/raglangchain/blob/main/rag_chatbot.py"
    click LLMInterface "https://github.com/skarthi369/raglangchain/blob/main/rag_chatbot.py"
    click Memory "https://github.com/skarthi369/raglangchain/blob/main/rag_chatbot.py"
    click RawDocs "https://github.com/skarthi369/raglangchain/tree/main/data/"
    click Env "https://github.com/skarthi369/raglangchain/blob/main/.env"
    click Req "https://github.com/skarthi369/raglangchain/blob/main/requirements.txt"
    click Pkg "https://github.com/skarthi369/raglangchain/blob/main/packages.txt"

    classDef frontend fill:#D0E8FF,stroke:#005B96,stroke-width:1px;
    classDef processing fill:#E0FFD8,stroke:#228B22,stroke-width:1px;
    classDef datastore fill:#FFF4AE,stroke:#DAA520,stroke-width:1px;
    classDef external fill:#E8D0FF,stroke:#800080,stroke-width:1px;
    classDef config fill:#F0F0F0,stroke:#A0A0A0,stroke-width:1px;
```
