# bhushanmaheshwari/langchain-chatwithfiles architecture

> Architecture diagram of the GitHub repository bhushanmaheshwari/langchain-chatwithfiles, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bhushanmaheshwari/langchain-chatwithfiles
- Repository: https://github.com/bhushanmaheshwari/langchain-chatwithfiles

## Overview

##### Project type and purpose
- This is a full-stack conversational-AI web application built in Python (likely using Streamlit given the `pages/` folder and `.py` naming).
- It lets users upload PDF documents, ingests and vectorizes their content, and then provides an interactive chat interface to query those documents. There is also a logs page for transaction history.

##### File structure
- Top-level directories:
- pages/ : the Streamlit front-end entrypoints (`1_📁_Chat_with_PDF.py`, `2_📝_Logs.py`)
- features/ : business-logic modules (`upload.py` handles PDF uploads, `conversation.py` manages the QA loop)
- processing/ : data-pipeline modules (`ingest.py`, `chunk.py`, `vectordb.py`, `storage.py`, `start.py`)
- utils/ : supporting utilities (`login.py`, `session.py`, `logs.py`)
- data/ : persistent storage (raw PDFs organized by project ID and possibly logs)
- htmlTemplates.py : custom HTML snippets for UI
- 👋_Hello.py : greeting/landing-page logic
- Supporting files:
- requirements.txt (Python dependencies, likely includes LLM SDK, vector-DB client, Streamlit)
- .devcontainer/ (development container config)

##### README for additional insights
- Screenshots show an introduction page, basic conversational AI interface, extraction prompts and a logs view.

a. Main components
1. User Interface (Streamlit front-end):
- Pages: Chat with PDF, Logs
- HTML templates for layout
2. Features Layer:
- Upload handler (PDF upload & validation)
- Conversation manager (query vector DB, call LLM API, format responses)
3. Processing Pipeline:
- Ingestion orchestrator (start.py)
- Chunking service (chunk.py)
- Vectorization & storage (vectordb.py + storage.py)
4. Data Stores:
- File system (raw PDFs under data/)
- Vector database (Chroma/FAISS or cloud-hosted)
- Transaction logs (could be flat files or a DB, surfaced via utils/logs.py)
5. External Services:
- LLM API (OpenAI, Anthropic, etc.)
- Authentication provider (if any, via utils/login.py)

b. Relationships and interactions
- User → Streamlit UI → Features.upload → data storage
- User question → Features.conversation → Processing.vectordb (query embeddings) → LLM API
- Ingestion job starts (manual or on upload) wired by processing/start → chunk → vectordb → storage
- All user actions logged by utils/logs → logs store → surfaced by Logs page

c. Architectural patterns/design principles
- Separation of concerns: UI (pages), business logic (features), data pipeline (processing), utilities (utils)
- Modular pipeline for ingestion (chunking, vectorization, storage)
- Stateless UI with session management (utils/session.py)

d. Key technologies/frameworks
- Python 3.x, Streamlit (front end), possibly FastAPI or direct Streamlit callbacks
- Vector DB client (Chroma/FAISS)
- LLM SDK (OpenAI or similar)
- PDF parser (PyPDF2, pdfminer)
- Containerization (via .devcontainer)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Actors
    User["User"]:::external

    %% Frontend Tier
    subgraph "Frontend Tier"
        ChatUI["Streamlit Chat UI"]:::frontend
        LogsUI["Streamlit Logs UI"]:::frontend
        Landing["Landing/Greeting Page"]:::frontend
        HTML["HTML Templates"]:::frontend
    end

    %% Backend Tier
    subgraph "Backend Tier"
        subgraph "Features Layer"
            Upload["PDF Upload Handler"]:::backend
            Conversation["Conversation Manager"]:::backend
        end
        subgraph "Processing Pipeline"
            Start["Ingestion Orchestrator"]:::backend
            Ingest["PDF Ingestion Module"]:::backend
            Chunk["PDF Chunking Service"]:::backend
            VectorDBModule["Vectorization Module"]:::backend
            Storage["Storage Module"]:::backend
        end
        subgraph "Utils Layer"
            LogsUtil["Logging Utilities"]:::backend
            Session["Session Management"]:::backend
            AuthUtil["Authentication Utility"]:::backend
        end
    end

    %% Data Stores
    subgraph "Data Stores"
        RawFS["Raw PDF Files (File System)"]:::datastore
        VectorDB["Vector Database"]:::datastore
        LogStore["Log Store"]:::datastore
    end

    %% External Services
    subgraph "External Services"
        LLM["LLM API"]:::external
        AuthProv["Auth Provider"]:::external
    end

    %% Flows
    User --> ChatUI
    User --> LogsUI
    User --> Landing

    Landing --> ChatUI
    ChatUI --> Upload
    ChatUI --> Conversation
    ChatUI --> HTML
    ChatUI --> Session
    ChatUI --> LogsUtil

    Upload --> RawFS
    Upload --> Start

    Start --> Ingest
    Ingest --> Chunk
    Chunk --> VectorDBModule
    VectorDBModule --> Storage
    Storage --> VectorDB

    Conversation --> VectorDB
    VectorDB --> Conversation
    Conversation --> LLM
    LLM --> Conversation

    LogsUI --> LogsUtil
    LogsUtil --> LogStore

    AuthUtil --> AuthProv
    ChatUI --> AuthUtil

    %% Click Events
    click ChatUI "https://github.com/bhushanmaheshwari/langchain-chatwithfiles/blob/main/pages/1_📁_Chat_with_PDF.py"
    click LogsUI "https://github.com/bhushanmaheshwari/langchain-chatwithfiles/blob/main/pages/2_📝_Logs.py"
    click Landing "https://github.com/bhushanmaheshwari/langchain-chatwithfiles/blob/main/👋_Hello.py"
    click HTML "https://github.com/bhushanmaheshwari/langchain-chatwithfiles/blob/main/htmlTemplates.py"
    click Upload "https://github.com/bhushanmaheshwari/langchain-chatwithfiles/blob/main/features/upload.py"
    click Conversation "https://github.com/bhushanmaheshwari/langchain-chatwithfiles/blob/main/features/conversation.py"
    click Start "https://github.com/bhushanmaheshwari/langchain-chatwithfiles/blob/main/processing/start.py"
    click Chunk "https://github.com/bhushanmaheshwari/langchain-chatwithfiles/blob/main/processing/chunk.py"
    click Ingest "https://github.com/bhushanmaheshwari/langchain-chatwithfiles/blob/main/processing/ingest.py"
    click VectorDBModule "https://github.com/bhushanmaheshwari/langchain-chatwithfiles/blob/main/processing/vectordb.py"
    click Storage "https://github.com/bhushanmaheshwari/langchain-chatwithfiles/blob/main/processing/storage.py"
    click RawFS "https://github.com/bhushanmaheshwari/langchain-chatwithfiles/tree/main/data/"
    click LogsUtil "https://github.com/bhushanmaheshwari/langchain-chatwithfiles/blob/main/utils/logs.py"
    click Session "https://github.com/bhushanmaheshwari/langchain-chatwithfiles/blob/main/utils/session.py"
    click AuthUtil "https://github.com/bhushanmaheshwari/langchain-chatwithfiles/blob/main/utils/login.py"

    %% Styles
    classDef frontend fill:#AED6F1,stroke:#3498DB,color:#1B4F72;
    classDef backend fill:#A3E4D7,stroke:#1ABC9C,color:#0E6251;
    classDef datastore fill:#F9E79F,stroke:#F1C40F,color:#7D6608;
    classDef external fill:#D5D8DC,stroke:#95A5A6,color:#424949;
```
