# tnguyen1228/vietnamese_pdf_chatbot architecture

> Architecture diagram of the GitHub repository tnguyen1228/vietnamese_pdf_chatbot, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tnguyen1228/vietnamese_pdf_chatbot
- Repository: https://github.com/tnguyen1228/vietnamese_pdf_chatbot

## Overview

##### Project Type and Purpose
- The project is a full-stack application with a strong focus on natural language processing (NLP) and document analysis.
- It is built as a PDF chatbot in Vietnamese, which lets users upload PDF files, extract text and embeddings, and then query the document’s content via a chat interface.
- The README mentions using Streamlit for the UI and references techniques like chunking and retrieval augmented generation (RAG), which hints at a layered processing workflow.

##### File Structure Analysis
- The top-level files indicate a simple folder structure:
- Files like app.py, chat.py, botMode.py, embedding.py, pdf_processor.py, and text_processor.py represent different logical components.
- The use of .devcontainer and .envexample shows preparation for containerized or configured deployment.
- The presence of requirements.txt confirms dependency management.
- app.py likely manages the Streamlit-based user interface.
- pdf_processor.py and text_processor.py are responsible for handling PDF extraction and text processing.
- embedding.py might deal with generating text embeddings for semantic search.
- chat.py and botMode.py likely manage the chat logic or conversation flow, possibly integrating with external or internal language models.

##### Key Components Identified
a. Frontend/UI Component:
- Main Interface: Represented by app.py, which uses Streamlit for rendering the web UI.
- User Interaction: Handles file uploads, displays chat interactions, and triggers backend processing.

b. PDF Processing & Text Extraction:
- pdf_processor.py: Responsible for reading and extracting content from PDF files.
- text_processor.py: Cleans, segments, or pre-processes the text extracted from the PDFs.

c. Embedding and Data Processing:
- embedding.py: Generates embeddings for extracted text, based on one or more referenced techniques (e.g., for RAG purposes).
- background processing: Possibly handles indexing or search optimization over the text chunks.

d. Chat/Dialogue Management:
- chat.py and botMode.py: Manage the conversation logic, query interpretation, and response generation using the processed content.

e. External Interactions and Dependencies:
- Integration with language models or external services (as indicated by the tutorial references and RAG techniques in the README).
- Dependencies specified in requirements.txt likely include necessary NLP and machine learning libraries.

##### Relationships and Data Flow
- User Interaction Flow:
- User interacts with the frontend (Streamlit UI in app.py) to upload a PDF and input queries.
- The uploaded PDF is passed to the PDF processing module (pdf_processor.py) which extracts the text.
- Extracted text is sent to text_processor.py for cleaning and chunking.
- Clean text is then processed by embedding.py to generate embeddings.
- The chatbot (chat.py and botMode.py) uses these embeddings and processed text to retrieve information and answer queries.
- Results are relayed back to the UI for display.

- External Dependencies:

##### Architectural Patterns and Design Principles
- Modular Design: Each file is responsible for a distinct part of the process, from UI to processing to conversation handling.
- Separation of Concerns: Clearly divided responsibilities make it easier to update or troubleshoot individual parts.
- Clear Data Flow: The processing pipeline goes from file ingestion (PDF) => text extraction => text processing => embedding creation => chatbot answering queries.
- Extensibility: With distinct modules (and an external chat mode), future enhancements like adding support for different document formats or languages are simplified.

## Diagram (Mermaid)

```mermaid
graph TD
    User["User"]:::user

    subgraph "Frontend (UI)"
        UI["Streamlit UI (app.py)"]:::ui
    end

    subgraph "Backend Processing"
        PDF["PDF Processor (pdf_processor.py)"]:::processing
        TXT["Text Processor (text_processor.py)"]:::processing
        EMB["Embedding Generator (embedding.py)"]:::processing
    end

    subgraph "Chat/Dialogue Management"
        Chat["Chat Module (chat.py)"]:::processing
        BotMode["Bot Mode (botMode.py)"]:::processing
    end

    subgraph "External Services"
        EXT["NLP/LLM External Service"]:::external
    end

    subgraph "Configuration & Dependencies"
        CFG["Dev Container Config"]:::config
        ENV["Environment Example (.envexample)"]:::config
        REQ["Dependencies (requirements.txt)"]:::config
    end

    User -->|"uploadPDF"| UI
    UI -->|"sendPDF"| PDF
    PDF -->|"extractedText"| TXT
    TXT -->|"processedText"| EMB
    EMB -->|"generatedEmbeddings"| Chat
    Chat -->|"initChat"| BotMode
    BotMode -->|"queryService"| EXT
    EXT -->|"responseData"| BotMode
    BotMode -->|"finalResponse"| UI
    UI -->|"displayOutput"| User

    CFG --- UI
    CFG --- PDF
    CFG --- TXT
    CFG --- EMB
    CFG --- Chat
    CFG --- BotMode

    ENV --- UI
    ENV --- PDF
    ENV --- TXT
    ENV --- EMB
    ENV --- Chat
    ENV --- BotMode

    REQ --- PDF
    REQ --- TXT
    REQ --- EMB
    REQ --- Chat
    REQ --- BotMode

    %% Click Events
    click UI "https://github.com/tnguyen1228/vietnamese_pdf_chatbot/blob/main/app.py"
    click PDF "https://github.com/tnguyen1228/vietnamese_pdf_chatbot/blob/main/pdf_processor.py"
    click TXT "https://github.com/tnguyen1228/vietnamese_pdf_chatbot/blob/main/text_processor.py"
    click EMB "https://github.com/tnguyen1228/vietnamese_pdf_chatbot/blob/main/embedding.py"
    click Chat "https://github.com/tnguyen1228/vietnamese_pdf_chatbot/blob/main/chat.py"
    click BotMode "https://github.com/tnguyen1228/vietnamese_pdf_chatbot/blob/main/botMode.py"
    click CFG "https://github.com/tnguyen1228/vietnamese_pdf_chatbot/blob/main/.devcontainer/devcontainer.json"
    click ENV "https://github.com/tnguyen1228/vietnamese_pdf_chatbot/blob/main/.envexample"
    click REQ "https://github.com/tnguyen1228/vietnamese_pdf_chatbot/blob/main/requirements.txt"

    %% Styles
    classDef user fill:#F9E79F,stroke:#B7950B,stroke-width:2px;
    classDef ui fill:#AED6F1,stroke:#1B4F72,stroke-width:2px;
    classDef processing fill:#A9DFBF,stroke:#145A32,stroke-width:2px;
    classDef external fill:#F5B7B1,stroke:#922B21,stroke-width:2px;
    classDef config fill:#D2B4DE,stroke:#6C3483,stroke-width:2px;
```
