# hinmq21/documinchat architecture

> Architecture diagram of the GitHub repository hinmq21/documinchat, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hinmq21/documinchat
- Repository: https://github.com/hinmq21/documinchat

## Overview

##### Main Components
- Configuration and Setup: config.py serves as the configuration center, possibly holding important system settings.
- Data Processing & File Management:
– File Loader: The file loading utilities in utils/file_loader.py will be responsible for reading uploaded files.
– Text Processing & Chunking: Use utils/text_processor.py for breaking the content into manageable chunks.
– Pagination: Also reflected in the UI and potentially managed by utils or modules in app.py.
- Embedding Generation and Model Interaction:
– Embeddings Module: models/embeddings.py handles the generation of embeddings.
– Language Model Integration: models/llm.py integrates the language model for conversational AI.
– Model Setup/Download Information: models/download.txt contains instructions on setting up the models.
- Utility Support:
– Chat History: utils/chat_history.py manages conversational state.
– Relevance Checking: utils/relevance_checker.py likely helps in filtering or selecting the most pertinent data segments.
– Token Utilities: utils/token_utils.py helps with tokenization tasks as needed for NLP models.

##### Out Relationships and Interactions
- The Frontend (app.py) is the primary touchpoint where users interact:
– It triggers file uploads.
– It hands off the file to the file loader (utils/file_loader.py) for processing.
– The output is then processed—chunked—through the text processing utility (utils/text_processor.py).
– Processed chunks are used by the embeddings module (models/embeddings.py) to generate embeddings.
– The language model module (models/llm.py) then uses these embeddings to handle and respond to user queries.
– The chat history utility (utils/chat_history.py) logs the conversation, and token_utils (utils/token_utils.py) ensure proper token management.
– Relevance checking (utils/relevance_checker.py) may be used to filter or rank chunks during query analysis.
- Configuration via config.py supports the overall system settings.

##### Architectural Patterns and Key Design Principles
- Layered Architecture: There is a clear separation of concerns between presentation (Streamlit app), business logic (file processing, data chunking, conversational AI), and model integrations.
- Modularity: The project is broken into independent directories (models and utils) that allow for extensibility and easier maintenance.
- Integration of External Services: Integration with external libraries (LangChain, Hugging Face) for NLP and language model operations.

a. Frontend/UI: Label it “Streamlit Application – app.py” and indicate its actions (file upload, pagination, query submission).
b. File Processing Module: Include the “File Loader” (utils/file_loader.py) and “Text Processor” (utils/text_processor.py) as subcomponents.
d. Utility Modules: Group together “Chat History”, “Relevance Checker”, and “Token Utils” from the utils directory.
e. Configuration: A separate node that feeds into different parts of the system, coming from config.py.
f. External Libraries & Frameworks: Notate Streamlit for the UI, and note LangChain and Hugging Face for the NLP/model aspects.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Frontend/UI Layer
    subgraph "Frontend/UI"
        UI["Streamlit Application (app.py)"]:::frontend
    end

    %% File Processing Module
    subgraph "File Processing"
        FL["File Loader (utils/file_loader.py)"]:::processing
        TP["Text Processor (utils/text_processor.py)"]:::processing
    end

    %% Embedding & NLP Processing
    subgraph "Embedding & NLP Processing"
        EG["Embeddings Generation (models/embeddings.py)"]:::nlp
        LM["Language Model Interaction (models/llm.py)"]:::nlp
        MD["Model Setup/Download (models/download.txt)"]:::nlp
    end

    %% Utility Services
    subgraph "Utility Services"
        CH["Chat History (utils/chat_history.py)"]:::utility
        RC["Relevance Checker (utils/relevance_checker.py)"]:::utility
        TU["Token Utils (utils/token_utils.py)"]:::utility
    end

    %% Configuration
    subgraph "Configuration"
        CFG["Configuration (config.py)"]:::configuration
    end

    %% External Frameworks
    subgraph "External Frameworks"
        ST["Streamlit"]:::external
        LC["LangChain"]:::external
        HF["Hugging Face"]:::external
    end

    %% Data Flow Connections
    UI -->|"upload_file"| FL
    FL -->|"load_file"| TP
    TP -->|"prepare_chunks"| EG
    EG -->|"generate_embeddings"| LM
    LM -->|"answer_query"| UI
    LM -->|"log_conversation"| CH
    LM -->|"manage_tokens"| TU
    LM -->|"filter_relevance"| RC
    MD -->|"provides_model_info"| EG

    %% Configuration Flows
    CFG -->|"configures"| UI
    CFG -->|"configures"| FL
    CFG -->|"configures"| TP
    CFG -->|"configures"| EG
    CFG -->|"configures"| LM

    %% External Dependencies
    UI --- ST
    LM --- LC
    LM --- HF

    %% Utility Interactions
    CH --- TU
    CH --- RC

    %% Click Events
    click UI "https://github.com/hinmq21/documinchat/blob/main/app.py"
    click CFG "https://github.com/hinmq21/documinchat/blob/main/config.py"
    click FL "https://github.com/hinmq21/documinchat/blob/main/utils/file_loader.py"
    click TP "https://github.com/hinmq21/documinchat/blob/main/utils/text_processor.py"
    click EG "https://github.com/hinmq21/documinchat/blob/main/models/embeddings.py"
    click LM "https://github.com/hinmq21/documinchat/blob/main/models/llm.py"
    click MD "https://github.com/hinmq21/documinchat/blob/main/models/download.txt"
    click CH "https://github.com/hinmq21/documinchat/blob/main/utils/chat_history.py"
    click RC "https://github.com/hinmq21/documinchat/blob/main/utils/relevance_checker.py"
    click TU "https://github.com/hinmq21/documinchat/blob/main/utils/token_utils.py"

    %% Class Styles
    classDef frontend fill:#CDEEFF,stroke:#333,stroke-width:2px;
    classDef processing fill:#FFDD99,stroke:#333,stroke-width:2px;
    classDef nlp fill:#D3F9D8,stroke:#333,stroke-width:2px;
    classDef utility fill:#FAD4D8,stroke:#333,stroke-width:2px;
    classDef configuration fill:#E8DAEF,stroke:#333,stroke-width:2px;
    classDef external fill:#D0E6A5,stroke:#333,stroke-width:2px;
```
