# pcastiglione99/ragify-search architecture

> Architecture diagram of the GitHub repository pcastiglione99/ragify-search, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pcastiglione99/ragify-search
- Repository: https://github.com/pcastiglione99/ragify-search

## Overview

##### Main Components
- User Interface: Streamlit-based front end (app.py) where the user interacts with the chatbot-like interface.
- Configuration Module: Configuration settings stored in config.py which determine parameters such as chunk sizes, model names, and directories.
- Web Scraping Module: Handled by web_scraper.py, responsible for fetching real-time web content and managing temporary files.
- Query Extraction Module: Managed by extract_queries.py; this module generates optimized queries based on user inputs.
- Document Processing & Indexing: Managed by db_operations.py. This component embeds document chunks and builds an index for similarity-based searches.
- Prompt Generation: Managed by prompt_generator.py that prepares structured prompts for interaction with the local LLM.
- Local LLM Integration: The system utilizes Ollama (with a specified model via langchain ollama) to process text and generate responses.
- Optional: A storage or caching layer may be implicitly present as part of temporary file management and document embedding.

##### Component Relationships and Data Flow
- The user inputs a query in the Streamlit interface.
- The query is sent from the frontend (app.py) to the Query Extraction Module which optimizes the query for search.
- The optimized query triggers the Web Scraping Module (web_scraper.py) to fetch relevant pages.
- Once the content is fetched, the Document Processing module (db_operations.py) processes, splits, embeds, and indexes the content.
- The Prompt Generation module (prompt_generator.py) then takes the processed content along with the user input to generate a structured prompt.
- Finally, the prompt is fed into the Local LLM (via Ollama) to obtain a response which is then displayed in the Streamlit UI.
- Configuration values from config.py guide parameters across all modules.

##### Architectural Patterns and Design Principles
- Separation of Concerns: Each module (UI, web scraping, query extraction, document processing, prompt generation) has a distinct responsibility.
- Modular Design: The code is split into separate files for each functionality which suggests an extensible and maintainable design.
- Real-Time Interaction: Integration with a local LLM for privacy and performance indicates an emphasis on quick, local inference.

## Diagram (Mermaid)

```mermaid
flowchart TD
    User("User"):::External

    subgraph "Frontend"
        UI["User Interface (Streamlit App)"]:::UI
    end

    subgraph "Backend"
        QE["Query Extraction Module"]:::Backend
        WS["Web Scraping Module (manages temporary files)"]:::Backend
        DP["Document Processing & Indexing"]:::Backend
        PG["Prompt Generation"]:::Backend
        LLM["Local LLM (Ollama)"]:::External
    end

    CFG["Configuration Module"]:::Config

    %% Data Flow
    User -->|"submits query"| UI
    UI -->|"sends query"| QE
    QE -->|"triggers"| WS
    WS -->|"processes and indexes"| DP
    DP -->|"prepares data for"| PG
    PG -->|"generates prompt for"| LLM
    LLM -->|"returns response to"| UI

    %% Configuration Support Arrows (dotted)
    CFG -.-> QE
    CFG -.-> WS
    CFG -.-> DP
    CFG -.-> PG

    %% Click Events
    click UI "https://github.com/pcastiglione99/ragify-search/blob/main/src/app.py"
    click CFG "https://github.com/pcastiglione99/ragify-search/blob/main/src/config.py"
    click QE "https://github.com/pcastiglione99/ragify-search/blob/main/src/extract_queries.py"
    click WS "https://github.com/pcastiglione99/ragify-search/blob/main/src/web_scraper.py"
    click DP "https://github.com/pcastiglione99/ragify-search/blob/main/src/db_operations.py"
    click PG "https://github.com/pcastiglione99/ragify-search/blob/main/src/prompt_generator.py"

    %% Styles
    classDef UI fill:#ADD8E6,stroke:#000,stroke-width:2px;
    classDef Backend fill:#90EE90,stroke:#000,stroke-width:2px;
    classDef External fill:#FFD700,stroke:#000,stroke-width:2px;
    classDef Config fill:#FFB6C1,stroke:#000,stroke-width:2px;
```
