# brunialti/rag-system architecture

> Architecture diagram of the GitHub repository brunialti/rag-system, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/brunialti/rag-system
- Repository: https://github.com/brunialti/rag-system

## Overview

This repository is a desktop-assisted RAG workbench: users ingest supported documents into persistent stores, then retrieve ranked text chunks or documents for use as LLM context. The main path runs from the Tkinter UI through document loading, chunking and embedding, indexing, and multi-strategy search; optional reranking and diversity/cutoff features refine results. The README documents the major retrieval capabilities, while some UI-to-runtime calls and storage details are not visible in the excerpts, so those relationships are omitted or marked unverified.

Main entry points: [`UI_manager.py`](https://github.com/brunialti/rag-system/blob/main/UI_manager.py) (Store and query UI), [`rag_system/vector_storage.py`](https://github.com/brunialti/rag-system/blob/main/rag_system/vector_storage.py) (Persistent indexes), [`rag_system/search_engine.py`](https://github.com/brunialti/rag-system/blob/main/rag_system/search_engine.py) (Hybrid and RRF fusion), [`rag_system/document_manager.py`](https://github.com/brunialti/rag-system/blob/main/rag_system/document_manager.py) (Document loading)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ui["User interface"]
  node_ui_app["Store and query UI<br/>[UI_manager.py]"]
  node_config["Runtime configuration<br/>[config.py]"]
end

subgraph group_ingest["Document ingestion"]
  node_doc_load["Document loading"]
  node_dedup["Duplicate detection<br/>[utils.py]"]
  node_chunking["Text chunking<br/>[chunker.py]"]
  node_embedding["Dense embeddings"]
end

subgraph group_index["Index and persistence"]
  node_store_manager["Store lifecycle<br/>[vector_storage.py]"]
  node_persistent_store[("Persistent indexes<br/>[vector_storage.py]")]
end

subgraph group_retrieve["Retrieval and ranking"]
  node_dense_search["FAISS retrieval<br/>[search_engine.py]"]
  node_sparse_search["BM25 retrieval<br/>[search_engine.py]"]
  node_tfidf_search["TF-IDF retrieval<br/>[search_engine.py]"]
  node_fusion["Hybrid and RRF fusion<br/>[search_engine.py]"]
  node_reranking["Cross-encoder reranking<br/>[search_engine.py]"]
  node_refinement["Knee and MMR refinement<br/>[search_engine.py]"]
end

node_user(("Workbench user"))
node_results["Ranked text results"]

node_user -->|"operates"| node_ui_app
node_ui_app -.->|"configures"| node_config
node_ui_app -.->|"adds documents"| node_doc_load
node_doc_load -->|"hashes content"| node_dedup
node_doc_load -.->|"supplies text"| node_embedding
node_embedding -->|"chunks long text"| node_chunking
node_ui_app -.->|"manages stores"| node_store_manager
node_embedding -.->|"indexes vectors"| node_persistent_store
node_store_manager -->|"persists indexes"| node_persistent_store
node_ui_app -.->|"queries"| node_dense_search
node_ui_app -.->|"queries"| node_sparse_search
node_ui_app -.->|"queries"| node_tfidf_search
node_dense_search -.->|"reads dense index"| node_persistent_store
node_sparse_search -.->|"reads BM25 index"| node_persistent_store
node_tfidf_search -.->|"reads TF-IDF index"| node_persistent_store
node_dense_search -->|"provides scores"| node_fusion
node_sparse_search -->|"provides scores"| node_fusion
node_tfidf_search -->|"provides scores"| node_fusion
node_fusion -->|"ranks candidates"| node_reranking
node_reranking -.->|"refines ranking"| node_refinement
node_refinement -.->|"returns results"| node_results
node_reranking -->|"returns results"| node_results
node_ui_app -->|"displays results"| node_results

click node_ui_app "https://github.com/brunialti/rag-system/blob/main/UI_manager.py"
click node_config "https://github.com/brunialti/rag-system/blob/main/rag_system/config.py"
click node_doc_load "https://github.com/brunialti/rag-system/blob/main/rag_system/document_manager.py"
click node_dedup "https://github.com/brunialti/rag-system/blob/main/rag_system/utils.py"
click node_chunking "https://github.com/brunialti/rag-system/blob/main/rag_system/chunker.py"
click node_embedding "https://github.com/brunialti/rag-system/blob/main/rag_system/embedding_manager.py"
click node_store_manager "https://github.com/brunialti/rag-system/blob/main/rag_system/vector_storage.py"
click node_persistent_store "https://github.com/brunialti/rag-system/blob/main/rag_system/vector_storage.py"
click node_dense_search "https://github.com/brunialti/rag-system/blob/main/rag_system/search_engine.py"
click node_sparse_search "https://github.com/brunialti/rag-system/blob/main/rag_system/search_engine.py"
click node_tfidf_search "https://github.com/brunialti/rag-system/blob/main/rag_system/search_engine.py"
click node_fusion "https://github.com/brunialti/rag-system/blob/main/rag_system/search_engine.py"
click node_reranking "https://github.com/brunialti/rag-system/blob/main/rag_system/search_engine.py"
click node_refinement "https://github.com/brunialti/rag-system/blob/main/rag_system/search_engine.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_ui_app,node_config,node_user toneBlue
class node_doc_load,node_dedup,node_chunking,node_embedding toneAmber
class node_store_manager,node_persistent_store toneMint
class node_dense_search,node_sparse_search,node_tfidf_search,node_fusion,node_reranking,node_refinement toneRose
class node_results toneIndigo
```

## Components

### User interface

- [Store and query UI](https://github.com/brunialti/rag-system/blob/main/UI_manager.py): `UI_manager.py`
- [Runtime configuration](https://github.com/brunialti/rag-system/blob/main/rag_system/config.py): `rag_system/config.py`

### Document ingestion

- [Document loading](https://github.com/brunialti/rag-system/blob/main/rag_system/document_manager.py): `rag_system/document_manager.py`
- [Duplicate detection](https://github.com/brunialti/rag-system/blob/main/rag_system/utils.py): `rag_system/utils.py`
- [Text chunking](https://github.com/brunialti/rag-system/blob/main/rag_system/chunker.py): `rag_system/chunker.py`
- [Dense embeddings](https://github.com/brunialti/rag-system/blob/main/rag_system/embedding_manager.py): `rag_system/embedding_manager.py`

### Index and persistence

- [Store lifecycle](https://github.com/brunialti/rag-system/blob/main/rag_system/vector_storage.py): `rag_system/vector_storage.py`
- [Persistent indexes](https://github.com/brunialti/rag-system/blob/main/rag_system/vector_storage.py): `rag_system/vector_storage.py`

### Retrieval and ranking

- [FAISS retrieval](https://github.com/brunialti/rag-system/blob/main/rag_system/search_engine.py): `rag_system/search_engine.py`
- [BM25 retrieval](https://github.com/brunialti/rag-system/blob/main/rag_system/search_engine.py): `rag_system/search_engine.py`
- [TF-IDF retrieval](https://github.com/brunialti/rag-system/blob/main/rag_system/search_engine.py): `rag_system/search_engine.py`
- [Hybrid and RRF fusion](https://github.com/brunialti/rag-system/blob/main/rag_system/search_engine.py): `rag_system/search_engine.py`
- [Cross-encoder reranking](https://github.com/brunialti/rag-system/blob/main/rag_system/search_engine.py): `rag_system/search_engine.py`
- [Knee and MMR refinement](https://github.com/brunialti/rag-system/blob/main/rag_system/search_engine.py): `rag_system/search_engine.py`

### Other components

- **Workbench user**
- **Ranked text results**

## Connections

- Workbench user → Store and query UI: operates (evidence: `README.md`)
- Store and query UI → Runtime configuration: configures
- Store and query UI → Document loading: adds documents (evidence: `UI_manager.py`)
- Document loading → Duplicate detection: hashes content (evidence: `rag_system/document_manager.py`)
- Document loading → Dense embeddings: supplies text
- Dense embeddings → Text chunking: chunks long text (evidence: `rag_system/embedding_manager.py`)
- Store and query UI → Store lifecycle: manages stores (evidence: `UI_manager.py`)
- Dense embeddings → Persistent indexes: indexes vectors (evidence: `rag_system/vector_storage.py`)
- Store lifecycle → Persistent indexes: persists indexes (evidence: `rag_system/vector_storage.py`)
- Store and query UI → FAISS retrieval: queries (evidence: `UI_manager.py`)
- Store and query UI → BM25 retrieval: queries (evidence: `UI_manager.py`)
- Store and query UI → TF-IDF retrieval: queries (evidence: `UI_manager.py`)
- FAISS retrieval → Persistent indexes: reads dense index
- BM25 retrieval → Persistent indexes: reads BM25 index
- TF-IDF retrieval → Persistent indexes: reads TF-IDF index
- FAISS retrieval → Hybrid and RRF fusion: provides scores (evidence: `README.md`)
- BM25 retrieval → Hybrid and RRF fusion: provides scores (evidence: `README.md`)
- TF-IDF retrieval → Hybrid and RRF fusion: provides scores (evidence: `README.md`)
- Hybrid and RRF fusion → Cross-encoder reranking: ranks candidates (evidence: `README.md`)
- Cross-encoder reranking → Knee and MMR refinement: refines ranking (evidence: `rag_system/search_engine.py`)
- Knee and MMR refinement → Ranked text results: returns results
- Cross-encoder reranking → Ranked text results: returns results (evidence: `rag_system/search_engine.py`)
- Store and query UI → Ranked text results: displays results (evidence: `UI_manager.py`)
