How brunialti/rag-system works
This repository is a desktop-assisted RAG workbench: users ingest supported documents into persistent stores, then retrieve ranked text chunks or documents for use as LLM context. The main path runs from the Tkinter UI through document loading, chunking and embedding, indexing, and multi-strategy search; optional reranking and diversity/cutoff features refine results. The README documents the major retrieval capabilities, while some UI-to-runtime calls and storage details are not visible in the excerpts, so those relationships are omitted or marked unverified.
Main entry points: UI_manager.py Store and query UI · vector_storage.py Persistent indexes · search_engine.py Hybrid and RRF fusion · document_manager.py Document loading
Components 16
User interface
- Store and query UI
UI_manager.py - Runtime configuration
rag_system/config.py
Document ingestion
- Document loading
rag_system/document_manager.py - Duplicate detection
rag_system/utils.py - Text chunking
rag_system/chunker.py - Dense embeddings
rag_system/embedding_manager.py
Index and persistence
- Store lifecycle
rag_system/vector_storage.py - Persistent indexes
rag_system/vector_storage.py
Retrieval and ranking
- FAISS retrieval
rag_system/search_engine.py - BM25 retrieval
rag_system/search_engine.py - TF-IDF retrieval
rag_system/search_engine.py - Hybrid and RRF fusion
rag_system/search_engine.py - Cross-encoder reranking
rag_system/search_engine.py - Knee and MMR refinement
rag_system/search_engine.py
Other components
- Workbench user
- Ranked text results
Connections 23
- Workbench user → Store and query UI: operates
README.md - Store and query UI → Runtime configuration: configures
- Store and query UI → Document loading: adds documents
UI_manager.py - Document loading → Duplicate detection: hashes content
rag_system/document_manager.py - Document loading → Dense embeddings: supplies text
- Dense embeddings → Text chunking: chunks long text
rag_system/embedding_manager.py - Store and query UI → Store lifecycle: manages stores
UI_manager.py - Dense embeddings → Persistent indexes: indexes vectors
rag_system/vector_storage.py - Store lifecycle → Persistent indexes: persists indexes
rag_system/vector_storage.py - Store and query UI → FAISS retrieval: queries
UI_manager.py - Store and query UI → BM25 retrieval: queries
UI_manager.py - Store and query UI → TF-IDF retrieval: queries
UI_manager.py - FAISS retrieval → Persistent indexes: reads dense index
- BM25 retrieval → Persistent indexes: reads BM25 index
- TF-IDF retrieval → Persistent indexes: reads TF-IDF index
- FAISS retrieval → Hybrid and RRF fusion: provides scores
README.md - BM25 retrieval → Hybrid and RRF fusion: provides scores
README.md - TF-IDF retrieval → Hybrid and RRF fusion: provides scores
README.md - Hybrid and RRF fusion → Cross-encoder reranking: ranks candidates
README.md - Cross-encoder reranking → Knee and MMR refinement: refines ranking
rag_system/search_engine.py - Knee and MMR refinement → Ranked text results: returns results
- Cross-encoder reranking → Ranked text results: returns results
rag_system/search_engine.py - Store and query UI → Ranked text results: displays results
UI_manager.py