How thegreatpratyush/rag works
Local RAG is a document question-answering product: users upload files, which are parsed, cleaned, chunked, embedded, and stored in dense and lexical indexes; questions are searched through hybrid retrieval, reranked, answered by a local LLM, and returned with page-aware citations or a refusal. The graph captures the documented product capabilities and source-supported orchestration. Major retrieval internals not individually excerpted are included where the README and source index establish them; those links are documented rather than inferred from module co-location.
Main entry points: rag_pipeline.py RAG pipeline · routes.py Upload and ask routes · vector_store.py FAISS vector store · index_manager.py FAISS index manager
Components 18
User interface
- RAG interface
frontend/src/App.jsx
API orchestration
- Upload and ask routes
src/api/routes.py - Request and response schemas
src/api/schemas.py - RAG pipeline
src/pipeline/rag_pipeline.py
Document ingestion
- Document parser
src/ingestion/parser.py - Text cleaner
src/ingestion/cleaner.py - Page-aware chunker
src/ingestion/chunker.py
Search and indexing
- BGE embedder
src/embeddings/embedder.py - FAISS index manager
src/retrieval/index_manager.py - FAISS vector store
src/retrieval/vector_store.py - BM25 index manager
src/retrieval/bm25_manager.py - BM25 lexical store
src/retrieval/bm25.py - Hybrid RRF retrieval
src/retrieval/hybrid.py - BGE reranker
src/retrieval/reranker.py
Grounded answering
- Grounded prompt builder
src/generation/prompt_builder.py - Ollama local LLM
src/generation/llm.py
Other components
- Document user
- Ollama service
Connections 24
- Document user → RAG interface: uploads and asks
frontend/src/App.jsx - RAG interface → Upload and ask routes: sends HTTP requests
frontend/src/App.jsx - Upload and ask routes → Request and response schemas: validates and formats
src/api/routes.py - Upload and ask routes → RAG pipeline: invokes indexing and ask
src/api/routes.py - RAG pipeline → Document parser: extracts document text
src/pipeline/rag_pipeline.py - RAG pipeline → Text cleaner: cleans page text
src/pipeline/rag_pipeline.py - RAG pipeline → Page-aware chunker: creates chunks
src/pipeline/rag_pipeline.py - RAG pipeline → BGE embedder: embeds chunks and queries
src/pipeline/rag_pipeline.py - RAG pipeline → FAISS index manager: loads and saves dense index
src/pipeline/rag_pipeline.py - RAG pipeline → BM25 index manager: loads and saves lexical index
src/pipeline/rag_pipeline.py - RAG pipeline → FAISS vector store: replaces and adds chunks
src/pipeline/rag_pipeline.py - RAG pipeline → BM25 lexical store: replaces and adds chunks
src/pipeline/rag_pipeline.py - FAISS index manager → BGE embedder: embeds indexed text
src/retrieval/index_manager.py - FAISS index manager → FAISS vector store: builds dense index
src/retrieval/index_manager.py - BM25 index manager → BM25 lexical store: builds lexical index
src/retrieval/bm25_manager.py - RAG pipeline → Hybrid RRF retrieval: retrieves candidates
src/pipeline/rag_pipeline.py - Hybrid RRF retrieval → FAISS vector store: searches semantic index
src/retrieval/hybrid.py - Hybrid RRF retrieval → BM25 lexical store: searches lexical index
src/retrieval/hybrid.py - RAG pipeline → BGE reranker: reranks candidates
src/pipeline/rag_pipeline.py - RAG pipeline → Grounded prompt builder: builds answer prompt
src/pipeline/rag_pipeline.py - RAG pipeline → Ollama local LLM: generates answer
src/pipeline/rag_pipeline.py - Ollama local LLM → Ollama service: requests local generation
- Upload and ask routes → RAG interface: returns answer and citations
src/api/routes.py - Grounded prompt builder → FAISS vector store: uses retrieved chunk type
src/generation/prompt_builder.py