How weiwill88/local_pdf_chat_rag works
Project Type & Purpose
• This is a monolithic Python “full-stack” RAG (Retrieval-Augmented Generation) demo: it combines a lightweight frontend (Gradio UI and optional FastAPI endpoints) with a backend pipeline for PDF ingestion, vectorization (FAISS), keyword indexing (BM25), retrieval, re-ranking, external search, and LLM‐powered answer generation (Ollama or SiliconFlow).
File Structure
• Top-level scripts:
• rag_demo_pro.py → launches the Gradio interface and ties together PDF upload, question handling, and the RAG pipeline.
• Configuration: example.env, requirements.txt
• Assets: images/ (UI screenshots, QR codes)
• utils/ → Python modules for PDF parsing, text splitting, indexing, retrieval, LLM calls, etc.
Core Components
a. User Interaction Layer
• Gradio Web UI (rag_demo_pro.py)
• FastAPI endpoints (api_router.py—optional)
b. Data Processing & Storage Layer
• PDF Processing Module (pdfminer → text extraction)
• Text Splitting (langchain-text-splitters)
• Vectorization (Sentence-Transformers)
• FAISS Vector Index (faiss-cpu)
• BM25 Index (rank_bm25 + jieba)
• Document store (filesystem or in-memory metadata)
c. Retrieval & Re-ranking Layer
• FAISS semantic search
• BM25 keyword search
• Hybrid merging & CrossEncoder re-ranking
d. Augmentation & Generation Layer
• Optional external SerpAPI search
• LLM Inference Module (local Ollama or cloud SiliconFlow)
• Recursive query generation & iteration logic
e. External Services
• SerpAPI (real-time web search)
• Ollama service (local LLM)
• SiliconFlow API (cloud LLM)
Relationships & Data Flow
1. User → UI/API → PDF Processing
2. PDF → Text Split → Vectorize → FAISS & BM25 indices
3. User Query → Vectorize → FAISS → candidate chunks
4. User Query → BM25 → keyword hits
5. Merge FAISS+BM25 → CrossEncoder re-rank → context set
6. (Optional) Query → SerpAPI → external snippets → merge
7. Context + Query → LLM → Answer → UI → (loop for recursive queries)
Architectural Patterns & Principles
• Layered architecture: separate UI, processing, storage, retrieval, generation.
• Modular pipeline: each processing step lives in utils/ modules.
• Hybrid retrieval pattern: semantic + keyword + external search.
• Recursive/iterative RAG: LLM can trigger further retrieval rounds.
Technologies & Frameworks
• Python 3.9+
• Gradio (UI layer)
• FastAPI + Uvicorn (API layer, optional)
• pdfminer.six (PDF parsing)
• langchain-text-splitters (splitting)
• sentence-transformers (vectorization & cross-encoder)
• faiss-cpu (vector index)
• rank_bm25 + jieba (keyword index)
• requests/urllib3 (HTTP calls)
• python-dotenv (env config)
• Ollama & SiliconFlow APIs (LLM inference)
• SerpAPI (external search)
Essential Elements to Capture
• All major modules (PDF Processor, Splitter, Vectorizer, FAISS, BM25, Hybrid Retriever, CrossEncoder, External Search, LLM).
• Data stores (index files or in-memory data structures).
• External dependencies (SerpAPI, Ollama, SiliconFlow).
• Interaction flows (upload, ask, retrieve, rerank, generate, deliver).
• Recursive iteration path.