How pbaniket/research_assistant works
1. What this repo is
- A fully local, document-grounded Q&A system using a standard RAG pipeline.
- It answers questions over PDFs stored on disk, without sending data to external APIs.
2. Top-level architecture
- app.py is the entrypoint and orchestrator.
- processing.py owns offline document ingestion and index creation.
- qa.py owns online question answering: retrieval, prompt assembly, and response generation.
- config.py centralizes paths and runtime settings.
3. Offline ingestion boundary
- The first major subsystem is the document pipeline.
- PDFs are loaded from a local docs/ folder, cleaned, split into overlapping chunks, embedded, and written into a FAISS index under faiss_index/.
- This stage is meant to happen once or whenever the source documents change.
4. Document preprocessing flow
- Source: local PDF files.
- Processing: page-level loading, text cleanup, chunking with overlap to preserve context across boundaries.
- Output: chunk metadata plus embeddings suitable for retrieval.
- The design keeps page/document provenance so answers can cite source document and page number.
5. Indexing and persistence
- FAISS is the persistent retrieval layer.
- The vector index is cached locally so startup does not require rebuilding embeddings every time.
- This makes the system stateful across runs, but only through local files.
6. Online question-answering flow
- User input enters through the chat UI and is handled by the QA subsystem.
- The question is embedded, matched against FAISS, and the most relevant chunks are pulled as context.
- That retrieved context is inserted into a prompt template along with the question.
- The local LLM generates an answer constrained to the retrieved context.
7. Model runtime boundary
- The generative model runs locally in GGUF format via llama.cpp-compatible tooling.
- The embedding model is separate from the generator: sentence-transformer embeddings for retrieval, Gemma-2 for answer synthesis.
- This separation is important architecturally because retrieval and generation have different compute roles and dependency stacks.
8. User interface and interaction
- Gradio provides the chat UI and serves the local web app.
- app.py likely wires the UI callback to the QA pipeline and initializes the index if needed before launching the server.
- The UI is thin; it does not own business logic.
9. Configuration and portability
- config.py holds project-relative paths for documents, models, and index storage.
- This makes the repo portable across OSes and local setups, because the runtime is driven by filesystem layout rather than hard-coded machine-specific paths.
10. Data boundaries
- Input boundary: local PDFs in docs/.
- Working boundary: chunked text and embeddings in memory during processing.
- Persistent boundary: FAISS index on disk in faiss_index/.
- Runtime boundary: model files in models/ and the Gradio server process.
11. Technology footprint that matters architecturally
- LangChain provides the RAG plumbing and likely document/loading abstractions.
- FAISS provides fast nearest-neighbor retrieval.
- Sentence-Transformers provides the retrieval embeddings.
- Gradio provides the local interactive front end.
- Llama.cpp enables local inference from a GGUF model file.
12. How to diagram it
- Draw two phases: an offline indexing pipeline and an online chat pipeline.
- Show local PDFs feeding ingestion, then a cached FAISS store.
- Show user queries entering Gradio, flowing to retrieval in FAISS, then to the local LLM, and finally back to the UI with citations.
- Emphasize that all compute and storage remain local to the machine.
Main entry points: app.py App · qa.py QA · processing.py Ingestion · config.py Config
Components 14
Offline indexing
Local document ingestion and FAISS index creation
- Ingestion · Loads PDFs, cleans text, chunks content, and builds embeddings
processing.py - Local PDFs · Input document set stored on disk
- Clean & split · Text cleanup and overlapping chunking with provenance
- Embed docs · Sentence-transformer embeddings for document chunks
- FAISS store · Persistent local retrieval index for chunks and metadata
Online QA
Interactive retrieval and local answer generation
- QA · Retrieves context, builds prompts, and generates answers
qa.py - Gradio UI · Thin local chat interface for user questions
- Query embedder · Embeds the user question for similarity search
- Retriever · Finds the most relevant chunks in FAISS
- Prompt builder · Assembles retrieved context and the question into an answer prompt
- Local LLM · GGUF llama.cpp-compatible model for grounded answer synthesis
- Answer · Returns the response with source-grounded context
Connections 16
- App → Config: loads settings · Reads project paths and runtime configuration
- App → Ingestion: initializes index · Ensures the local vector store exists before serving
- App → Gradio UI: starts UI · Launches the local Gradio application
- Local PDFs → Ingestion: documents · Source PDFs are ingested from disk
- Ingestion → Clean & split: prepares text · Cleans pages and splits them into overlapping chunks
- Clean & split → Embed docs: chunk embeddings · Creates dense representations for each chunk
- Embed docs → FAISS store: stores vectors · Persists embeddings and metadata into FAISS
- FAISS store → Retriever: search corpus · Provides nearest-neighbor lookup over indexed chunks
- Gradio UI → QA: user question · Sends the user prompt into the QA pipeline
- QA → Query embedder: embed query · Encodes the question for retrieval
- Query embedder → Retriever: similarity search · Uses the query vector to find relevant chunks
- Retriever → Prompt builder: context · Passes top matching chunks and provenance
- QA → Prompt builder: assemble prompt · Combines question and retrieved context
- Prompt builder → Local LLM: generate answer · Feeds the grounded prompt to the local model
- Local LLM → Answer: response · Produces the final answer text
- Answer → Gradio UI: display · Returns the answer back to the chat interface