How rafsun-jany-rafy/rag-doc-assistant works
1. What this repository is
- This is a local Retrieval-Augmented Generation (RAG) document assistant for PDFs.
- The core architecture is a single-user Streamlit app that sends a user question through a local retrieval pipeline and then into a local LLM served by Ollama.
2. Primary runtime entrypoint
- app.py is the documented UI entrypoint for the Streamlit experience.
- main.py likely provides an alternate launch path or helper entrypoint, but the README positions app.py as the main application boundary.
3. User-facing layer
- The only interactive surface is the Streamlit UI.
- It accepts a PDF upload, accepts a natural-language query, and renders the generated answer with streaming output.
- There is no separate web API tier in the current design.
4. Orchestration boundary
- src/pipeline.py is the central coordinator.
- Its role is to connect ingestion, embedding, vector retrieval, prompt/context assembly, and LLM generation into one RAG flow.
- This is the main “application logic” layer, sitting between UI and model/storage subsystems.
5. Document ingestion subsystem
- src/ingest.py handles PDF text extraction and chunking.
- The README indicates PyMuPDF is used for parsing PDFs.
- This stage converts uploaded documents into text chunks suitable for embedding and later retrieval, with chunk overlap used to preserve context.
6. Embedding subsystem
- src/embeddings.py converts text chunks into vectors using sentence-transformers.
- This is the semantic representation layer that feeds the vector index.
- It is local and model-based, so no external embedding API is part of the architecture.
7. Retrieval and persistence subsystem
- src/retriever.py owns the FAISS vector store logic.
- It is responsible for adding chunk embeddings, searching for relevant chunks, and persisting the index so documents do not need to be re-indexed on every run.
- The architecture therefore includes a local durable retrieval store on disk, separate from the UI and LLM.
8. Generation subsystem
- src/llm.py wraps the Ollama connection for the local Llama 3.2 model.
- It receives the prompt plus retrieved context and produces the final answer.
- Responses are streamed back to the UI, so generation is not a single blocking “all at once” step from the user’s perspective.
9. End-to-end data flow
- The runtime path is: upload PDF → extract/chunk text → embed chunks → store/search FAISS index → retrieve top context → build prompt → call local LLM → stream answer to UI.
- The first document query is slower because ingestion and indexing happen before retrieval can reuse the persisted index.
10. State and storage boundaries
- The repo is designed around local persistence rather than remote services.
- The README references project directories for uploaded documents and saved vector indexes, which means the system keeps both raw document inputs and retrieval state on disk.
- This creates a clear separation between ephemeral request handling and durable document/index storage.
11. Infrastructure and deployment shape
- The presence of Dockerfile and docker-compose.yml indicates the project is intended to be containerizable, even though the README’s primary workflow is local execution.
- The Ollama dependency is an important runtime boundary: the app expects a locally running model server rather than a cloud API.
12. Non-production / experimental code
- notebooks/rag_experiment.ipynb is an R&D workspace for trying retrieval or prompting ideas.
- It is not part of the primary application path, but it likely informed the chunking, retrieval, or prompting strategy used in src/.
13. How to draw the architecture
- The diagram should show three main zones:
- UI layer: Streamlit app
- RAG backend layer: ingestion, embeddings, retriever, pipeline, LLM wrapper
- Local persistence/runtime layer: stored PDFs, FAISS index files, Ollama/Llama 3.2
- The key external dependency is Ollama, and the key storage dependency is FAISS-backed local index files.
Main entry points: pipeline.py RAG Pipeline · retriever.py Retriever · ingest.py PDF Ingest · app.py Streamlit App
Components 11
UI
Streamlit user interface and launch entrypoints
- Streamlit App · Primary user-facing Streamlit experience for upload, query, and streaming answers
app.py - Main Entrypoint · Alternate application launch path or wrapper around the main app
main.py
RAG Backend
Document ingestion, retrieval, and answer generation
- RAG Pipeline · Coordinates ingestion, retrieval, prompt assembly, and generation
src/pipeline.py - PDF Ingest · Extracts PDF text and chunks it for downstream embedding
src/ingest.py - Embeddings · Converts text chunks into local semantic vectors
src/embeddings.py - Retriever · Adds vectors, searches relevant chunks, and manages FAISS-backed retrieval
src/retriever.py - LLM Client · Builds prompts with context and streams answers from the local model
src/llm.py
Local Runtime
On-disk persistence and local model server
- Uploaded PDFs · Locally stored source PDFs used for ingestion and reuse
- Chunk Store · Intermediate text chunks produced from PDF parsing
- FAISS Index · Persistent local vector index used for reuse across runs
- Ollama Server · Local runtime serving the Llama 3.2 model for answer generation
Connections 13
- Streamlit App → RAG Pipeline: query / upload · User actions enter the orchestration layer
- Main Entrypoint → Streamlit App: starts · Alternate entrypoint launches the Streamlit app
- RAG Pipeline → PDF Ingest: ingest docs · Routes uploaded PDFs into parsing and chunking
- RAG Pipeline → Retriever: search context · Requests relevant chunks for a question
- RAG Pipeline → LLM Client: generate answer · Sends prompt and retrieved context to the model wrapper
- PDF Ingest → Uploaded PDFs: read · Consumes stored or uploaded PDF sources
- PDF Ingest → Chunk Store: emit chunks · Produces chunked text for embedding
- Chunk Store → Embeddings: vectorize · Chunk text is converted into embeddings
- Embeddings → Retriever: store vectors · Embeddings are indexed for later lookup
- Retriever → FAISS Index: persist / load · Uses the local FAISS index across runs
- Retriever → RAG Pipeline: top-k context · Returns the most relevant retrieved passages
- LLM Client → Ollama Server: call model · Delegates generation to the local Ollama service
- Ollama Server → Streamlit App: stream output · Generated tokens flow back to the UI for streaming display