How alielbaha/rag-pipeline works
1. What this repo is
- This is a local end-to-end RAG application centered on a single PDF corpus (data/document1.pdf).
- The system follows the classic flow: document ingestion → chunking/embedding → vector storage → semantic retrieval → prompt construction → LLM answer generation.
2. Main runtime boundaries
- The repo is organized into a small number of Python modules under src/, with two top-level entrypoints:
- app.py for the Streamlit UI
- cli.py for command-line execution
- Shared logic lives in src/, while UI/entrypoint code stays thin and delegates to those modules.
3. Ingestion and indexing subsystem
- src/indexer.py is the document-processing layer.
- It likely handles extracting text from the PDF, splitting it into chunks, generating embeddings, and writing those vectors to the index.
- The embedding model called out in the README is sentence-transformers/all-MiniLM-L6-v2.
4. Persistent vector store
- The repository includes a local chroma_db/ directory with chroma.sqlite3, which indicates ChromaDB is used as the persistent vector database.
- This is the main durable storage boundary in the architecture: once indexed, the chunks and embeddings are reused across sessions instead of recomputed every time.
5. Retrieval subsystem
- src/retriever.py is the semantic search layer.
- It takes a user query, embeds it with the same embedding model family, and performs top-k similarity search against ChromaDB.
- Its output is a ranked list of relevant text chunks, which becomes the context for generation.
6. Generation / LLM subsystem
- src/llm_handler.py is responsible for calling the answer model and assembling the final response.
- The README identifies the QA model as Zephyr 7B Beta.
- Architecturally, this module is the boundary where retrieved context is turned into a prompt and sent to the LLM.
7. RAG orchestration path
- The core serving flow is: user question → retriever → retrieved chunks → llm_handler → final answer.
- This is the central data flow to capture in a diagram, because it connects the vector store and LLM into one pipeline.
8. UI and interaction layer
- app.py provides a Streamlit interface for interactive use.
- It likely wraps the same backend pipeline used by the CLI, but exposes it through a browser-based chat/query experience.
- The demo/ video suggests this is the main user-facing experience.
9. CLI and non-UI entrypoint
- cli.py provides a headless way to run the same RAG logic.
- In an architecture diagram, it should be shown as an alternate client path into the same indexing/retrieval/generation backend, not as a separate system.
10. Evaluation subsystem
- src/evaluator.py is an auxiliary analysis component.
- It is likely used for offline assessment of retrieval quality or answer quality, rather than being part of the live query path.
- This makes it an experimental/supporting module, not a runtime dependency of the user-facing app.
11. Shared utilities and configuration
- src/utils.py contains common helpers used across modules, likely for text handling, file operations, or prompt/chunk utilities.
- config.yaml is the central configuration boundary for settings such as model names, chunk sizes, retrieval parameters, and database paths.
12. Overall architecture to draw
- The repo is best diagrammed as a single-machine RAG stack:
- source document(s) in data/
- indexing pipeline in src/indexer.py
- persistent Chroma vector store in chroma_db/
- retrieval in src/retriever.py
- answer generation in src/llm_handler.py
- presentation via app.py or cli.py
- evaluation via src/evaluator.py
- There are no obvious external web services in the repo; the key infrastructure is local Python code plus local persistent storage.
Main entry points: retriever.py Retriever · llm_handler.py LLM handler · indexer.py Indexer · utils.py Utils
Components 11
Entry points
User-facing runtime entrypoints
- Streamlit app · Interactive browser UI for asking questions
app.py - CLI · Headless command-line access to the same RAG flow
cli.py
RAG core
Indexing, retrieval, and generation pipeline
- Indexer · Loads the PDF, chunks text, and builds embeddings/index entries
src/indexer.py - Retriever · Embeds queries and runs similarity search over the vector store
src/retriever.py - LLM handler · Builds prompts from retrieved context and calls the answer model
src/llm_handler.py
Storage
Persistent local data stores
- Chroma DB · Persistent local vector database for chunks and embeddings
chroma_db - Chroma SQLite · On-disk Chroma metadata and index storage
chroma_db/chroma.sqlite3
Support
Shared config and offline tooling
- Evaluator · Offline evaluation of retrieval or answer quality
src/evaluator.py - Utils · Shared helper functions used by the core modules
src/utils.py - Config · Central configuration for models, chunking, retrieval, and paths
config.yaml - Corpus PDF · Primary knowledge source indexed by the pipeline
data/document1.pdf
Connections 17
- Streamlit app → Retriever: query · Interactive questions are routed into semantic retrieval
- CLI → Retriever: query · CLI requests use the same retrieval path
- Indexer → Corpus PDF: reads · Document ingestion starts from the source PDF
- Indexer → Config: uses · Chunking and model settings come from config
- Indexer → Chroma DB: writes · Indexed chunks and embeddings are persisted locally
- Chroma DB → Chroma SQLite: persists · Chroma uses the local SQLite backing store
- Retriever → Chroma DB: searches · Similarity search runs against the persisted vector store
- Retriever → Config: uses · Retrieval parameters and embedding settings are configured centrally
- Retriever → LLM handler: passes context · Ranked chunks become prompt context for generation
- LLM handler → Config: uses · Model and prompt settings are driven by config
- Streamlit app → LLM handler: answer · The UI surfaces generated responses to the user
- CLI → LLM handler: answer · The CLI prints generated responses to stdout
- Evaluator → Retriever: checks · Offline evaluation can inspect retrieval behavior
- Evaluator → LLM handler: checks · Offline evaluation can also assess generated answers
- Utils → Indexer: supports · Shared helpers are reused during ingestion
- Utils → Retriever: supports · Shared helpers are reused during retrieval
- Utils → LLM handler: supports · Shared helpers are reused during prompt/response handling