How sankethsj/phi3-rag-application works
Project Type and Purpose
• The project is a Retrieval-Augmented Generation (RAG) application. It reads PDF documents, generates embeddings (using the Alibaba-NLP/gte-large-en-v1.5 model), stores them in ChromaDB, and then leverages these embeddings during query processing through an LLM layer to provide contextual responses.
• It is a full-stack tool in a sense—but it’s focused on data processing (PDF ingestion and embedding) and AI-based retrieval augmented generation rather than a typical web frontend/backend split.
File Structure
• Key project entry point: “main.py” which likely orchestrates the workflow.
• Notebooks: “RAG-Workbook.ipynb” for demonstration or interactive data processing steps.
• Code organization: The “rag” directory contains core modules:
• document_loader.py: Responsible for reading and extracting text from PDF documents.
• keyword_generator.py: Likely handles processing of text (e.g., generating keywords or assisting with feature extraction).
• db.py: Encapsulates interactions with ChromaDB for storing and retrieving embeddings.
• llm.py: Interfaces with the Phi3 (and maybe Ollama for local LLM execution) to generate responses based on retrieved information.
• Supporting files: requirements.txt (dependencies), LICENSE, and docs (with a project report that may provide further context).
Extract Insights from the README
• The README details the functionality:
• PDF reading and extraction.
• Embedding generation using an NLP model.
• Storage of embeddings in ChromaDB.
• Retrieval of context and generation (augmented generation).
a. Main Components:
• Entry Point: main.py
• PDF Document Ingestion: Module “document_loader.py”
• Embedding Generation & Keyword Extraction: Partly in “keyword_generator.py” (and possibly integrated with document_loader.py)
• Database Storage: Module “db.py” managing ChromaDB integration.
• Retrieval and Generation: Module “llm.py” which interacts with the local LLM (via Ollama) to generate responses.
b. Visualizing Relationships and Data Flow:
• The process begins with PDF ingestion (document_loader.py) that extracts text.
• The extracted text is sent into the Embedding Generation module (supported by keyword generation if needed).
• Generated embeddings are passed into the database storage (db.py interaction with ChromaDB).
• When a query comes in, the system retrieves relevant embeddings from the database.
• Retrieved contextual data is then sent to the LLM (llm.py) to produce an augmented generation response.
c. Architectural Patterns and Design Principles:
• Modular separation: Each core functionality is isolated in its own module.
• Clean separation of concerns: Document processing, embedding storage, and natural language processing components are distinct.
• Data flow is linear – from PDF ingestion/processing, to embedding storage, then to retrieval and generation.
d. Relevant Technologies, Frameworks, and Libraries:
• Alibaba-NLP/gte-large-en-v1.5 for embeddings.
• ChromaDB for storing and retrieving embeddings.
• Ollama (as per README) for running the local LLM.
• Python’s ecosystem (suggested by the project structure and use of virtual environments).