How mrzaizai2k/llm-with-rag works
Step 1 – Identify the Project Type and Purpose
• This is a specialized backend service for building an LLM-powered question-answering system using Retrieval-Augmented Generation (RAG).
• It provides two main REST endpoints:
– POST /update – ingests documents, extracts text, computes or fine-tunes embeddings, and upserts vectors into a vector database.
– POST /query – takes a user question, retrieves similar passages from the vector DB, runs a routing model if configured, and calls an LLM (OpenAI) to generate an answer.
Step 2 – Analyze the File Structure
• Top-level orchestration:
– Dockerfile, compose.yaml, Makefile, .github/workflows (CI/CD)
• Configuration:
– config/\*.yaml & custom_prompt_template.txt – define model parameters, data-pipeline settings, routing training configs.
• Source code (src/):
– Utils/ – logging & shared helper routines
– ingest.py & push_dataset.py – document ingestion and vector DB upsert
– data_processing.py, py_ocr.py – text extraction from PDFs/PPTs
– finetune_embedding.py – embedding fine-tuning with SentenceTransformer
– train_gpt_routing.py – trains a small GPT-based routing model to pick the best response strategy
– ragqa.py – the core RAG pipeline that retrieves, assembles context, and invokes the LLM
– api.py & test_api.py – REST interface and its tests
• Notebooks: experimental workflows for data prep, embedding tuning, reranking, routing, DB updates.
Step 3 – Extract Architectural Insights from the README
• The system is containerized (Docker + Compose) for easy deployment.
• It uses an external LLM provider (OpenAI via OPENAI_API_KEY).
• It leverages a vector DB (FAISS or other) to store/retrieve embeddings.
• CI/CD is set up via GitHub Actions.
Step 4 – Main Components & Their Interactions
Data Sources
– Raw documents (PDF, PPT, etc.) stored in data/ or mounted volume.
Ingestion Pipeline (batch or on-demand)
– OCR/Text-Extraction (py_ocr.py) → Data Cleaning (data_processing.py) → Embedding Computation/Fine-tuning (finetune_embedding.py) → Vector Upsert (push_dataset.py)
– Config driven by config/data_config.yaml & config/finetune_embedding.yaml
Vector Database
– Stores document embeddings and metadata, allows k-NN searches
Routing Model (Optional)
– train_gpt_routing.py produces a small “router” that selects best retrieval-vs-generation strategy
Query Service (api.py)
– Exposes POST /query → Vector DB search → (Router) → LLM call (ragqa.py + OpenAI) → Returns answer JSON
CI/CD Pipeline
– .github/workflows/workloads.yaml – runs tests and builds Docker images
Deployment
– Dockerfile for single‐container; compose.yaml for multi‐service (vector DB, service, optional UI)
Step 5 – Architectural Patterns & Design Principles
• Pipeline Pattern – clear separation between ingestion, training, and serving
• Microservice Style – the API service is single‐responsibility, containerized for scaling
• Config‐Driven – YAML files manage model, data, and training settings
• Layout:
– Left side: Batch Ingestion Pipeline (Data Sources → Text-Extraction → Embeddings → Vector DB)
– Right side: Online API Service (User → API Server → Vector DB → Router/LLM → Response)
– Bottom/Side: Config Store + CI/CD + Deployment blocks
• Notation:
– Rectangles for components (e.g., “API Server”, “Vector DB”, “OCR Service”)
– Blue for data pipelines
– Green for online serving
– Orange for external services (OpenAI)
– Grey for orchestration (Docker/Compose)
• Labels & Details:
– Show config files attached to each pipeline stage
– Indicate notebooks as “Experiments” offshoot
– Mark CI/CD workflow feeding into Docker image build
• That there is no “frontend” UI – it’s a headless API
• Dual nature: offline training/ingestion vs. online query serving
• How GitHub Actions triggers builds and how Docker Compose wires up the service + vector DB
1. Clear component names (Ingestion Pipeline, Vector Store, API Service, Routing Model, LLM Provider)