# mrzaizai2k/llm-with-rag architecture

> Architecture diagram of the GitHub repository mrzaizai2k/llm-with-rag, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mrzaizai2k/llm-with-rag
- Repository: https://github.com/mrzaizai2k/llm-with-rag

## Overview

Step 1 – Identify the Project Type and Purpose
- This is a specialized backend service for building an LLM-powered question-answering system using Retrieval-Augmented Generation (RAG).
- It provides two main REST endpoints:
– POST /update – ingests documents, extracts text, computes or fine-tunes embeddings, and upserts vectors into a vector database.
– POST /query  – takes a user question, retrieves similar passages from the vector DB, runs a routing model if configured, and calls an LLM (OpenAI) to generate an answer.

Step 2 – Analyze the File Structure
- Top-level orchestration:
– Dockerfile, compose.yaml, Makefile, .github/workflows (CI/CD)
- Configuration:
– config/\*.yaml & custom_prompt_template.txt – define model parameters, data-pipeline settings, routing training configs.
- Source code (src/):
– Utils/ – logging & shared helper routines
– ingest.py & push_dataset.py – document ingestion and vector DB upsert
– data_processing.py, py_ocr.py – text extraction from PDFs/PPTs
– finetune_embedding.py – embedding fine-tuning with SentenceTransformer
– train_gpt_routing.py – trains a small GPT-based routing model to pick the best response strategy
– ragqa.py – the core RAG pipeline that retrieves, assembles context, and invokes the LLM
– api.py & test_api.py – REST interface and its tests
- Notebooks: experimental workflows for data prep, embedding tuning, reranking, routing, DB updates.

Step 3 – Extract Architectural Insights from the README
- The system is containerized (Docker + Compose) for easy deployment.
- It uses an external LLM provider (OpenAI via OPENAI_API_KEY).
- It leverages a vector DB (FAISS or other) to store/retrieve embeddings.
- CI/CD is set up via GitHub Actions.

Step 4 – Main Components & Their Interactions
##### Data Sources
– Raw documents (PDF, PPT, etc.) stored in data/ or mounted volume.
##### Ingestion Pipeline (batch or on-demand)
– OCR/Text-Extraction (py_ocr.py) → Data Cleaning (data_processing.py) → Embedding Computation/Fine-tuning (finetune_embedding.py) → Vector Upsert (push_dataset.py)
– Config driven by config/data_config.yaml & config/finetune_embedding.yaml
##### Vector Database
– Stores document embeddings and metadata, allows k-NN searches
##### Routing Model (Optional)
– train_gpt_routing.py produces a small “router” that selects best retrieval-vs-generation strategy
##### Query Service (api.py)
– Exposes POST /query → Vector DB search → (Router) → LLM call (ragqa.py + OpenAI) → Returns answer JSON
##### CI/CD Pipeline
– .github/workflows/workloads.yaml – runs tests and builds Docker images
##### Deployment
– Dockerfile for single‐container; compose.yaml for multi‐service (vector DB, service, optional UI)

Step 5 – Architectural Patterns & Design Principles
- Pipeline Pattern – clear separation between ingestion, training, and serving
- Microservice Style – the API service is single‐responsibility, containerized for scaling
- Config‐Driven – YAML files manage model, data, and training settings

- Layout:
– Left side: Batch Ingestion Pipeline (Data Sources → Text-Extraction → Embeddings → Vector DB)
– Right side: Online API Service (User → API Server → Vector DB → Router/LLM → Response)
– Bottom/Side: Config Store + CI/CD + Deployment blocks
- Notation:
– Rectangles for components (e.g., “API Server”, “Vector DB”, “OCR Service”)
– Blue for data pipelines
– Green for online serving
– Orange for external services (OpenAI)
– Grey for orchestration (Docker/Compose)
- Labels & Details:
– Show config files attached to each pipeline stage
– Indicate notebooks as “Experiments” offshoot
– Mark CI/CD workflow feeding into Docker image build

- That there is no “frontend” UI – it’s a headless API
- Dual nature: offline training/ingestion vs. online query serving
- How GitHub Actions triggers builds and how Docker Compose wires up the service + vector DB

1. Clear component names (Ingestion Pipeline, Vector Store, API Service, Routing Model, LLM Provider)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Ingestion Pipeline (Blue)
    subgraph "Ingestion Pipeline" 
        direction TB
        OCR["OCR/Text Extraction"]:::pipeline
        DataClean["Data Cleaning & Preprocessing"]:::pipeline
        EmbedTune["Embedding Fine-tuning"]:::pipeline
        IngestUpsert["Document Ingestion & Vector Upsert"]:::pipeline
    end

    %% API Service (Green)
    subgraph "Online Query Service"
        direction TB
        User["User"]:::external
        API["API Server\n(POST /query & /update)"]:::serving
        VecSearch["k-NN Search"]:::serving
        Router["Routing Model"]:::serving
        LLMInvoke["LLM Invocation\n(OpenAI)"]:::serving
        Response["Answer JSON"]:::serving
    end

    %% Core Components (Central)
    VectorDB[(Vector DB)]:::pipeline
    ConfigStore["Configuration Store"]:::orchestr
    CI["CI/CD Pipeline"]:::orchestr
    Deploy["Deployment & Orchestration"]:::orchestr
    Experiments["Notebooks & Experiments"]:::external

    %% Ingestion Arrows
    OCR -->|"text output"| DataClean
    DataClean -->|"cleaned text"| EmbedTune
    EmbedTune -->|"embeddings"| IngestUpsert
    IngestUpsert -->|"vectors"| VectorDB

    %% Query Flow
    User -->|"/query"| API
    API -->|"k-NN query"| VecSearch
    VecSearch --> VectorDB
    VectorDB -->|"results"| VecSearch
    VecSearch --> Router
    Router --> LLMInvoke
    LLMInvoke -->|"answer"| Response

    %% Update Flow
    User -->|"/update"| API
    API --> IngestUpsert
    IngestUpsert --> VectorDB

    %% Config & Orchestration
    ConfigStore --> OCR
    ConfigStore --> DataClean
    ConfigStore --> EmbedTune
    ConfigStore --> Router
    CI --> Deploy
    Deploy --> API
    Deploy --> VectorDB
    Experiments -.-> DataClean
    Experiments -.-> EmbedTune

    %% Click Events
    click ConfigStore "https://github.com/mrzaizai2k/llm-with-rag/tree/main/config/"
    click OCR "https://github.com/mrzaizai2k/llm-with-rag/blob/main/src/py_ocr.py"
    click DataClean "https://github.com/mrzaizai2k/llm-with-rag/blob/main/src/data_processing.py"
    click EmbedTune "https://github.com/mrzaizai2k/llm-with-rag/blob/main/src/finetune_embedding.py"
    click IngestUpsert "https://github.com/mrzaizai2k/llm-with-rag/blob/main/src/ingest.py"
    click IngestUpsert "https://github.com/mrzaizai2k/llm-with-rag/blob/main/src/push_dataset.py"
    click Router "https://github.com/mrzaizai2k/llm-with-rag/blob/main/src/train_gpt_routing.py"
    click CI "https://github.com/mrzaizai2k/llm-with-rag/blob/main/.github/workflows/workloads.yaml"
    click API "https://github.com/mrzaizai2k/llm-with-rag/blob/main/src/api.py"
    click Response "https://github.com/mrzaizai2k/llm-with-rag/blob/main/src/test_api.py"
    click LLMInvoke "https://github.com/mrzaizai2k/llm-with-rag/blob/main/src/ragqa.py"
    click Deploy "https://github.com/mrzaizai2k/llm-with-rag/tree/main/Dockerfile"
    click Deploy "https://github.com/mrzaizai2k/llm-with-rag/blob/main/compose.yaml"
    click Deploy "https://github.com/mrzaizai2k/llm-with-rag/tree/main/Makefile"
    click ConfigStore "https://github.com/mrzaizai2k/llm-with-rag/blob/main/config/data_config.yaml"
    click ConfigStore "https://github.com/mrzaizai2k/llm-with-rag/blob/main/config/finetune_embedding.yaml"
    click ConfigStore "https://github.com/mrzaizai2k/llm-with-rag/blob/main/config/gpt_routing_train_config.yaml"
    click ConfigStore "https://github.com/mrzaizai2k/llm-with-rag/blob/main/config/model_config.yaml"
    click Experiments "https://github.com/mrzaizai2k/llm-with-rag/tree/main/src/"

    %% Styles
    classDef pipeline fill:#cce5ff,stroke:#3399ff,color:#000
    classDef serving fill:#d4edda,stroke:#28a745,color:#000
    classDef external fill:#fff3cd,stroke:#ffcd39,color:#000
    classDef orchestr fill:#e2e3e5,stroke:#6c757d,color:#000
```
