# duongngyn0510/deploy-highly-scalable-rag-for-vietnamese-law-qa-system architecture

> Architecture diagram of the GitHub repository duongngyn0510/deploy-highly-scalable-rag-for-vietnamese-law-qa-system, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/duongngyn0510/deploy-highly-scalable-rag-for-vietnamese-law-qa-system
- Repository: https://github.com/duongngyn0510/deploy-highly-scalable-rag-for-vietnamese-law-qa-system

## Overview

##### Main components
- Embedding Component – Deployed in its own GKE cluster with Terraform and managed via Helm charts located under the “embedding” directory. This component runs a text-embeddings-inference service that pulls models from Huggingface.
- Weaviate Component – Deployed separately on its own GKE cluster (configured via Terraform and managed by Helm charts under “weaviate”). This serves as the vector store (likely a database and search layer) for storing processed embeddings.
- Reranker and LLM Component – Runs on a dedicated GPU server using Docker Compose (found in the “reranker-llm” directory). This component handles heavy model inference tasks like reranking and language model operations.
- RAG Controller Component – A central FastAPI service (with main.py as the entry point) that orchestrates communication between the embedding, retrieval, and LLM services. It is deployed in a GKE cluster (with its own Helm chart and Terraform scripts in “rag-controller”) and also contains various subcomponents (chat, completions, translation, vector store, etc.) within its “src” folder.
- Monitoring & CI/CD – Monitoring is implemented using Prometheus (with Helm charts under “monitoring” and Grafana configurations) and there is a CI/CD pipeline indicated by the Jenkinsfile and Ansible playbooks under “gce”. These show integration with provisioning and deployment tools.

##### Relationships and interactions between components
- Frontend/API gateway (RAG Controller) interfaces with backend processes and routes traffic. It also contains UI elements (referenced in the README’s UI image) and communicates with the other core service components.
- The RAG Controller interacts with the Embedding Component to obtain text embeddings and with the Weaviate Component to perform vector store queries.
- The Reranker and LLM component (running on the GPU server) likely receives requests from the RAG Controller, providing enhanced language model responses or re-ranking results.
- Monitoring components observe the state of each GKE cluster (via Prometheus installed on each cluster with a remote-write pattern) and feed dashboards in Grafana.
- CI/CD components (Jenkins, Ansible, Terraform, Helm) ensure automated deployment and updates across these clusters.

##### Architectural patterns and technologies
- Cloud Native / Microservices Architecture: Components are split into distinct Kubernetes clusters or Docker deployments for service isolation and scalability.
- Infrastructure as Code: Terraform scripts are used to provision the clusters, and Helm charts manage deployments for Embedding, Weaviate, and RAG Controller components.
- Containerization: Docker is used across the board (including local development via Docker Compose and deployment with Dockerfiles).
- Automation: CI/CD pipelines leverage Jenkins and Ansible for configuration, testing, and deployments.
- Monitoring: A dedicated monitoring stack (Prometheus + Grafana) collects telemetry from multiple clusters, with remote-write patterns to centralize data.

##### Need for detail and clarity
- Do not shy away from breaking down the architecture into as many components as you feel necessary. Given the complexity (multiple helm charts, Terraform scripts, several CI/CD pipelines, and diverse service groups), the goal is to ensure that every critical interaction and dependency is visible.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Infrastructure Layer: GKE Clusters and GPU Server
    subgraph "Embedding Cluster (GKE)"
        ES["Embedding Service\n(Text-embeddings-inference)"]:::app
        EH["Embedding Helm Charts"]:::config
        ET["Embedding Terraform Scripts"]:::config
    end

    subgraph "Weaviate Cluster (GKE)"
        WS["Weaviate Vector Store"]:::db
        WH["Weaviate Helm Charts"]:::config
        WT["Weaviate Terraform Scripts"]:::config
    end

    subgraph "RAG Controller Cluster (GKE)"
        RC["RAG Controller API\n(FastAPI Service)"]:::app
        RCH["RAG Helm Charts"]:::config
        RCT["RAG Terraform Scripts"]:::config
        RS["RAG Subcomponents\n(chat, completions, translation, vector store, etc.)"]:::app
    end

    subgraph "GPU Server"
        RL["Reranker & LLM Service\n(Docker Compose on GPU)"]:::app
        RD["Docker Compose Configuration"]:::config
    end

    %% Integration & Support Layer
    subgraph "Integration & Support"
        MON["Monitoring\n(Prometheus + Grafana)"]:::integration
        CICD["CI/CD Pipeline\n(Jenkins + Ansible)"]:::integration
    end

    %% External API Example
    TP["ThirdParty LLM API"]:::external

    %% Service Interactions
    RC -->|"embedding_API_call"| ES
    RC -->|"vector_query"| WS
    RC -->|"inference_request"| RL
    RC -->|"subcomponent_call"| RS
    RC -->|"auto_redirect"| TP

    %% CI/CD triggering deployments to configuration components
    CICD -->|"deploys"| EH
    CICD -->|"deploys"| ET
    CICD -->|"deploys"| WH
    CICD -->|"deploys"| WT
    CICD -->|"deploys"| RCH
    CICD -->|"deploys"| RCT
    CICD -->|"deploys"| RD

    %% Monitoring flow from services
    ES -->|"metrics"| MON
    WS -->|"metrics"| MON
    RC -->|"metrics"| MON
    RL -->|"metrics"| MON
    RS -->|"metrics"| MON

    %% Click Events for Component Mapping
    click EH "https://github.com/duongngyn0510/deploy-highly-scalable-rag-for-vietnamese-law-qa-system/tree/master/embedding/helm-charts/emb"
    click ET "https://github.com/duongngyn0510/deploy-highly-scalable-rag-for-vietnamese-law-qa-system/tree/master/embedding/terraform"
    click WH "https://github.com/duongngyn0510/deploy-highly-scalable-rag-for-vietnamese-law-qa-system/tree/master/weaviate/helm-charts/weaviate"
    click WT "https://github.com/duongngyn0510/deploy-highly-scalable-rag-for-vietnamese-law-qa-system/tree/master/weaviate/terraform"
    click RC "https://github.com/duongngyn0510/deploy-highly-scalable-rag-for-vietnamese-law-qa-system/blob/master/rag-controller/main.py"
    click RCH "https://github.com/duongngyn0510/deploy-highly-scalable-rag-for-vietnamese-law-qa-system/tree/master/rag-controller/helm-charts/rag"
    click RCT "https://github.com/duongngyn0510/deploy-highly-scalable-rag-for-vietnamese-law-qa-system/tree/master/rag-controller/terraform"
    click RS "https://github.com/duongngyn0510/deploy-highly-scalable-rag-for-vietnamese-law-qa-system/tree/master/rag-controller/src"
    click RD "https://github.com/duongngyn0510/deploy-highly-scalable-rag-for-vietnamese-law-qa-system/blob/master/reranker-llm/docker-compose.yml"
    click MON "https://github.com/duongngyn0510/deploy-highly-scalable-rag-for-vietnamese-law-qa-system/tree/master/monitoring/helm-charts/prometheus"
    click CICD "https://github.com/duongngyn0510/deploy-highly-scalable-rag-for-vietnamese-law-qa-system/tree/master/gce/ansible"

    %% Styles
    classDef app fill:#CFF1BF,stroke:#333,stroke-width:2px;
    classDef config fill:#E8E8E8,stroke:#333,stroke-width:2px,stroke-dasharray: 5 5;
    classDef integration fill:#FFF3BF,stroke:#333,stroke-width:2px;
    classDef external fill:#F1C0C0,stroke:#333,stroke-width:2px;
    classDef db fill:#BFD7EA,stroke:#333,stroke-width:2px;
```
