# opea-project/genaiexamples architecture

> Architecture diagram of the GitHub repository opea-project/genaiexamples, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/opea-project/genaiexamples
- Repository: https://github.com/opea-project/genaiexamples

## Overview

##### Project type and purpose
- GenAIExamples is an open-source, microservice-based sample suite showcasing Generative AI use cases (e.g., ChatQnA, DocSum, CodeGen).
- It leverages GenAIComps (collection of LLM, embedding, retrieval, reranking microservices), GenAIInfra (deployment connectors for Docker/Kubernetes), and GenAIEval (benchmarking).

##### File structure
- Top-level folders: one per sample (ChatQnA, AudioQnA, DocSum, etc.). Each contains:
- Core code (e.g., `chatqna.py`)
- `docker_compose/` (per-hardware-compose files)
- `kubernetes/` (Helm charts + GMC manifests)
- `ui/` (React, Svelte, Vue, or Gradio frontends)
- `tests/`, `benchmark/`, `docker_image_build/`
- Root contains deployment scripts (`deploy.py`, `benchmark.py`), Docker image list, README, and Python requirements.

##### README for insights
- Describes architecture: GenAIComps microservices + GenAIInfra deployment + GenAIEval benchmarking
- Highlights deployment options: Python startup, Docker Compose, Kubernetes (Helm/GMC)
- Lists use cases and points to sample folders for detailed instructions.

##### Main components
a. Core microservices (from GenAIComps):
- LLM Service
- Embedding Service
- Retrieval Service
- Reranking Service
- (Optional) Fine-tuning, Summarization, Translator, etc.
b. Pipeline orchestrator
- In-process orchestrator or external GMC connector
c. API Gateway / Load Balancer
- Exposes a unified REST/WebSocket API to clients
d. Frontends (per sample UI)
- React/Svelte/Vue/Gradio clients
e. Data stores & external systems
- Vector DB (Milvus, Qdrant, Pinecone)
- Relational DB or Redis (for session state or caching)
- Object storage (for assets, audio/video)
f. Deployment & infrastructure
- Docker Compose network or Kubernetes cluster
- Helm charts or GMC microservices connector
- CI/CD workflows (GitHub Actions)

##### Relationships and interactions (data flow)
1. User → Frontend UI (React/Svelte/Gradio)
2. Frontend → API Gateway / Orchestrator
3. Orchestrator dispatches to:
- Retrieval Service (query vector store)
- LLM Service (generate response)
- Optional Reranking Service
4. Microservices read/write to vector DB, cache in Redis, store logs/metrics
5. Response flows back through orchestrator → Frontend → User
6. Benchmarking/monitoring via GenAIEval (metrics exported to Prometheus/Grafana)

##### Architectural patterns & design principles
- Microservices: each GenAI capability is its own Docker image + service
- Polyglot UI: multiple frontends can coexist behind the same API
- Infrastructure as code: Docker Compose + Helm charts + GMC manifests
- Clear separation of concerns: compute services (LLM, embedding) vs. storage services (vector DB) vs. UI
- Horizontal scalability: services can be replicated in Kubernetes

##### Elements
- Rectangles (microservices)
- Hexagons (API gateway)
- Cloud outline (Kubernetes cluster) vs. local Docker Compose block

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User and Frontend
    User(["User"]):::user
    subgraph "Frontend Layer"
        direction TB
        ReactUI["ChatQnA React UI"]:::frontend
        SvelteUI["ChatQnA Svelte UI"]:::frontend
        GradioUI["DocSum Gradio UI"]:::frontend
    end

    %% API Gateway / Orchestrator
    Gateway["API Gateway / Orchestrator"]:::gateway

    %% Deployment Boundary
    subgraph "Deployment Boundary (Docker Compose / Kubernetes)" 
        direction TB
        GenAIComps["GenAIComps Base Image"]:::service
        LLM["LLM Service"]:::service
        Embedding["Embedding Service"]:::service
        Retrieval["Retrieval Service"]:::service
        Reranking["Reranking Service"]:::service
        Summarizer["Summarization Service (optional)"]:::service
        Translator["Translator Service (optional)"]:::service
    end

    %% Datastores & Monitoring
    subgraph "Data & Monitoring"
        direction TB
        VectorDB["Vector DB (Milvus)"]:::db
        Redis["Redis Cache"]:::db
        ObjectStore["Object Storage"]:::infra
        Prometheus["Prometheus"]:::monitor
        Grafana["Grafana"]:::monitor
        Benchmark["Benchmarking / GenAIEval"]:::eval
    end

    %% Deployment Options & CI/CD
    subgraph "Infrastructure & CI/CD"
        direction TB
        DockerCompose["Docker Compose"]:::infra
        Kubernetes["Kubernetes (Helm/GMC)"]:::infra
        CICD["CI/CD (GitHub Actions)"]:::ci
        ImageList["Docker Image List"]:::ci
    end

    %% Connections
    User -->|"interacts with"| ReactUI
    User -->|"interacts with"| SvelteUI
    User -->|"interacts with"| GradioUI
    ReactUI -->|"API calls"| Gateway
    SvelteUI -->|"API calls"| Gateway
    GradioUI -->|"API calls"| Gateway

    Gateway -->|"dispatch"| LLM
    Gateway -->|"dispatch"| Embedding
    Gateway -->|"dispatch"| Retrieval
    Gateway -->|"dispatch"| Reranking
    Gateway -->|"dispatch"| Summarizer
    Gateway -->|"dispatch"| Translator

    Retrieval -->|"query"| VectorDB
    Embedding -->|"vectorize"| VectorDB
    LLM -->|"cache"| Redis
    Reranking -->|"cache"| Redis

    GenAIComps -->|build base| LLM
    GenAIComps -->|build base| Embedding
    GenAIComps -->|build base| Retrieval
    GenAIComps -->|build base| Reranking

    VectorDB -->|"store logs"| Prometheus
    Redis -->|"metrics"| Prometheus
    Prometheus -->|"visualize"| Grafana
    Benchmark -->|"collect metrics"| Prometheus

    %% Deployment & CI/CD links
    DockerCompose -->|"deploy services"| Gateway
    Kubernetes -->|"deploy services"| Gateway
    CICD -->|"triggers builds"| GenAIComps
    CICD -->|"push images"| DockerCompose
    CICD -->|"push images"| Kubernetes
    ImageList -->|"defines images"| CICD

    %% Click Events
    click GenAIComps "https://github.com/opea-project/genaiexamples/blob/main/.github/workflows/_build_comps_base_image.yml"
    click Gateway "https://github.com/opea-project/genaiexamples/blob/main/deploy.py"
    click Benchmark "https://github.com/opea-project/genaiexamples/blob/main/benchmark.py"
    click DockerCompose "https://github.com/opea-project/genaiexamples/blob/main/ChatQnA/docker_compose/amd/gpu/rocm/compose.yaml"
    click Kubernetes "https://github.com/opea-project/genaiexamples/blob/main/ChatQnA/kubernetes/helm/cpu-values.yaml"
    click ReactUI "https://github.com/opea-project/genaiexamples/tree/main/ChatQnA/ui/react/"
    click SvelteUI "https://github.com/opea-project/genaiexamples/tree/main/ChatQnA/ui/svelte/"
    click GradioUI "https://github.com/opea-project/genaiexamples/blob/main/DocSum/ui/gradio/docsum_ui_gradio.py"
    click VectorDB "https://github.com/opea-project/genaiexamples/blob/main/ChatQnA/docker_compose/intel/cpu/xeon/compose_milvus.yaml"
    click Redis "https://github.com/opea-project/genaiexamples/blob/main/ChatQnA/docker_compose/intel/cpu/xeon/compose.telemetry.yaml"
    click Prometheus "https://github.com/opea-project/genaiexamples/blob/main/ChatQnA/docker_compose/intel/cpu/xeon/prometheus.yaml"
    click Grafana "https://github.com/opea-project/genaiexamples/tree/main/ChatQnA/docker_compose/intel/cpu/xeon/grafana/provisioning/"
    click CICD "https://github.com/opea-project/genaiexamples/tree/main/.github/workflows/"
    click ImageList "https://github.com/opea-project/genaiexamples/blob/main/docker_images_list.md"

    %% Styles
    classDef user fill:#FFE599,stroke:#333,stroke-width:1px;
    classDef frontend fill:#D9EAD3,stroke:#333,stroke-width:1px;
    classDef gateway fill:#B4A7D6,stroke:#333,stroke-width:1px;
    classDef service fill:#F4CCCC,stroke:#333,stroke-width:1px;
    classDef db fill:#C9DAF8,stroke:#333,stroke-width:1px;
    classDef infra fill:#FFF2CC,stroke:#333,stroke-width:1px;
    classDef monitor fill:#EAD1DC,stroke:#333,stroke-width:1px;
    classDef eval fill:#D0E0E3,stroke:#333,stroke-width:1px;
    classDef ci fill:#EAD1DC,stroke:#333,stroke-width:1px;
```
