# opea-project/genaicomps architecture

> Architecture diagram of the GitHub repository opea-project/genaicomps, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/opea-project/genaicomps
- Repository: https://github.com/opea-project/genaicomps

## Overview

##### Overall architecture
- The project follows a microservice architecture.
- Each microservice (e.g., Embedding, Retriever, ASR, TTS, Dataprep, LLM, Reranking, etc.) is implemented as a containerized component with its own deployment configuration (Docker Compose and Kubernetes manifests are used throughout the “comps” directory).
- There is a ServiceOrchestrator (or “MegaService”) that composes these microservices into a cohesive solution, as shown in the README’s example.
- CI/CD workflows and deployment scripts are extensively set up within the “.github/workflows” area.

##### Main components
a. Front Door / API Gateway:
– The ChatQnAGateway, which receives incoming requests and directs them to the appropriate microservices.
b. Microservices:
– Embedding Service(s): Providing vector embedding functionalities.
– Retriever and Reranking Services: For retrieval-augmented generation, processing queries using LangChain and LlamaIndex integrations.
– ASR and TTS Services: For speech recognition and synthesis.
– Dataprep: Pre-processing and data ingestion pipelines.
– LLM (Chat and Text generation) Services: Depending on deployment, supporting inference engines like TGI, vLLM or Ray Serve.
– Other specialized components (e.g., agent, animation, chathistory, guardrails) as needed.
c. Service Orchestrator / MegaService:
– This component ties individual microservices together, creating customized workflow flows (e.g., connecting an embedding service to an LLM service).
d. Infrastructure Components:
– Container orchestration platforms (Docker, Kubernetes).
– Configuration management (YAML configuration files, Helm charts in the “kubernetes” directories).
– CI/CD pipelines for builds, tests, and deployments.
– Possible integration with external services (Redis, Qdrant, Milvus, etc.) based on the microservice choices.
e. Testing and Deployment:
– Include test cases and deployment workflows (as seen in the “tests” and “.github/workflows” directories).

##### Diagramming Relationships and Data Flow
– Name of the microservice (e.g., “Embedding on Gaudi2”, “ASR on Xeon CPU”, “LLM via Ray Serve”).
– Relevant technology or frameworks (e.g., LangChain, LlamaIndex, specific container images).
- API Gateway / ChatQnAGateway as the entry point.
– From the gateway to the microservices.
– From one microservice to another when composing a workflow.
– From microservices back to the gateway for response aggregation.

##### Architectural Patterns and Technologies
- Modularity and service isolation: Each component is built (and deployed) independently with its own Dockerfiles, Kubernetes manifests, and CI configurations.
- Container orchestration aspects by indicating that deployment artifacts exist in production-ready formats (Docker Compose and Kubernetes).
- Point out the usage of service decorators (such as the “register_microservice” decorator) to abstract microservice registration.

##### Final Advice
– Review the extensive directory structure in “comps/” to find all individual microservices; you might group them into categories if there are many.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% API Layer
    subgraph "API Layer"
        direction TB
        ChatQnAGateway("ChatQnAGateway"):::api
    end

    %% Orchestration Layer
    subgraph "Orchestration Layer"
        direction TB
        ServiceOrchestrator("Service Orchestrator / MegaService"):::orchestration
    end

    %% Microservices Layer
    subgraph "Microservices Layer"
        direction TB
        %% NLP Services Cluster
        subgraph "NLP Services"
            direction TB
            Embedding("Embedding Service"):::microservice
            Retriever("Retriever Service"):::microservice
            Reranking("Reranking Service"):::microservice
            LLM("LLM Services"):::microservice
        end
        %% Speech Services Cluster
        subgraph "Speech Services"
            direction TB
            ASR("ASR Service"):::microservice
            TTS("TTS Service"):::microservice
        end
        %% Data & Specialized Services Cluster
        subgraph "Data & Specialized Services"
            direction TB
            Dataprep("Dataprep Service"):::microservice
            Agent("Agent Service"):::microservice
            Animation("Animation Service"):::microservice
            Chathistory("Chathistory Service"):::microservice
            Guardrails("Guardrails Service"):::microservice
        end
    end

    %% CI/CD and Deployment
    subgraph "CI/CD & Deployment"
        direction TB
        CI_CD("CI/CD and Deployment Workflows"):::ci
        OrchestrationInfra("Container Orchestration (Docker/K8s)"):::infra
    end

    %% Data/Control Flow Connections
    ChatQnAGateway -->|"forwards_request"| ServiceOrchestrator

    ServiceOrchestrator -->|"calls"| Embedding
    ServiceOrchestrator -->|"calls"| Retriever
    ServiceOrchestrator -->|"calls"| Reranking
    ServiceOrchestrator -->|"calls"| LLM
    ServiceOrchestrator -->|"calls"| ASR
    ServiceOrchestrator -->|"calls"| TTS
    ServiceOrchestrator -->|"calls"| Dataprep
    ServiceOrchestrator -->|"calls"| Agent
    ServiceOrchestrator -->|"calls"| Animation
    ServiceOrchestrator -->|"calls"| Chathistory
    ServiceOrchestrator -->|"calls"| Guardrails

    Embedding -->|"result"| ServiceOrchestrator
    Retriever -->|"result"| ServiceOrchestrator
    Reranking -->|"result"| ServiceOrchestrator
    LLM -->|"result"| ServiceOrchestrator
    ASR -->|"result"| ServiceOrchestrator
    TTS -->|"result"| ServiceOrchestrator
    Dataprep -->|"result"| ServiceOrchestrator
    Agent -->|"result"| ServiceOrchestrator
    Animation -->|"result"| ServiceOrchestrator
    Chathistory -->|"result"| ServiceOrchestrator
    Guardrails -->|"result"| ServiceOrchestrator

    ServiceOrchestrator -->|"aggregate_response"| ChatQnAGateway

    %% CI/CD and Deployment Flow
    CI_CD -->|"deploys"| OrchestrationInfra
    OrchestrationInfra -->|"manages"| ChatQnAGateway
    OrchestrationInfra -->|"manages"| ServiceOrchestrator
    OrchestrationInfra -->|"manages"| Embedding
    OrchestrationInfra -->|"manages"| Retriever
    OrchestrationInfra -->|"manages"| Reranking
    OrchestrationInfra -->|"manages"| LLM
    OrchestrationInfra -->|"manages"| ASR
    OrchestrationInfra -->|"manages"| TTS
    OrchestrationInfra -->|"manages"| Dataprep
    OrchestrationInfra -->|"manages"| Agent
    OrchestrationInfra -->|"manages"| Animation
    OrchestrationInfra -->|"manages"| Chathistory
    OrchestrationInfra -->|"manages"| Guardrails

    %% Click Events
    click ChatQnAGateway "https://github.com/opea-project/genaicomps/blob/main/tests/cores/mega/ChatQnA_E2E_manifests_base.yaml"
    click Embedding "https://github.com/opea-project/genaicomps/tree/main/comps/embeddings"
    click Retriever "https://github.com/opea-project/genaicomps/tree/main/comps/retrievers"
    click Reranking "https://github.com/opea-project/genaicomps/tree/main/comps/rerankings"
    click ASR "https://github.com/opea-project/genaicomps/tree/main/comps/asr"
    click TTS "https://github.com/opea-project/genaicomps/tree/main/comps/tts"
    click Dataprep "https://github.com/opea-project/genaicomps/tree/main/comps/dataprep"
    click LLM "https://github.com/opea-project/genaicomps/tree/main/comps/llms"
    click Agent "https://github.com/opea-project/genaicomps/tree/main/comps/agent"
    click Animation "https://github.com/opea-project/genaicomps/tree/main/comps/animation"
    click Chathistory "https://github.com/opea-project/genaicomps/tree/main/comps/chathistory"
    click Guardrails "https://github.com/opea-project/genaicomps/tree/main/comps/guardrails"
    click ServiceOrchestrator "https://github.com/opea-project/genaicomps/tree/main/comps/cores/mega"
    click CI_CD "https://github.com/opea-project/genaicomps/tree/main/.github/workflows"

    %% Styles
    classDef api fill:#a6cee3,stroke:#1f78b4,stroke-width:2px;
    classDef orchestration fill:#b2df8a,stroke:#33a02c,stroke-width:2px;
    classDef microservice fill:#fb9a99,stroke:#e31a1c,stroke-width:2px;
    classDef ci fill:#fdbf6f,stroke:#ff7f00,stroke-width:2px;
    classDef infra fill:#cab2d6,stroke:#6a3d9a,stroke-width:2px;
```
