# oeg-upm/inesdata-ai-services architecture

> Architecture diagram of the GitHub repository oeg-upm/inesdata-ai-services, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/oeg-upm/inesdata-ai-services
- Repository: https://github.com/oeg-upm/inesdata-ai-services

## Overview

##### Main Components
- Kubernetes Cluster Infrastructure: Start with the underlying Kubernetes cluster, specifying the three master nodes, 10 worker nodes, and 2 LEN nodes (with associated CPU, RAM, and GPU resources).
- Kubeflow Core Components: Represent the primary Kubeflow applications as the backbone of the platform. This includes:
- Central Dashboard (kubeflow/apps/centraldashboard)
- Jupyter Components (jupyter web app and notebook-controller)
- Pipeline Components (kubeflow/apps/pipeline including subcomponents like metadata, cache, API services, UI, etc.)
- Profiles (kubeflow/apps/profiles)
- Additional UI services such as Volumes Web App (kubeflow/apps/volumes-web-app) and PVCViewer Controller (kubeflow/apps/pvcviewer-controller)
- Admission Webhook (for cluster admission operations)
- Common Infrastructure & Third-Party Dependencies:
- Istio configuration (in kubeflow/common/istio and overlays in various app directories)
- Cert-Manager and Keycloak for authentication/security (kubeflow/common/cert-manager, common/keycloak)
- Additional components like Knative or OIDC clients, indicating areas for event-driven behavior or authentication services.

##### By Function and Interaction
- Frontend/UI Layer: The central dashboard, jupyter-web-app, volumes-web-app and associated overlays that expose a user-friendly interface.
- Backend Service Layer: Pipeline API, Notebook controller, Profile manager and supporting components that process requests, interact with the database (if integrated), and coordinate jobs.
- Admission and Security: Admission webhook components, cert-manager, and keycloak can be drawn as separate modules that secure the platform.
- Integration Layer: The overlays that integrate with Istio, Knative, and other networking pieces to control traffic flow (with virtual services, destination rules, and authorization policies).

##### Architectural Patterns and Technologies
- That the project employs a layered architecture with clear separations between “base” definitions and “overlays” for customization. Each component is managed declaratively via Kubernetes manifests and kustomize.
- How Istio is used for service mesh and secure ingress/egress, with overlays (in apps like centraldashboard, jupyter, volumes-web-app) containing virtual service configurations and authorization policies.
- Extensibility of the platform by showing the relationships between the core Kubeflow components and their external integrations (for example, how the pipeline components interface with underlying third-party services such as Minio, MySQL, and Argo Workflows).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External User
    User("User"):::UI

    %% Kubernetes Cluster
    subgraph "Kubernetes Cluster"
        %% Cluster Nodes
        subgraph "Cluster Nodes"
            Masters("Master Nodes: 3 (CPU,RAM)"):::Infra
            Workers("Worker Nodes: 10 (CPU,RAM)"):::Infra
            LEN("LEN Nodes: 2 (GPU)"):::Infra
        end

        %% Kubeflow Core Components
        subgraph "Kubeflow Core Components"
            CD("Central Dashboard"):::UI
            JWA("Jupyter Web App"):::UI
            NC("Notebook Controller"):::Backend
            PC("Pipeline Components"):::Backend
            P("Profiles"):::Backend
            VWA("Volumes Web App"):::UI
            PVC("PVCViewer Controller"):::Backend
            AW("Admission Webhook"):::Security
        end

        %% Infrastructure & Integration
        subgraph "Infrastructure & Integration"
            IST("Istio Service Mesh"):::Infra
            CM("Cert-Manager"):::Security
            KC("Keycloak"):::Security
            KN("Knative"):::Infra
            OIDC("OIDC Client"):::Security
        end

        %% Kubernetes API Server for Admission
        KAS("Kubernetes API Server"):::Infra
    end

    %% User accessing UI components
    User -->|"access"| CD
    User -->|"access"| JWA
    User -->|"access"| VWA

    %% UI to Backend flows
    CD -->|"calls"| PC
    JWA -->|"triggers"| NC
    VWA -->|"calls"| PVC

    %% Backend internal communication
    PC -->|"uses"| P

    %% Admission Webhook flow
    AW -->|"communicates"| KAS

    %% Integration flows via Service Mesh and Security Modules
    IST -->|"routes"| CD
    IST -->|"routes"| JWA
    IST -->|"routes"| VWA
    KN -->|"triggers"| AW
    CM -->|"secures"| AW
    KC -->|"authenticates"| CD
    OIDC -->|"authenticates"| JWA

    %% Click Events
    click CD "https://github.com/oeg-upm/inesdata-ai-services/tree/develop/kubeflow/apps/centraldashboard"
    click JWA "https://github.com/oeg-upm/inesdata-ai-services/tree/develop/kubeflow/apps/jupyter/jupyter-web-app"
    click NC "https://github.com/oeg-upm/inesdata-ai-services/tree/develop/kubeflow/apps/jupyter/notebook-controller"
    click PC "https://github.com/oeg-upm/inesdata-ai-services/tree/develop/kubeflow/apps/pipeline"
    click P "https://github.com/oeg-upm/inesdata-ai-services/tree/develop/kubeflow/apps/profiles"
    click VWA "https://github.com/oeg-upm/inesdata-ai-services/tree/develop/kubeflow/apps/volumes-web-app"
    click PVC "https://github.com/oeg-upm/inesdata-ai-services/tree/develop/kubeflow/apps/pvcviewer-controller"
    click AW "https://github.com/oeg-upm/inesdata-ai-services/tree/develop/kubeflow/apps/admission-webhook"
    click IST "https://github.com/oeg-upm/inesdata-ai-services/tree/develop/kubeflow/common/istio"
    click CM "https://github.com/oeg-upm/inesdata-ai-services/tree/develop/kubeflow/common/cert-manager"
    click KC "https://github.com/oeg-upm/inesdata-ai-services/tree/develop/kubeflow/common/keycloak"
    click KN "https://github.com/oeg-upm/inesdata-ai-services/tree/develop/kubeflow/common/knative"
    click OIDC "https://github.com/oeg-upm/inesdata-ai-services/tree/develop/kubeflow/common/oidc-client"

    %% Styles
    classDef UI fill:#ADD8E6,stroke:#333,stroke-width:2px;
    classDef Backend fill:#90EE90,stroke:#333,stroke-width:2px;
    classDef Infra fill:#FFD700,stroke:#333,stroke-width:2px;
    classDef Security fill:#FFB6C1,stroke:#333,stroke-width:2px;
```
