How oeg-upm/inesdata-ai-services works
Main Components
• Kubernetes Cluster Infrastructure: Start with the underlying Kubernetes cluster, specifying the three master nodes, 10 worker nodes, and 2 LEN nodes (with associated CPU, RAM, and GPU resources).
• Kubeflow Core Components: Represent the primary Kubeflow applications as the backbone of the platform. This includes:
• Central Dashboard (kubeflow/apps/centraldashboard)
• Jupyter Components (jupyter web app and notebook-controller)
• Pipeline Components (kubeflow/apps/pipeline including subcomponents like metadata, cache, API services, UI, etc.)
• Profiles (kubeflow/apps/profiles)
• Additional UI services such as Volumes Web App (kubeflow/apps/volumes-web-app) and PVCViewer Controller (kubeflow/apps/pvcviewer-controller)
• Admission Webhook (for cluster admission operations)
• Common Infrastructure & Third-Party Dependencies:
• Istio configuration (in kubeflow/common/istio and overlays in various app directories)
• Cert-Manager and Keycloak for authentication/security (kubeflow/common/cert-manager, common/keycloak)
• Additional components like Knative or OIDC clients, indicating areas for event-driven behavior or authentication services.
By Function and Interaction
• Frontend/UI Layer: The central dashboard, jupyter-web-app, volumes-web-app and associated overlays that expose a user-friendly interface.
• Backend Service Layer: Pipeline API, Notebook controller, Profile manager and supporting components that process requests, interact with the database (if integrated), and coordinate jobs.
• Admission and Security: Admission webhook components, cert-manager, and keycloak can be drawn as separate modules that secure the platform.
• Integration Layer: The overlays that integrate with Istio, Knative, and other networking pieces to control traffic flow (with virtual services, destination rules, and authorization policies).
Architectural Patterns and Technologies
• That the project employs a layered architecture with clear separations between “base” definitions and “overlays” for customization. Each component is managed declaratively via Kubernetes manifests and kustomize.
• How Istio is used for service mesh and secure ingress/egress, with overlays (in apps like centraldashboard, jupyter, volumes-web-app) containing virtual service configurations and authorization policies.
• Extensibility of the platform by showing the relationships between the core Kubeflow components and their external integrations (for example, how the pipeline components interface with underlying third-party services such as Minio, MySQL, and Argo Workflows).