# opendatahub-io/kubeflow-sdk architecture

> Architecture diagram of the GitHub repository opendatahub-io/kubeflow-sdk, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/opendatahub-io/kubeflow-sdk
- Repository: https://github.com/opendatahub-io/kubeflow-sdk

## Overview

##### Project type and purpose
- Kubeflow SDK is an open-source Python library (SDK) that provides AI practitioners with a unified, high-level API to launch and manage training workloads on Kubeflow (Kubernetes) or locally.
- It abstracts the complexity of Kubernetes, offers consistent Python interfaces, and supports extensibility for new “backends” (runtimes).

##### File structure
- Top-level:
- `kubeflow/` – the main Python package
- `kubeflow/trainer/api/` – public client APIs (`TrainerClient`, `CustomTrainer`)
- `kubeflow/trainer/backends/` – pluggable runtimes:
– `kubernetes/` backend talks to the Kubernetes API
– `localprocess/` backend spins up local OS processes
- `kubeflow/trainer/constants/`, `types/`, `utils/` – shared constants, data models, and helper functions
- `tests/` alongside each module – unit tests
- CI/CD and release: GitHub workflows, `Makefile`, `scripts/`, `pyproject.toml`

##### README for additional insights
- Confirms core components: TrainerClient, CustomTrainer, built-in backends (Kubeflow Trainer, Katib planned).
- Use cases: local development, distributed PyTorch jobs, future support for hyperparameter optimization and pipelines.

##### Final recommendation
- Be exhaustive in capturing each major module and its dependencies.
- Keep the user’s perspective front-and-center: show the simple API entry point and how it propagates down to actual execution.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% User layer
    User["User Code / Notebook / CLI"]:::external

    %% SDK Layer
    subgraph "Kubeflow SDK"
        direction TB
        %% API Layer
        subgraph "API Layer"
            direction TB
            TrainerClient["TrainerClient"]:::sdk
            CustomTrainer["CustomTrainer"]:::sdk
        end

        %% Backend Abstraction
        subgraph "Backend Abstraction"
            direction TB
            BaseBackend["BaseBackend Interface"]:::sdk
        end

        %% Built-in Backends
        subgraph "Built-in Backends"
            direction TB
            subgraph "Kubernetes Backend"
                direction TB
                K8sBackend["Kubernetes Backend"]:::sdk
                K8sTypes["Kubernetes Backend Types"]:::sdk
            end
            subgraph "LocalProcess Backend"
                direction TB
                LPBackend["LocalProcess Backend"]:::sdk
                LPConstants["LocalProcess Constants"]:::sdk
                LPJob["LocalProcess Job Launcher"]:::sdk
                LPTypes["LocalProcess Types"]:::sdk
                LPUtils["LocalProcess Utilities"]:::sdk
            end
        end

        %% Shared Modules
        subgraph "Shared Modules"
            direction TB
            SharedConsts["Shared Constants"]:::sdk
            SharedTypes["Shared Types"]:::sdk
            SharedUtils["Shared Utilities"]:::sdk
        end
    end

    %% External Systems
    subgraph "External Systems"
        direction TB
        Kubernetes["Kubernetes Control Plane"]:::external
        OSProcess["Local OS Process Launcher"]:::external
    end

    %% CI/CD & Packaging
    subgraph "CI/CD & Packaging"
        direction TB
        Workflows["CI/CD Workflows"]:::external
        BuildScripts["Build & Release Scripts"]:::external
        Packaging["Packaging Configuration"]:::external
    end

    %% Connections
    User -->|"calls train()"| TrainerClient
    TrainerClient -->|"uses"| BaseBackend
    CustomTrainer -->|"uses"| BaseBackend
    BaseBackend -->|"implements"| K8sBackend
    BaseBackend -->|"implements"| LPBackend
    K8sBackend -->|"uses API"| Kubernetes
    K8sBackend -->|"relies on"| K8sTypes
    LPBackend -->|"spawns"| OSProcess
    LPBackend -->|"relies on"| LPConstants
    LPBackend -->|"relies on"| LPJob
    LPBackend -->|"relies on"| LPTypes
    LPBackend -->|"relies on"| LPUtils
    K8sBackend -->|"uses"| SharedConsts
    K8sBackend -->|"uses"| SharedTypes
    K8sBackend -->|"uses"| SharedUtils
    LPBackend -->|"uses"| SharedConsts
    LPBackend -->|"uses"| SharedTypes
    LPBackend -->|"uses"| SharedUtils

    %% CI/CD Connections
    SharedUtils -->|"tested by"| Workflows
    SDK -->|"packaged by"| Packaging
    Packaging -->|"build scripts"| BuildScripts

    %% Click Events
    click TrainerClient "https://github.com/opendatahub-io/kubeflow-sdk/blob/main/kubeflow/trainer/api/trainer_client.py"
    click BaseBackend "https://github.com/opendatahub-io/kubeflow-sdk/blob/main/kubeflow/trainer/backends/base.py"
    click K8sBackend "https://github.com/opendatahub-io/kubeflow-sdk/blob/main/kubeflow/trainer/backends/kubernetes/backend.py"
    click K8sTypes "https://github.com/opendatahub-io/kubeflow-sdk/blob/main/kubeflow/trainer/backends/kubernetes/types.py"
    click LPBackend "https://github.com/opendatahub-io/kubeflow-sdk/blob/main/kubeflow/trainer/backends/localprocess/backend.py"
    click LPConstants "https://github.com/opendatahub-io/kubeflow-sdk/blob/main/kubeflow/trainer/backends/localprocess/constants.py"
    click LPJob "https://github.com/opendatahub-io/kubeflow-sdk/blob/main/kubeflow/trainer/backends/localprocess/job.py"
    click LPTypes "https://github.com/opendatahub-io/kubeflow-sdk/blob/main/kubeflow/trainer/backends/localprocess/types.py"
    click LPUtils "https://github.com/opendatahub-io/kubeflow-sdk/blob/main/kubeflow/trainer/backends/localprocess/utils.py"
    click SharedConsts "https://github.com/opendatahub-io/kubeflow-sdk/blob/main/kubeflow/trainer/constants/constants.py"
    click SharedTypes "https://github.com/opendatahub-io/kubeflow-sdk/blob/main/kubeflow/trainer/types/types.py"
    click SharedUtils "https://github.com/opendatahub-io/kubeflow-sdk/blob/main/kubeflow/trainer/utils/utils.py"
    click Workflows "https://github.com/opendatahub-io/kubeflow-sdk/tree/main/.github/workflows/"
    click BuildScripts "https://github.com/opendatahub-io/kubeflow-sdk/tree/main/Makefile"
    click Packaging "https://github.com/opendatahub-io/kubeflow-sdk/blob/main/pyproject.toml"

    %% Styles
    classDef sdk fill:#E0F7FA,stroke:#0288D1,stroke-width:1px
    classDef external fill:#E8F5E9,stroke:#388E3C,stroke-width:1px
    classDef dashed fill:none,stroke:#000,stroke-dasharray: 5 5;
```
