How opendatahub-io/kubeflow-sdk works
Project type and purpose
• Kubeflow SDK is an open-source Python library (SDK) that provides AI practitioners with a unified, high-level API to launch and manage training workloads on Kubeflow (Kubernetes) or locally.
• It abstracts the complexity of Kubernetes, offers consistent Python interfaces, and supports extensibility for new “backends” (runtimes).
File structure
• Top-level:
• kubeflow/ – the main Python package
• kubeflow/trainer/api/ – public client APIs (TrainerClient, CustomTrainer)
• kubeflow/trainer/backends/ – pluggable runtimes:
– kubernetes/ backend talks to the Kubernetes API
– localprocess/ backend spins up local OS processes
• kubeflow/trainer/constants/, types/, utils/ – shared constants, data models, and helper functions
• tests/ alongside each module – unit tests
• CI/CD and release: GitHub workflows, Makefile, scripts/, pyproject.toml
README for additional insights
• Confirms core components: TrainerClient, CustomTrainer, built-in backends (Kubeflow Trainer, Katib planned).
• Use cases: local development, distributed PyTorch jobs, future support for hyperparameter optimization and pipelines.
Final recommendation
• Be exhaustive in capturing each major module and its dependencies.
• Keep the user’s perspective front-and-center: show the simple API entry point and how it propagates down to actual execution.