How kubeflow/kale works
Kale turns tagged Jupyter notebooks into Kubeflow Pipelines: users configure cell metadata in JupyterLab, compile and launch a run, then use deployment/status views. The Python runtime processes notebook cells into a dependency-aware pipeline, compiles it to KFP DSL, and interacts with Kubeflow and Kubernetes. The README documents the principal workflow and capabilities; source-index references support many internal dependencies, but unsampled files limit certainty about some runtime links.
Main entry points: nbprocessor.py Notebook processor · pipeline.py Pipeline model · compiler.py KFP DSL compiler · nb.py Notebook RPC operations
Components 24
Notebook authoring
- Cell metadata editor
labextension/src/widgets/cell-metadata/CellMetadataEditor.tsx - Cell tags and metadata
labextension/src/lib/TagsUtils.ts - Notebook state loader
labextension/src/widgets/hooks/useNotebookLoader.ts - Notebook commands
labextension/src/lib/Commands.ts - Deploy and status UI
labextension/src/widgets/deploys-progress/DeployProgress.tsx - Jupyter RPC handlers
jupyterlab_kubeflow_kale/handlers.py - Notebook RPC operations
kale/rpc/nb.py
Pipeline construction
- Notebook processor
kale/processors/nbprocessor.py - Code and dependency analysis
kale/common/astutils.py - Pipeline graph utilities
kale/common/graphutils.py - Pipeline model
kale/pipeline.py - Step model
kale/step.py - KFP DSL compiler
kale/compiler.py
Execution integrations
- KFP client factory
kale/common/kfp_client_factory.py - KFP operations
kale/common/kfputils.py - Kubernetes utilities
kale/common/k8sutils.py - Pod utilities
kale/common/podutils.py - Katib RPC
kale/rpc/katib.py - Katib utilities
kale/common/katibutils.py
Shared services
- Configuration validation
kale/config/validators.py - Object serialization
kale/marshal/backend.py
Other components
- Notebook user
- Kubeflow Pipelines
- Kubernetes cluster
Connections 24
- Notebook user → Cell metadata editor: tags cells
README.md - Cell metadata editor → Cell tags and metadata: updates tags
labextension/src/widgets/cell-metadata/CellMetadataEditor.tsx - Notebook state loader → Notebook commands: loads notebook
labextension/src/widgets/hooks/useNotebookLoader.ts - Notebook commands → Jupyter RPC handlers: requests backend
- Jupyter RPC handlers → Notebook RPC operations: dispatches operations
- Notebook user → Deploy and status UI: starts compile and run
README.md - Deploy and status UI → Notebook commands: uses commands
labextension/src/lib/Commands.ts - Notebook RPC operations → Notebook processor: processes notebook
kale/rpc/nb.py - Notebook processor → Code and dependency analysis: analyzes code
kale/processors/nbprocessor.py - Notebook processor → Pipeline graph utilities: builds dependencies
kale/processors/nbprocessor.py - Notebook processor → Pipeline model: creates pipeline
kale/processors/nbprocessor.py - Pipeline model → Step model: stores steps
kale/pipeline.py - Pipeline model → Configuration validation: validates configuration
kale/pipeline.py - KFP DSL compiler → Pipeline model: compiles pipeline
kale/compiler.py - KFP DSL compiler → Step model: emits step tasks
kale/compiler.py - Step model → Code and dependency analysis: analyzes source
kale/step.py - Notebook RPC operations → Object serialization: serializes objects
kale/rpc/nb.py - KFP DSL compiler → KFP operations: uses KFP operations
kale/compiler.py - KFP operations → KFP client factory: creates client
kale/common/kfputils.py - KFP operations → Kubeflow Pipelines: calls KFP
- Pod utilities → Kubernetes utilities: gets API client
kale/common/podutils.py - Katib RPC → Katib utilities: uses helpers
kale/rpc/katib.py - Katib RPC → Pod utilities: uses pod helpers
kale/rpc/katib.py - Kubernetes utilities → Kubernetes cluster: accesses cluster