How databricks/cli works
Databricks CLI exposes command-line operations against Databricks services and supports defining, generating, deploying, and running Databricks Asset Bundles. The clearest sampled workflow is pipeline management: users invoke workspace commands or bundle commands; bundle generation can fetch an existing pipeline and download its libraries and root-path files, while bundle runtime code can start updates and poll for status. Bundle configuration, resource modeling, validation, and Python pipeline models form supporting layers. Evidence is strongest for the shown pipeline calls; other command and deployment relationships are intentionally left unconnected where excerpts do not establish them.
Main entry points: ingestion_pipeline_definition.py Ingestion definitions · ingestion_gateway_pipeline_definition.py Gateway definition · generate/pipeline.go Generate pipeline · run/pipeline.go Pipeline update runner
Components 24
CLI commands
- Workspace pipelines
cmd/workspace/pipelines/pipelines.go - Feature engineering
cmd/workspace/feature-engineering/feature-engineering.go - Account storage
cmd/account/storage/storage.go - Generate pipeline
cmd/bundle/generate/pipeline.go
Asset Bundles
- Config engine
bundle/config/engine/engine.go - Pipeline config
bundle/config/resources/pipeline.go - Development validation
bundle/config/validate/pipeline_development_deprecated.go - Pipeline metadata
bundle/deploy/metadata/annotate_pipelines.go - Direct resource
bundle/direct/dresources/pipeline.go - Config conversion
bundle/generate/pipeline.go
Pipeline runtime
- Pipeline update runner
bundle/run/pipeline.go - Progress reporting
bundle/run/progress/pipeline.go - Local Python setup
libs/localenv/pipeline.go
Python bundle models
- Bundle core
python/databricks/bundles/core/_bundle.py - Bundle locations
python/databricks/bundles/core/_location.py - Model transforms
python/databricks/bundles/core/_transform.py - JSON serialization
python/databricks/bundles/core/_transform_to_json.py - Bundle variables
python/databricks/bundles/core/_variable.py - Pipeline model set
python/databricks/bundles/pipelines/_models - Ingestion definitions
python/databricks/bundles/pipelines/_models/ingestion_pipeline_definition.py - Gateway definition
python/databricks/bundles/pipelines/_models/ingestion_gateway_pipeline_definition.py
Databricks services
- Workspace API
- Account API
Other components
- CLI user
Connections 18
- CLI user → Workspace pipelines: invokes
README.md - CLI user → Feature engineering: invokes
README.md - CLI user → Account storage: invokes
README.md - CLI user → Generate pipeline: invokes
README.md - Generate pipeline → Workspace API: fetches pipeline
cmd/bundle/generate/pipeline.go - Generate pipeline → Config conversion: converts spec
cmd/bundle/generate/pipeline.go - Pipeline update runner → Workspace API: starts and polls
bundle/run/pipeline.go - Pipeline update runner → Progress reporting: reports progress
- Direct resource → Workspace API: manages pipeline
bundle/direct/dresources/pipeline.go - Ingestion definitions → Model transforms: transforms from dict
python/databricks/bundles/pipelines/_models/ingestion_pipeline_definition.py - Ingestion definitions → JSON serialization: serializes to JSON
python/databricks/bundles/pipelines/_models/ingestion_pipeline_definition.py - Ingestion definitions → Bundle variables: supports variables
python/databricks/bundles/pipelines/_models/ingestion_pipeline_definition.py - Gateway definition → Model transforms: transforms from dict
python/databricks/bundles/pipelines/_models/ingestion_gateway_pipeline_definition.py - Gateway definition → JSON serialization: serializes to JSON
python/databricks/bundles/pipelines/_models/ingestion_gateway_pipeline_definition.py - Gateway definition → Bundle variables: supports variables
python/databricks/bundles/pipelines/_models/ingestion_gateway_pipeline_definition.py - Account storage → Account API: calls API
cmd/account/storage/storage.go - Workspace pipelines → Workspace API: calls API
- Feature engineering → Workspace API: calls API