# databricks/mlops-stacks architecture

> Architecture diagram of the GitHub repository databricks/mlops-stacks, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/databricks/mlops-stacks
- Repository: https://github.com/databricks/mlops-stacks

## Overview

Databricks MLOps Stacks is a project template, initialized through the Databricks CLI, that scaffolds ML code, Databricks resources-as-code, and CI/CD workflows. The documented lifecycle is development and pull requests, automated testing in staging, deployment of merged changes to staging, then promotion to production. The tree also shows feature engineering, validation, deployment, and monitoring template areas. Source excerpts cover only a retired hook and test configuration, so relationships below rely on README statements; internal calls among generated components are not inferred.

Main entry points: [`template/{{.input_root_dir}}/{{template `project_name_alphanumeric_underscore` .}}/resources`](https://github.com/databricks/mlops-stacks/tree/main/template/%7B%7B.input_root_dir%7D%7D/%7B%7Btemplate%20%60project_name_alphanumeric_underscore%60%20.%7D%7D/resources) (ML job resources), [`template/{{.input_root_dir}}/.github/workflows`](https://github.com/databricks/mlops-stacks/tree/main/template/%7B%7B.input_root_dir%7D%7D/.github/workflows) (GitHub Actions), [`template/{{.input_root_dir}}/.azure/devops-pipelines`](https://github.com/databricks/mlops-stacks/tree/main/template/%7B%7B.input_root_dir%7D%7D/.azure/devops-pipelines) (Azure DevOps), [`template/{{.input_root_dir}}/.gitlab/pipelines`](https://github.com/databricks/mlops-stacks/tree/main/template/%7B%7B.input_root_dir%7D%7D/.gitlab/pipelines) (GitLab pipelines)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ml["ML project code"]
  node_features["Feature engineering"]
  node_training["Model training<br/>[training]"]
  node_validation["Model validation<br/>[validation]"]
  node_batch["Batch inference<br/>[batch_inference]"]
  node_deployment["Model deployment<br/>[model_deployment]"]
  node_monitoring["Model monitoring<br/>[monitoring]"]
end

subgraph group_resources["Databricks resources"]
  node_mlresources["ML job resources<br/>[resources]"]
end

subgraph group_automation["CI/CD automation"]
  node_github["GitHub Actions<br/>[workflows]"]
  node_azure["Azure DevOps<br/>[devops-pipelines]"]
  node_gitlab["GitLab pipelines<br/>[pipelines]"]
end

subgraph group_environments["Workspaces"]
  node_staging[("Staging workspace")]
  node_production[("Production workspace")]
end

node_scientists(("Data scientists"))
node_ops(("Ops engineers"))
node_cli["Databricks CLI"]

node_scientists -->|"initialize project"| node_cli
node_ops -->|"initialize CI/CD"| node_cli
node_cli -->|"scaffolds project"| node_features
node_cli -->|"scaffolds resources"| node_mlresources
node_cli -->|"scaffolds workflows"| node_github
node_cli -->|"scaffolds workflows"| node_azure
node_cli -->|"scaffolds workflows"| node_gitlab
node_scientists -->|"iterate on ML"| node_training
node_scientists -->|"iterate on ML"| node_batch
node_github -->|"test and deploy"| node_staging
node_azure -->|"test and deploy"| node_staging
node_gitlab -->|"test and deploy"| node_staging
node_github -->|"deploy changes"| node_production
node_azure -->|"deploy changes"| node_production
node_gitlab -->|"deploy changes"| node_production
node_mlresources -->|"deploy ML jobs"| node_staging
node_mlresources -->|"deploy ML jobs"| node_production

click node_features "https://github.com/databricks/mlops-stacks/tree/main/template/%7B%7B.input_root_dir%7D%7D/%7B%7Btemplate%20%60project_name_alphanumeric_underscore%60%20.%7D%7D/feature_engineering"
click node_training "https://github.com/databricks/mlops-stacks/tree/main/template/%7B%7B.input_root_dir%7D%7D/%7B%7Btemplate%20%60project_name_alphanumeric_underscore%60%20.%7D%7D/training"
click node_validation "https://github.com/databricks/mlops-stacks/tree/main/template/%7B%7B.input_root_dir%7D%7D/%7B%7Btemplate%20%60project_name_alphanumeric_underscore%60%20.%7D%7D/validation"
click node_batch "https://github.com/databricks/mlops-stacks/tree/main/template/%7B%7B.input_root_dir%7D%7D/%7B%7Btemplate%20%60project_name_alphanumeric_underscore%60%20.%7D%7D/deployment/batch_inference"
click node_deployment "https://github.com/databricks/mlops-stacks/tree/main/template/%7B%7B.input_root_dir%7D%7D/%7B%7Btemplate%20%60project_name_alphanumeric_underscore%60%20.%7D%7D/deployment/model_deployment"
click node_monitoring "https://github.com/databricks/mlops-stacks/tree/main/template/%7B%7B.input_root_dir%7D%7D/%7B%7Btemplate%20%60project_name_alphanumeric_underscore%60%20.%7D%7D/monitoring"
click node_mlresources "https://github.com/databricks/mlops-stacks/tree/main/template/%7B%7B.input_root_dir%7D%7D/%7B%7Btemplate%20%60project_name_alphanumeric_underscore%60%20.%7D%7D/resources"
click node_github "https://github.com/databricks/mlops-stacks/tree/main/template/%7B%7B.input_root_dir%7D%7D/.github/workflows"
click node_azure "https://github.com/databricks/mlops-stacks/tree/main/template/%7B%7B.input_root_dir%7D%7D/.azure/devops-pipelines"
click node_gitlab "https://github.com/databricks/mlops-stacks/tree/main/template/%7B%7B.input_root_dir%7D%7D/.gitlab/pipelines"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_features,node_training,node_validation,node_batch,node_deployment,node_monitoring toneBlue
class node_mlresources toneAmber
class node_github,node_azure,node_gitlab toneMint
class node_staging,node_production toneRose
class node_scientists,node_ops,node_cli toneIndigo
```

## Components

### ML project code

- [Feature engineering](https://github.com/databricks/mlops-stacks/tree/main/template/%7B%7B.input_root_dir%7D%7D/%7B%7Btemplate%20%60project_name_alphanumeric_underscore%60%20.%7D%7D/feature_engineering): `template/{{.input_root_dir}}/{{template project_name_alphanumeric_underscore .}}/feature_engineering`
- [Model training](https://github.com/databricks/mlops-stacks/tree/main/template/%7B%7B.input_root_dir%7D%7D/%7B%7Btemplate%20%60project_name_alphanumeric_underscore%60%20.%7D%7D/training): `template/{{.input_root_dir}}/{{template project_name_alphanumeric_underscore .}}/training`
- [Model validation](https://github.com/databricks/mlops-stacks/tree/main/template/%7B%7B.input_root_dir%7D%7D/%7B%7Btemplate%20%60project_name_alphanumeric_underscore%60%20.%7D%7D/validation): `template/{{.input_root_dir}}/{{template project_name_alphanumeric_underscore .}}/validation`
- [Batch inference](https://github.com/databricks/mlops-stacks/tree/main/template/%7B%7B.input_root_dir%7D%7D/%7B%7Btemplate%20%60project_name_alphanumeric_underscore%60%20.%7D%7D/deployment/batch_inference): `template/{{.input_root_dir}}/{{template project_name_alphanumeric_underscore .}}/deployment/batch_inference`
- [Model deployment](https://github.com/databricks/mlops-stacks/tree/main/template/%7B%7B.input_root_dir%7D%7D/%7B%7Btemplate%20%60project_name_alphanumeric_underscore%60%20.%7D%7D/deployment/model_deployment): `template/{{.input_root_dir}}/{{template project_name_alphanumeric_underscore .}}/deployment/model_deployment`
- [Model monitoring](https://github.com/databricks/mlops-stacks/tree/main/template/%7B%7B.input_root_dir%7D%7D/%7B%7Btemplate%20%60project_name_alphanumeric_underscore%60%20.%7D%7D/monitoring): `template/{{.input_root_dir}}/{{template project_name_alphanumeric_underscore .}}/monitoring`

### Databricks resources

- [ML job resources](https://github.com/databricks/mlops-stacks/tree/main/template/%7B%7B.input_root_dir%7D%7D/%7B%7Btemplate%20%60project_name_alphanumeric_underscore%60%20.%7D%7D/resources): `template/{{.input_root_dir}}/{{template project_name_alphanumeric_underscore .}}/resources`

### CI/CD automation

- [GitHub Actions](https://github.com/databricks/mlops-stacks/tree/main/template/%7B%7B.input_root_dir%7D%7D/.github/workflows): `template/{{.input_root_dir}}/.github/workflows`
- [Azure DevOps](https://github.com/databricks/mlops-stacks/tree/main/template/%7B%7B.input_root_dir%7D%7D/.azure/devops-pipelines): `template/{{.input_root_dir}}/.azure/devops-pipelines`
- [GitLab pipelines](https://github.com/databricks/mlops-stacks/tree/main/template/%7B%7B.input_root_dir%7D%7D/.gitlab/pipelines): `template/{{.input_root_dir}}/.gitlab/pipelines`

### Workspaces

- **Staging workspace**
- **Production workspace**

### Other components

- **Data scientists**
- **Ops engineers**
- **Databricks CLI**

## Connections

- Data scientists → Databricks CLI: initialize project (evidence: `README.md`)
- Ops engineers → Databricks CLI: initialize CI/CD (evidence: `README.md`)
- Databricks CLI → Feature engineering: scaffolds project (evidence: `README.md`)
- Databricks CLI → ML job resources: scaffolds resources (evidence: `README.md`)
- Databricks CLI → GitHub Actions: scaffolds workflows (evidence: `README.md`)
- Databricks CLI → Azure DevOps: scaffolds workflows (evidence: `README.md`)
- Databricks CLI → GitLab pipelines: scaffolds workflows (evidence: `README.md`)
- Data scientists → Model training: iterate on ML (evidence: `README.md`)
- Data scientists → Batch inference: iterate on ML (evidence: `README.md`)
- GitHub Actions → Staging workspace: test and deploy (evidence: `README.md`)
- Azure DevOps → Staging workspace: test and deploy (evidence: `README.md`)
- GitLab pipelines → Staging workspace: test and deploy (evidence: `README.md`)
- GitHub Actions → Production workspace: deploy changes (evidence: `README.md`)
- Azure DevOps → Production workspace: deploy changes (evidence: `README.md`)
- GitLab pipelines → Production workspace: deploy changes (evidence: `README.md`)
- ML job resources → Staging workspace: deploy ML jobs (evidence: `README.md`)
- ML job resources → Production workspace: deploy ML jobs (evidence: `README.md`)