# revodatanl/revo-asset-bundle-templates architecture

> Architecture diagram of the GitHub repository revodatanl/revo-asset-bundle-templates, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/revodatanl/revo-asset-bundle-templates
- Repository: https://github.com/revodatanl/revo-asset-bundle-templates

## Overview

##### Project type and purpose
- This is an open-source “scaffolding” or template-generator repository for Databricks Asset Bundles.
- Purpose: provide a turnkey project structure (code, pipelines, docs, devcontainer, tests, CI/CD) that users can initialize via `databricks bundle init`.

##### Top-level structure and key directories
- `.github/` + `.github/workflows/`: GitHub Actions CI/CD templates
- `modules/Makefile`: reusable Make targets for building, testing, packaging
- `release.config.mjs`: semantic-release configuration
- `template/`: Jinja-style templates (`{{.project_name}}`, `{{.package_name}}`) for generating a new project
- `.azuredevops/`: Azure Pipelines YAML templates
- `.devcontainer/`: Dockerfile and devcontainer configuration
- `docs/`: markdown docs templates (getting started, coding standards)
- `notebooks/`, `resources/`: example DLT pipelines and jobs
- `src/{{.package_name}}`: stub Python package
- `tests/`: example Pytest tests

A. Template Repository
– Houses all scaffolding artifacts (CI/CD, devcontainer, docs, code stubs)
B. Template Engine (Databricks CLI plugin)
– `databricks bundle init` plugs into this repo to render templates into a new project
C. Generated Project Workspace
– The user’s new repo with concrete files, pipelines, docs, code, tests
D. Development Environment
– Local machine using DevContainer (Docker + VSCode)
– Pre-commit hooks, Mypy, Ruff linters configured via `.pre-commit-config.yaml`
E. CI/CD Platforms
– GitHub Actions (defined in `.github/workflows/`)
– Azure DevOps Pipelines (in `.azuredevops/`)
– semantic-release for automated versioning and publishing
F. Databricks Environment
– Target cloud environment (Azure or AWS)
– Databricks CLI profiles to authenticate & deploy asset bundles

##### Relationships and data flows
1. User invokes Databricks CLI → reads `template/` → Template Engine renders files → outputs Generated Project
2. Developer opens Generated Project in DevContainer → writes code & tests → runs Make targets locally
3. Developer pushes to GitHub/Azure Repos → triggers CI/CD workflows (build, lint, test)
4. CI/CD pipelines invoke Databricks CLI → deploy notebooks, DLT pipelines, jobs to Databricks workspace
5. semantic-release auto-increments version → publishes package (e.g., to PyPI)

##### Architectural patterns and principles
- Template-based scaffolding (separation of concerns between template repo vs generated project)
- Pipeline-as-code (CI/CD defined alongside code stubs)
- Infrastructure automation (Databricks CLI + pipeline scripts)
- Modular Makefile for common tasks

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Template Repository
    subgraph "Template Repository"
        direction TB
        GH[".github/"]:::code
        MF["modules/Makefile"]:::code
        RC["release.config.mjs"]:::code
        DS["databricks_template_schema.json"]:::code
        TP["template/"]:::code
    end

    %% Template Engine
    TE["Template Engine\n(Databricks CLI plugin)"]:::code

    PRE["template/__preamble.tmpl"]:::code
    TMP["template/{{.project_name}}/**"]:::code

    %% Generated Project
    GP_ROOT["Generated Project\n(template/{{.project_name}}/)"]:::code
    GP_MF["Makefile.tmpl"]:::code
    GP_RD["README.md.tmpl"]:::code
    GP_PY["pyproject.toml.tmpl"]:::code
    GP_SRC["src/{{.package_name}}/"]:::code
    GP_TST["tests/"]:::code

    %% Development Environment
    subgraph "Dev Environment"
        direction TB
        PC[".pre-commit-config.yaml"]:::dev
        DC[".devcontainer/"]:::dev
        VS[".vscode/"]:::dev
    end

    %% Repos
    REPO(("GitHub/Azure Repos")):::code

    %% CI/CD
    subgraph "CI/CD Pipelines"
        direction TB
        subgraph "GitHub Actions"
            direction TB
            CI1["ci-dabs.yml"]:::cicd
            CI2["semantic-release.yml"]:::cicd
            CI3["ci.yml.tmpl"]:::cicd
            CI4["deploy-dab.yml"]:::cicd
            CI5["semantic-release.yml"]:::cicd
        end
        subgraph "Azure Pipelines"
            direction TB
            AZ1["ci.yml.tmpl"]:::cicd
            AZ2["deploy-dab.yml"]:::cicd
            AZ3["semantic-release.yml"]:::cicd
            AZ4["deploy-dab-template.yml"]:::cicd
        end
    end

    %% Databricks and Package Registry
    DBW(("Databricks Workspace")):::cloud
    PR(("Package Registry")):::cloud

    %% Flows
    GH -->|renders templates| TE
    MF --> TE
    RC --> TE
    DS --> TE
    TP --> TE
    PRE --> TE
    TMP --> TE

    TE -->|init → render| GP_ROOT
    GP_ROOT --> GP_MF
    GP_ROOT --> GP_RD
    GP_ROOT --> GP_PY
    GP_ROOT --> GP_SRC
    GP_ROOT --> GP_TST

    GP_ROOT -->|"develop locally"| DC
    GP_ROOT -->|"develop locally"| PC
    GP_ROOT -->|"develop locally"| VS

    GP_ROOT -->|"git push"| REPO
    REPO -->|"trigger CI"| CI1
    REPO --> CI2
    REPO -->|"render project workflows"| CI3
    REPO --> CI4
    REPO --> CI5
    REPO --> AZ1
    REPO --> AZ2
    REPO --> AZ3
    REPO --> AZ4

    CI1 -->|"databricks deploy"| DBW
    CI2 -.->|"semantic-release publish"| PR
    CI3 --> DBW
    CI4 --> DBW
    CI5 -.-> PR
    AZ1 --> DBW
    AZ2 --> DBW
    AZ3 -.-> PR
    AZ4 --> DBW

    %% Click Events
    click GH "https://github.com/revodatanl/revo-asset-bundle-templates/tree/main/.github/"
    click MF "https://github.com/revodatanl/revo-asset-bundle-templates/tree/main/modules/Makefile"
    click RC "https://github.com/revodatanl/revo-asset-bundle-templates/blob/main/release.config.mjs"
    click DS "https://github.com/revodatanl/revo-asset-bundle-templates/blob/main/databricks_template_schema.json"
    click TP "https://github.com/revodatanl/revo-asset-bundle-templates/tree/main/template/"
    click PRE "https://github.com/revodatanl/revo-asset-bundle-templates/blob/main/template/__preamble.tmpl"
    click TMP "https://github.com/revodatanl/revo-asset-bundle-templates/tree/main/template/{{.project_name}}/**"
    click GP_ROOT "https://github.com/revodatanl/revo-asset-bundle-templates/tree/main/template/{{.project_name}}/"
    click GP_MF "https://github.com/revodatanl/revo-asset-bundle-templates/blob/main/template/{{.project_name}}/Makefile.tmpl"
    click GP_RD "https://github.com/revodatanl/revo-asset-bundle-templates/blob/main/template/{{.project_name}}/README.md.tmpl"
    click GP_PY "https://github.com/revodatanl/revo-asset-bundle-templates/blob/main/template/{{.project_name}}/pyproject.toml.tmpl"
    click GP_SRC "https://github.com/revodatanl/revo-asset-bundle-templates/tree/main/template/{{.project_name}}/src/{{.package_name}}/"
    click GP_TST "https://github.com/revodatanl/revo-asset-bundle-templates/tree/main/template/{{.project_name}}/tests/"
    click PC "https://github.com/revodatanl/revo-asset-bundle-templates/blob/main/.pre-commit-config.yaml"
    click DC "https://github.com/revodatanl/revo-asset-bundle-templates/tree/main/template/{{.project_name}}/.devcontainer/"
    click VS "https://github.com/revodatanl/revo-asset-bundle-templates/tree/main/template/{{.project_name}}/.vscode/"
    click CI1 "https://github.com/revodatanl/revo-asset-bundle-templates/blob/main/.github/workflows/ci-dabs.yml"
    click CI2 "https://github.com/revodatanl/revo-asset-bundle-templates/blob/main/.github/workflows/semantic-release.yml"
    click CI3 "https://github.com/revodatanl/revo-asset-bundle-templates/blob/main/template/{{.project_name}}/.github/workflows/ci.yml.tmpl"
    click CI4 "https://github.com/revodatanl/revo-asset-bundle-templates/blob/main/template/{{.project_name}}/.github/workflows/deploy-dab.yml"
    click CI5 "https://github.com/revodatanl/revo-asset-bundle-templates/blob/main/template/{{.project_name}}/.github/workflows/semantic-release.yml"
    click AZ1 "https://github.com/revodatanl/revo-asset-bundle-templates/blob/main/template/{{.project_name}}/.azuredevops/.azure-pipelines/ci.yml.tmpl"
    click AZ2 "https://github.com/revodatanl/revo-asset-bundle-templates/blob/main/template/{{.project_name}}/.azuredevops/.azure-pipelines/deploy-dab.yml"
    click AZ3 "https://github.com/revodatanl/revo-asset-bundle-templates/blob/main/template/{{.project_name}}/.azuredevops/.azure-pipelines/semantic-release.yml"
    click AZ4 "https://github.com/revodatanl/revo-asset-bundle-templates/blob/main/template/{{.project_name}}/.azuredevops/.azure-pipelines/templates/deploy-dab-template.yml"

    %% Styles
    classDef code fill:#D0E6FF,stroke:#009,stroke-width:1px
    classDef cicd fill:#E6FFE6,stroke:#090,stroke-width:1px
    classDef dev fill:#F0F0F0,stroke:#888,stroke-width:1px
    classDef cloud fill:#FFE6CC,stroke:#E60,stroke-width:1px
```
