# starburstdata/dbt-trino-utils architecture

> Architecture diagram of the GitHub repository starburstdata/dbt-trino-utils, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-04-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/starburstdata/dbt-trino-utils
- Repository: https://github.com/starburstdata/dbt-trino-utils

## Overview

1. **Repository purpose**
   - `trino_utils` is a **dbt package** that provides Trino/Starburst-compatible macro implementations.
   - Its main role is to act as a **dispatch layer**: when another dbt package calls a macro, dbt can route to `trino_utils` first for Trino-specific SQL, then fall back to the upstream package.

2. **Core architectural boundary**
   - The package is not a standalone application or service.
   - The main boundary is between:
     - **consumer dbt projects**
     - **this package’s macros**
     - **upstream packages** (`dbt_utils`, `dbt_date`, and `dbt_metrics`)
     - the **target warehouse**: Trino or Starburst

3. **Primary code surface: macros**
   - The `macros/` directory contains the actual compatibility implementations.
   - It is organized by upstream package namespace:
     - `macros/dbt_utils/`
     - `macros/dbt_date/`
     - `macros/dbt_metrics/`
   - These macros define Trino/Starburst-specific SQL behavior for functions like date spine generation, deduplication, generic tests, timezone conversion, and metric SQL generation.

4. **Compatibility model**
   - This repo is a set of **“shims”** rather than a full fork.
   - The important architectural idea is that a consuming dbt project enables dispatch in `dbt_project.yml`, so macro resolution searches `trino_utils` before the upstream namespace.
   - That means this repo only overrides the parts of upstream packages that need warehouse-specific SQL.

5. **Upstream package linkage**
   - The repository includes git submodules for:
     - `dbt-utils`
     - `dbt-date`
     - `dbt_metrics`
   - These submodules are useful as a reference/compatibility source and help keep the shim implementations aligned with upstream package behavior.

6. **Macro categories**
   - The macros fall into three functional groups:
     - **SQL generation helpers** for Trino-compatible query patterns
     - **generic tests** adapted for Trino semantics
     - **schema cleanup / maintenance operations** for dev database hygiene
   - Cleanup macros are operational utilities rather than model logic, used via `dbt run-operation`.

7. **Integration test architecture**
   - The `integration_tests/` directory contains separate dbt projects for validating each supported upstream package behavior:
     - `integration_tests/dbt_utils`
     - `integration_tests/dbt_date`
     - `integration_tests/dbt_metrics`
   - Each test project packages `trino_utils` and exercises the dispatched macros in realistic dbt flows using models, seeds, and tests.

8. **Data flow in tests**
   - Test data typically flows as:
     - CSV seed files or test models in the integration project
     - dbt runs invoking macros from `trino_utils`
     - generated SQL executed against a Trino/Starburst instance
     - results compared to expected output or schema assertions
   - This validates that the shimmed macros behave correctly in warehouse-specific execution.

9. **Warehouse/runtime environment**
   - The repo includes Docker assets for local and CI execution against both:
     - **Trino**
     - **Starburst**
   - `docker-compose-trino.yml` and `docker-compose-starburst.yml` define those environments.
   - The `docker/` directory contains warehouse config, init/removal scripts, and a dbt container image used to run integration tests.

10. **CI and automation**
   - GitHub Actions (`.github/workflows/ci.yml`) runs the repository’s validation pipeline.
   - The CI setup appears centered on spinning up the containerized warehouse, running dbt integration tests, and verifying macro compatibility across supported packages.

11. **dbt project structure**
   - The root `dbt_project.yml` makes this repository behave as a dbt package.
   - The package layout is intentionally macro-heavy: there are no primary analytical models at the root, because the product is the macro library itself.

12. **Operational utilities**
   - The cleanup macros in `macros/dbt_utils/schema_cleanup/` are notable because they support developer workflow around Trino schemas:
     - drop schemas by prefix
     - drop a schema by name
     - drop relations no longer present in the project
   - These are invoked as dbt operations, not as part of model builds.

13. **How to diagram it**
   - A useful architecture diagram would show:
     - consumer dbt project on the left
     - `trino_utils` in the middle as the dispatch/override layer
     - upstream packages as fallback references
     - Trino/Starburst as the execution target on the right
     - integration test projects and Dockerized warehouse setup as a validation path beneath the main flow

14. **Overall design takeaway**
   - The repo’s architecture is intentionally small and focused: it does not own data pipelines itself.
   - Its job is to make existing dbt package macros portable to Trino/Starburst by providing a thin, warehouse-aware SQL compatibility layer and testing that layer in isolated dbt projects.

Main entry points: [`dbt_project.yml`](https://github.com/starburstdata/dbt-trino-utils/blob/main/dbt_project.yml) (trino_utils), [`integration_tests/dbt_utils/dbt_project.yml`](https://github.com/starburstdata/dbt-trino-utils/blob/main/integration_tests/dbt_utils/dbt_project.yml) (dbt_utils tests), [`docker/trino`](https://github.com/starburstdata/dbt-trino-utils/tree/main/docker/trino) (Trino), [`.github/workflows/ci.yml`](https://github.com/starburstdata/dbt-trino-utils/blob/main/.github/workflows/ci.yml) (CI workflow)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_core["Core package"]
  node_consumer_dbt["Consumer dbt<br/>dbt project"]
  node_package_root["trino_utils<br/>dbt package<br/>[dbt_project.yml]"]
  node_dbt_utils_sql["dbt_utils SQL<br/>macro shims"]
  node_dbt_utils_tests["dbt_utils tests<br/>macro shims"]
  node_schema_cleanup["Schema cleanup<br/>ops macros"]
  node_dbt_date_macros["dbt_date<br/>macro shim"]
  node_dbt_metrics_sql["dbt_metrics SQL<br/>macro shim"]
end

subgraph group_upstream["Upstream refs"]
  node_upstream_dbt_utils["dbt-utils<br/>upstream package"]
  node_upstream_dbt_date["dbt-date<br/>upstream package"]
  node_upstream_dbt_metrics["dbt_metrics<br/>upstream package"]
end

subgraph group_validation["Validation"]
  node_itest_utils["dbt_utils tests<br/>integration project<br/>[dbt_project.yml]"]
  node_itest_date["dbt_date tests<br/>integration project<br/>[dbt_project.yml]"]
  node_itest_metrics["dbt_metrics tests<br/>integration project<br/>[dbt_project.yml]"]
  node_ci_workflow["CI workflow<br/>automation<br/>[ci.yml]"]
end

subgraph group_runtime["Runtime"]
  node_docker_compose["Docker compose<br/>infra config"]
  node_docker_assets["Docker assets<br/>container runtime"]
  node_trino[("Trino<br/>warehouse runtime")]
  node_starburst[("Starburst<br/>warehouse runtime")]
end

node_consumer_dbt -->|"dispatches to"| node_package_root
node_package_root -->|"exposes"| node_dbt_utils_sql
node_package_root -->|"exposes"| node_dbt_utils_tests
node_package_root -->|"exposes"| node_schema_cleanup
node_package_root -->|"exposes"| node_dbt_date_macros
node_package_root -->|"exposes"| node_dbt_metrics_sql
node_dbt_utils_sql -.->|"matches"| node_upstream_dbt_utils
node_dbt_utils_tests -.->|"matches"| node_upstream_dbt_utils
node_dbt_date_macros -.->|"matches"| node_upstream_dbt_date
node_dbt_metrics_sql -.->|"matches"| node_upstream_dbt_metrics
node_itest_utils -->|"validates"| node_dbt_utils_sql
node_itest_utils -->|"validates"| node_dbt_utils_tests
node_itest_date -->|"validates"| node_dbt_date_macros
node_itest_metrics -->|"validates"| node_dbt_metrics_sql
node_ci_workflow -->|"runs"| node_itest_utils
node_ci_workflow -->|"runs"| node_itest_date
node_ci_workflow -->|"runs"| node_itest_metrics
node_docker_compose -->|"starts"| node_trino
node_docker_compose -->|"starts"| node_starburst
node_docker_assets -->|"configures"| node_trino
node_docker_assets -->|"configures"| node_starburst
node_itest_utils -->|"executes on"| node_trino
node_itest_date -->|"executes on"| node_trino
node_itest_metrics -->|"executes on"| node_starburst

click node_package_root "https://github.com/starburstdata/dbt-trino-utils/blob/main/dbt_project.yml"
click node_dbt_utils_sql "https://github.com/starburstdata/dbt-trino-utils/tree/main/macros/dbt_utils/sql"
click node_dbt_utils_tests "https://github.com/starburstdata/dbt-trino-utils/tree/main/macros/dbt_utils/generic_tests"
click node_schema_cleanup "https://github.com/starburstdata/dbt-trino-utils/tree/main/macros/dbt_utils/schema_cleanup"
click node_dbt_date_macros "https://github.com/starburstdata/dbt-trino-utils/blob/main/macros/dbt_date/convert_timezone.sql"
click node_dbt_metrics_sql "https://github.com/starburstdata/dbt-trino-utils/tree/main/macros/dbt_metrics/sql_gen"
click node_upstream_dbt_utils "https://github.com/starburstdata/dbt-trino-utils/tree/main/dbt-utils"
click node_upstream_dbt_date "https://github.com/starburstdata/dbt-trino-utils/tree/main/dbt-date"
click node_upstream_dbt_metrics "https://github.com/starburstdata/dbt-trino-utils/tree/main/dbt_metrics"
click node_itest_utils "https://github.com/starburstdata/dbt-trino-utils/blob/main/integration_tests/dbt_utils/dbt_project.yml"
click node_itest_date "https://github.com/starburstdata/dbt-trino-utils/blob/main/integration_tests/dbt_date/dbt_project.yml"
click node_itest_metrics "https://github.com/starburstdata/dbt-trino-utils/blob/main/integration_tests/dbt_metrics/dbt_project.yml"
click node_ci_workflow "https://github.com/starburstdata/dbt-trino-utils/blob/main/.github/workflows/ci.yml"
click node_docker_compose "https://github.com/starburstdata/dbt-trino-utils/blob/main/docker-compose-trino.yml"
click node_docker_assets "https://github.com/starburstdata/dbt-trino-utils/tree/main/docker"
click node_trino "https://github.com/starburstdata/dbt-trino-utils/tree/main/docker/trino"
click node_starburst "https://github.com/starburstdata/dbt-trino-utils/tree/main/docker/starburst"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_consumer_dbt,node_package_root,node_dbt_utils_sql,node_dbt_utils_tests,node_schema_cleanup,node_dbt_date_macros,node_dbt_metrics_sql toneBlue
class node_upstream_dbt_utils,node_upstream_dbt_date,node_upstream_dbt_metrics toneAmber
class node_itest_utils,node_itest_date,node_itest_metrics,node_ci_workflow toneMint
class node_docker_compose,node_docker_assets,node_trino,node_starburst toneRose
```

## Components

### Core package

Dispatch layer and compatibility macros

- **Consumer dbt**: dbt project · A downstream project enabling dispatch to resolve macros through trino_utils first
- [trino_utils](https://github.com/starburstdata/dbt-trino-utils/blob/main/dbt_project.yml): dbt package · Root dbt package entrypoint that defines this repository as a macro library · `dbt_project.yml`
- [dbt_utils SQL](https://github.com/starburstdata/dbt-trino-utils/tree/main/macros/dbt_utils/sql): macro shims · Trino-compatible implementations for upstream dbt_utils SQL helpers · `macros/dbt_utils/sql`
- [dbt_utils tests](https://github.com/starburstdata/dbt-trino-utils/tree/main/macros/dbt_utils/generic_tests): macro shims · Generic tests adapted for Trino semantics · `macros/dbt_utils/generic_tests`
- [Schema cleanup](https://github.com/starburstdata/dbt-trino-utils/tree/main/macros/dbt_utils/schema_cleanup): ops macros · dbt run-operation utilities for dropping schemas and stale relations · `macros/dbt_utils/schema_cleanup`
- [dbt_date](https://github.com/starburstdata/dbt-trino-utils/blob/main/macros/dbt_date/convert_timezone.sql): macro shim · Timezone conversion compatibility for dbt_date · `macros/dbt_date/convert_timezone.sql`
- [dbt_metrics SQL](https://github.com/starburstdata/dbt-trino-utils/tree/main/macros/dbt_metrics/sql_gen): macro shim · Metric SQL generation adapted for Trino and Starburst · `macros/dbt_metrics/sql_gen`

### Upstream refs

Referenced packages mirrored by the shims

- [dbt-utils](https://github.com/starburstdata/dbt-trino-utils/tree/main/dbt-utils): upstream package · Referenced upstream package used as compatibility target and fallback · `dbt-utils`
- [dbt-date](https://github.com/starburstdata/dbt-trino-utils/tree/main/dbt-date): upstream package · Referenced upstream date package used as compatibility target and fallback · `dbt-date`
- [dbt_metrics](https://github.com/starburstdata/dbt-trino-utils/tree/main/dbt_metrics): upstream package · Referenced upstream metrics package used as compatibility target and fallback · `dbt_metrics`

### Validation

Integration test projects and CI

- [dbt_utils tests](https://github.com/starburstdata/dbt-trino-utils/blob/main/integration_tests/dbt_utils/dbt_project.yml): integration project · dbt project that exercises dbt_utils dispatch behavior against Trino · `integration_tests/dbt_utils/dbt_project.yml`
- [dbt_date tests](https://github.com/starburstdata/dbt-trino-utils/blob/main/integration_tests/dbt_date/dbt_project.yml): integration project · dbt project that validates dbt_date compatibility macros · `integration_tests/dbt_date/dbt_project.yml`
- [dbt_metrics tests](https://github.com/starburstdata/dbt-trino-utils/blob/main/integration_tests/dbt_metrics/dbt_project.yml): integration project · dbt project that validates metric SQL generation and calendar joins · `integration_tests/dbt_metrics/dbt_project.yml`
- [CI workflow](https://github.com/starburstdata/dbt-trino-utils/blob/main/.github/workflows/ci.yml): automation · GitHub Actions pipeline that runs containerized integration validation · `.github/workflows/ci.yml`

### Runtime

Containerized Trino and Starburst environments

- [Docker compose](https://github.com/starburstdata/dbt-trino-utils/blob/main/docker-compose-trino.yml): infra config · Compose entrypoints for local and CI execution on Trino and Starburst · `docker-compose-trino.yml`
- [Docker assets](https://github.com/starburstdata/dbt-trino-utils/tree/main/docker): container runtime · Warehouse configs and dbt image used to initialize and run tests · `docker`
- [Trino](https://github.com/starburstdata/dbt-trino-utils/tree/main/docker/trino): warehouse runtime · Execution target for generated SQL and dispatched macros · `docker/trino`
- [Starburst](https://github.com/starburstdata/dbt-trino-utils/tree/main/docker/starburst): warehouse runtime · Alternative execution target for generated SQL and dispatched macros · `docker/starburst`

## Connections

- Consumer dbt → trino_utils: dispatches to · Downstream dbt projects route macro resolution into this package first
- trino_utils → dbt_utils SQL: exposes · Package root publishes the Trino-compatible dbt_utils SQL overrides
- trino_utils → dbt_utils tests: exposes · Package root publishes generic test overrides for dbt_utils
- trino_utils → Schema cleanup: exposes · Package root includes operational cleanup macros for dbt run-operation
- trino_utils → dbt_date: exposes · Package root includes dbt_date compatibility macros
- trino_utils → dbt_metrics SQL: exposes · Package root includes dbt_metrics SQL generation macros
- dbt_utils SQL → dbt-utils: matches · Implementations track upstream dbt-utils behavior while changing warehouse-specific SQL
- dbt_utils tests → dbt-utils: matches · Test overrides preserve upstream semantics under Trino
- dbt_date → dbt-date: matches · Timezone conversion shim aligns with the upstream package contract
- dbt_metrics SQL → dbt_metrics: matches · Metric SQL generation follows the upstream dbt_metrics interface
- dbt_utils tests → dbt_utils SQL: validates · Integration project executes deduplication and related dbt_utils macros
- dbt_utils tests → dbt_utils tests: validates · Integration project runs generic tests against Trino semantics
- dbt_date tests → dbt_date: validates · Integration project checks date and timezone compatibility behavior
- dbt_metrics tests → dbt_metrics SQL: validates · Integration project exercises metric SQL generation and calendar joins
- CI workflow → dbt_utils tests: runs · CI executes the dbt_utils integration project
- CI workflow → dbt_date tests: runs · CI executes the dbt_date integration project
- CI workflow → dbt_metrics tests: runs · CI executes the dbt_metrics integration project
- Docker compose → Trino: starts · Compose file provisions the Trino warehouse for tests
- Docker compose → Starburst: starts · Compose file provisions the Starburst warehouse for tests
- Docker assets → Trino: configures · Warehouse config and init scripts prepare the Trino runtime
- Docker assets → Starburst: configures · Warehouse config and init scripts prepare the Starburst runtime
- dbt_utils tests → Trino: executes on · Integration dbt runs generate SQL against Trino
- dbt_date tests → Trino: executes on · Integration dbt runs generate SQL against Trino
- dbt_metrics tests → Starburst: executes on · Integration dbt runs generate SQL against Starburst or Trino-compatible warehouse targets
