# dbt-labs/dbt-utils architecture

> Architecture diagram of the GitHub repository dbt-labs/dbt-utils, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dbt-labs/dbt-utils
- Repository: https://github.com/dbt-labs/dbt-utils

## Overview

##### Main Components

a. Core Components:
- Macro Categories (represented as modules)
- Generic Tests
- SQL Generators
- Introspective Macros
- Web Macros
- Jinja Helpers

b. Integration Layer:
- dbt Core Integration
- Database Adapters Support
- Dispatch System

**┌─────────────────────────────────────────┐**
- dbt Project Layer            │
- ┘

**┌─────────────────────────────────────────┐**
- dbt-utils Macros              │
- ┬───────────┬────────────────┤
- Generic  │   SQL     │  Introspective │
- Tests   │ Generators│    Macros      │
- ┼───────────┼────────────────┤
- Web     │  Jinja    │   Cross-DB     │
- Macros   │ Helpers   │    Macros      │
- ┴───────────┴────────────────┘

**┌─────────────────────────────────────────┐**
- Database Adapter Layer          │
- ┬───────────┬────────────────┤
- Postgres  │ Snowflake │    BigQuery    │
- ┴───────────┴────────────────┘

##### Key Elements to Highlight

- Blue: Core macro functionality
- Green: Integration components
- Yellow: Database adapters
- Gray: Testing and utilities

b. Connection Types:

##### Important Relationships

- How the dispatch system allows for database-specific implementations
- Integration testing framework's connection to each macro category
- Demonstrate how the Jinja helpers support other macro types
- Relationship between generic tests and SQL generators

##### Additional Details

- The different macro categories and their purposes
- The dispatch system's role in cross-database compatibility
- The integration testing framework's coverage
- The relationship with dbt Core

##### Implementation Notes

- Use clear, hierarchical layout
- Version information
- Supported database adapters
- Test coverage
- Document macro dependencies

##### Technical Specifications

A sidebar or footer with:
- Supported dbt versions
- Required dependencies
- Minimum database versions
- Installation requirements

Remember to maintain a balance between detail and clarity, focusing on the architectural relationships rather than individual macro implementations. This will help users understand both the high-level structure and the practical application of the package.

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph Project
        dbtProject[dbt Project]
    end

    subgraph Core["Core Functionality"]
        direction TB
        macros[dbt-utils Macros]
        
        subgraph Categories["Macro Categories"]
            direction TB
            genericTests["Generic Tests"]:::test
            sqlGen["SQL Generators"]:::sql
            introMacros["Introspective Macros"]:::macro
            webMacros["Web Macros"]:::web
            jinjaHelpers["Jinja Helpers"]:::helper
            crossDB["Cross-DB Macros"]:::crossdb
        end
        
        macros --> |contains| Categories
    end

    subgraph Testing["Testing Framework"]
        direction TB
        testFramework["Integration Testing Framework"]:::testing
        testModels["Test Models"]:::model
        testData["Test Data"]:::data
        cicd["CI/CD Pipeline"]:::cicd
    end

    subgraph DB["Database Layer"]
        direction TB
        dbAdapters["Database Adapters"]:::db
        postgres["PostgreSQL"]:::db
        snowflake["Snowflake"]:::db
        bigquery["BigQuery"]:::db
        
        dbAdapters --> postgres
        dbAdapters --> snowflake
        dbAdapters --> bigquery
    end

    dbtProject --> macros
    macros --> dbAdapters
    Categories --> testFramework
    testFramework --> testModels
    testFramework --> testData
    testFramework --> cicd

    %% Click events for component mapping
    click genericTests "https://github.com/dbt-labs/dbt-utils/tree/main/macros/generic_tests/"
    click sqlGen "https://github.com/dbt-labs/dbt-utils/tree/main/macros/sql/"
    click webMacros "https://github.com/dbt-labs/dbt-utils/tree/main/macros/web/"
    click jinjaHelpers "https://github.com/dbt-labs/dbt-utils/tree/main/macros/jinja_helpers/"
    click dbAdapters "https://github.com/dbt-labs/dbt-utils/tree/main/integration_tests/.env/"
    click testFramework "https://github.com/dbt-labs/dbt-utils/tree/main/integration_tests/tests/"
    click crossDB "https://github.com/dbt-labs/dbt-utils/blob/main/docs/decisions/adr-0002-cross-database-utils.md"
    click testModels "https://github.com/dbt-labs/dbt-utils/tree/main/integration_tests/models/"
    click testData "https://github.com/dbt-labs/dbt-utils/tree/main/integration_tests/data/"
    click cicd "https://github.com/dbt-labs/dbt-utils/blob/main/.github/workflows/ci.yml"

    %% Styling
    classDef default fill:#f9f9f9,stroke:#333,stroke-width:1px
    classDef test fill:#e1f5fe,stroke:#0288d1
    classDef sql fill:#e8f5e9,stroke:#388e3c
    classDef macro fill:#fff3e0,stroke:#f57c00
    classDef web fill:#fce4ec,stroke:#c2185b
    classDef helper fill:#f3e5f5,stroke:#7b1fa2
    classDef crossdb fill:#fff8e1,stroke:#ffa000
    classDef testing fill:#efebe9,stroke:#5d4037
    classDef model fill:#e0f2f1,stroke:#00796b
    classDef data fill:#e8eaf6,stroke:#3f51b5
    classDef cicd fill:#ffebee,stroke:#c62828
    classDef db fill:#e3f2fd,stroke:#1976d2

    %% Legend
    subgraph Legend
        direction TB
        test1[Generic Tests]:::test
        sql1[SQL Operations]:::sql
        macro1[Core Macros]:::macro
        web1[Web Features]:::web
        helper1[Helpers]:::helper
        crossdb1[Cross-DB]:::crossdb
        testing1[Testing]:::testing
        db1[Databases]:::db
    end
```
