# mridul98/dbt_unit_test_best_practice architecture

> Architecture diagram of the GitHub repository mridul98/dbt_unit_test_best_practice, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mridul98/dbt_unit_test_best_practice
- Repository: https://github.com/mridul98/dbt_unit_test_best_practice

## Overview

##### Main Components
- Docker Environment – Represent the Docker environment as a container group that includes:
- PostgreSQL Database Container (referred to as “db” in the README)
- Adminer Client Container for UI-based database management.
- DBT Projects – Clearly identify two separate DBT projects:
- dbt_unit_testing: The original implementation (with directories such as models/iot, macros, seeds, snapshots, tests, and analyses).
- dbt_unit_testing_refactored: The refactored version with a more modular structure (including subdirectories like staging, intermediate, and reporting under models/iot and additional macros under macros/iot).
- Data Schema Setup – Show the DDL (ddl.txt) as the starting point for setting up the database tables.
- Python Environment – Include the setup of the Python environment (requirements.txt and virtual environment) that supports DBT operations.

##### Relationships and Data Flow
- That the Docker Compose flow is used to initialize and run the PostgreSQL (and Adminer) containers.
- In the refactored DBT project, illustrate the internal processing stages:
- Staging models load raw data from the sources.
- Intermediate models process the data.
- Reporting models generate final metrics or insights.
- How the DBT macros (and tests) are used to support transformations and ensure unit testing of models throughout the data pipeline.

##### Architectural Patterns & Design Principles
- Separation of concerns:
- Docker handles environment orchestration.
- DBT projects manage transformation logic and testing.
- SQL (ddl.txt) defines schema setup.
- That the modular directory structure in the refactored DBT project suggests a layered approach (staging > intermediate > reporting), which is useful for improved maintainability and clarity.
- That test-driven development is at the core, as unit tests and tests directories indicate the integration of automated testing in data pipeline development.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Docker Environment
    subgraph "Docker Environment"
        DC["Docker Compose"]:::infrastructure
        click DC "https://github.com/mridul98/dbt_unit_test_best_practice/blob/master/docker-compose.yaml"
    end

    %% Containers
    subgraph "Containers"
        PG["PostgreSQL DB Container"]:::infrastructure
        ADMINER["Adminer Container"]:::infrastructure
    end

    DC -->|"initializes"| PG
    DC -->|"initializes"| ADMINER

    %% Schema Setup
    ddl["DDL Script (ddl.txt)"]:::infrastructure
    click ddl "https://github.com/mridul98/dbt_unit_test_best_practice/blob/master/ddl.txt"
    ddl -->|"creates_schema"| PG

    %% Python Environment
    py["Python Environment (requirements.txt)"]:::application
    click py "https://github.com/mridul98/dbt_unit_test_best_practice/blob/master/requirements.txt"

    %% DBT Projects
    subgraph "DBT Projects"
        orig["DBT Project (Original)"]:::application
        click orig "https://github.com/mridul98/dbt_unit_test_best_practice/tree/master/dbt_unit_testing/"

        refac["DBT Project (Refactored)"]:::application
        click refac "https://github.com/mridul98/dbt_unit_test_best_practice/tree/master/dbt_unit_testing_refactored/"

        %% Refactored DBT Models Breakdown
        subgraph "Refactored DBT Models"
            staging["Staging Models"]:::application
            intermediate["Intermediate Models"]:::application
            reporting["Reporting Models"]:::application
        end
        refac -->|"loads_to"| staging
        staging -->|"processes_into"| intermediate
        intermediate -->|"generates"| reporting
    end

    %% Python supports DBT operations
    py --> orig
    py --> refac

    %% DBT Projects transform and load data into PostgreSQL
    orig -->|"transforms_data"| PG
    refac -->|"transforms_data"| PG

    %% Styles
    classDef infrastructure fill:#ADD8E6,stroke:#000,stroke-width:2px;
    classDef application fill:#90EE90,stroke:#000,stroke-width:2px;
```
