# dbt-labs/jaffle-shop architecture

> Architecture diagram of the GitHub repository dbt-labs/jaffle-shop, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dbt-labs/jaffle-shop
- Repository: https://github.com/dbt-labs/jaffle-shop

## Overview

##### Project type and purpose
- This is a data-transformation (“ELT”) project built with dbt (data build tool).
- Its purpose is to ingest raw CSV data (“jaffle-data” or S3) into a warehouse, apply staging transformations, and build presentation-ready marts.
- It’s an open-source learning project (the “Jaffle Shop”) demonstrating dbt Cloud workflows, CI/CD, environments, and dbt model organization.

##### File structure
- Top-level config:
- dbt_project.yml (defines project name, model folders, seed paths)
- packages.yml (external dbt packages)
- Taskfile.yml (automation tasks)
- GitHub workflows under .github/workflows (CI, CD for prod/staging)
- Data sources:
- jaffle-data/ (example CSV seeds)
- seeds/ (empty by default, can be used for controlled reference data)
- Macros: reusable SQL snippets in macros/
- Models:
- models/staging/ (one-to-one raw → stg_ tables)
- models/marts/ (business-centric tables, metrics, time spine)
- Tests: data-tests/ and analyses/ hold tests and exploratory queries
- CI/CD scripts: workflows trigger dbt Cloud runs or CLI commands via dbt Cloud CLI scripts.

##### README for additional insights
- Describes two environments: “staging” branch and “main” branch tied to respective dbt Cloud environments/schemas.
- Outlines how data is loaded (dbt seed for CSV, S3 copy, or `jafgen` tool).
- Highlights UI (dbt Cloud IDE) vs. CLI workflows.
- Mentions lineage (DAG) exploration in dbt Cloud Explorer.

## Diagram (Mermaid)

```mermaid
flowchart TD
  subgraph "Local Development"
    CLI["dbt CLI"]:::exec
    Taskfile["Taskfile.yml"]:::config
    Precommit[".pre-commit-config.yaml"]:::config
    SQLFluffCfg[".sqlfluffignore"]:::config
  end

  subgraph "Version Control & CI/CD"
    Repo["GitHub Repo"]:::exec
    CI["CI (.github/workflows/ci.yml)"]:::exec
    CDStg["CD Staging (.github/workflows/cd_staging.yml)"]:::exec
    CDProd["CD Prod (.github/workflows/cd_prod.yml)"]:::exec
    Helper["dbt Cloud Run Script"]:::exec
  end

  subgraph "dbt Cloud"
    IDE["dbt Cloud IDE"]:::exec
    EnvStg["Staging Environment"]:::exec
    EnvProd["Prod Environment"]:::exec
    JobStg["Staging Job"]:::exec
    JobProd["Prod Job"]:::exec
  end

  subgraph "dbt Project"
    Config["dbt_project.yml"]:::config
    Packages["packages.yml"]:::config
    Lockfile["package-lock.yml"]:::config

    subgraph "Macros"
      Macro1["cents_to_dollars.sql"]:::code
      Macro2["generate_schema_name.sql"]:::code
    end

    subgraph "Models - Staging"
      SRC["__sources.yml"]:::staging
      STG_Cust["stg_customers.sql"]:::staging
      STG_Loc["stg_locations.sql"]:::staging
      STG_Items["stg_order_items.sql"]:::staging
      STG_Ord["stg_orders.sql"]:::staging
      STG_Prod["stg_products.sql"]:::staging
      STG_Supp["stg_supplies.sql"]:::staging
    end

    subgraph "Models - Marts"
      MART_Cust["customers.sql"]:::marts
      MART_Loc["locations.sql"]:::marts
      MART_Items["order_items.sql"]:::marts
      MART_Ord["orders.sql"]:::marts
      MART_Prod["products.sql"]:::marts
      MART_Supp["supplies.sql"]:::marts
      TimeSpine["metricflow_time_spine.sql"]:::marts
    end

    subgraph "Testing & Analysis"
      DataTests["data-tests/"]:::code
      Analyses["analyses/"]:::code
    end
  end

  subgraph "Data Warehouse"
    RAW["raw schema"]:::warehouse
    STG_SCHEMA["staging schema"]:::warehouse
    MART_SCHEMA["marts schema"]:::warehouse
  end

  subgraph "Data Sources"
    CSV1["raw_customers.csv"]:::data
    CSV2["raw_items.csv"]:::data
    CSV3["raw_orders.csv"]:::data
    CSV4["raw_products.csv"]:::data
    CSV5["raw_stores.csv"]:::data
    CSV6["raw_supplies.csv"]:::data
    SEEDS["seeds/"]:::data
  end

  CLI -->|runs tasks| Taskfile
  CLI -->|runs dbt| Config
  Taskfile -->|executes| Config

  Repo -->|push| CI
  Repo -->|push| CDStg
  Repo -->|merge| CDProd
  CI -->|triggers| JobStg
  CDStg -->|triggers| JobStg
  CDProd -->|triggers| JobProd

  JobStg -->|runs seed & raw| RAW
  JobStg -->|build staging| STG_SCHEMA
  JobStg -->|build marts| MART_SCHEMA
  JobProd -->|runs seed & raw| RAW
  JobProd -->|build staging| STG_SCHEMA
  JobProd -->|build marts| MART_SCHEMA

  CLI -->|invokes| RAW
  CLI -->|invokes| STG_SCHEMA
  CLI -->|invokes| MART_SCHEMA

  CSV1 --> RAW
  CSV2 --> RAW
  CSV3 --> RAW
  CSV4 --> RAW
  CSV5 --> RAW
  CSV6 --> RAW
  SEEDS --> RAW

  RAW --> STG_Cust
  RAW --> STG_Loc
  RAW --> STG_Items
  RAW --> STG_Ord
  RAW --> STG_Prod
  RAW --> STG_Supp

  STG_Cust --> MART_Cust
  STG_Loc --> MART_Loc
  STG_Items --> MART_Items
  STG_Ord --> MART_Ord
  STG_Prod --> MART_Prod
  STG_Supp --> MART_Supp

  Macro1 -.-> STG_Cust
  Macro2 -.-> STG_Cust

  classDef data fill:#e5f5e0 stroke:#31b03c
  classDef staging fill:#e0f0ff stroke:#3182bd
  classDef marts fill:#fff0e0 stroke:#dd6b20
  classDef exec fill:#f5e0ff stroke:#9f4ecb
  classDef config fill:#f0f0f0 stroke:#888888
  classDef code fill:#f8f8f8 stroke:#bbbbbb
  classDef warehouse fill:#f0f0f5 stroke:#8884aa

  click CI "https://github.com/dbt-labs/jaffle-shop/blob/main/.github/workflows/ci.yml"
  click CDStg "https://github.com/dbt-labs/jaffle-shop/blob/main/.github/workflows/cd_staging.yml"
  click CDProd "https://github.com/dbt-labs/jaffle-shop/blob/main/.github/workflows/cd_prod.yml"
  click Helper "https://github.com/dbt-labs/jaffle-shop/blob/main/.github/workflows/scripts/dbt_cloud_run_job.py"
  click Config "https://github.com/dbt-labs/jaffle-shop/blob/main/dbt_project.yml"
  click Packages "https://github.com/dbt-labs/jaffle-shop/blob/main/packages.yml"
  click Lockfile "https://github.com/dbt-labs/jaffle-shop/blob/main/package-lock.yml"
  click Taskfile "https://github.com/dbt-labs/jaffle-shop/blob/main/Taskfile.yml"
  click Precommit "https://github.com/dbt-labs/jaffle-shop/blob/main/.pre-commit-config.yaml"
  click SQLFluffCfg "https://github.com/dbt-labs/jaffle-shop/blob/main/.sqlfluffignore"
  click Macro1 "https://github.com/dbt-labs/jaffle-shop/blob/main/macros/cents_to_dollars.sql"
  click Macro2 "https://github.com/dbt-labs/jaffle-shop/blob/main/macros/generate_schema_name.sql"
  click STG_Cust "https://github.com/dbt-labs/jaffle-shop/blob/main/models/staging/stg_customers.sql"
  click STG_Loc "https://github.com/dbt-labs/jaffle-shop/blob/main/models/staging/stg_locations.sql"
  click STG_Items "https://github.com/dbt-labs/jaffle-shop/blob/main/models/staging/stg_order_items.sql"
  click STG_Ord "https://github.com/dbt-labs/jaffle-shop/blob/main/models/staging/stg_orders.sql"
  click STG_Prod "https://github.com/dbt-labs/jaffle-shop/blob/main/models/staging/stg_products.sql"
  click STG_Supp "https://github.com/dbt-labs/jaffle-shop/blob/main/models/staging/stg_supplies.sql"
  click MART_Cust "https://github.com/dbt-labs/jaffle-shop/blob/main/models/marts/customers.sql"
  click MART_Loc "https://github.com/dbt-labs/jaffle-shop/blob/main/models/marts/locations.sql"
  click MART_Items "https://github.com/dbt-labs/jaffle-shop/blob/main/models/marts/order_items.sql"
  click MART_Ord "https://github.com/dbt-labs/jaffle-shop/blob/main/models/marts/orders.sql"
  click MART_Prod "https://github.com/dbt-labs/jaffle-shop/blob/main/models/marts/products.sql"
  click MART_Supp "https://github.com/dbt-labs/jaffle-shop/blob/main/models/marts/supplies.sql"
  click TimeSpine "https://github.com/dbt-labs/jaffle-shop/blob/main/models/marts/metricflow_time_spine.sql"
  click DataTests "https://github.com/dbt-labs/jaffle-shop/tree/main/data-tests/"
  click Analyses "https://github.com/dbt-labs/jaffle-shop/tree/main/analyses/"
  click SEEDS "https://github.com/dbt-labs/jaffle-shop/tree/main/seeds/"
  click CSV1 "https://github.com/dbt-labs/jaffle-shop/blob/main/jaffle-data/raw_customers.csv"
  click CSV2 "https://github.com/dbt-labs/jaffle-shop/blob/main/jaffle-data/raw_items.csv"
  click CSV3 "https://github.com/dbt-labs/jaffle-shop/blob/main/jaffle-data/raw_orders.csv"
  click CSV4 "https://github.com/dbt-labs/jaffle-shop/blob/main/jaffle-data/raw_products.csv"
  click CSV5 "https://github.com/dbt-labs/jaffle-shop/blob/main/jaffle-data/raw_stores.csv"
  click CSV6 "https://github.com/dbt-labs/jaffle-shop/blob/main/jaffle-data/raw_supplies.csv"
```
