How haiderabbasi6357-cloud/project-end-to-end-dbt-elt-data-warehouse-on-databricks works
This repository is a dbt e-commerce analytics pipeline for Databricks. Its user-to-result workflow loads CSV seeds into Bronze, transforms data through Silver staging and intermediate models, and builds Gold facts and dimensions; a Type 2 customer snapshot preserves history. The README also documents incremental facts, data-quality tests, exposures, run hooks, and a Databricks job. Source excerpts were not provided, so relationships below rely on README statements; model-level SQL dependencies are not independently verified.
Main entry points: intermediate Intermediate views · dbt_shopflow_job.yml dbt Databricks job · _exposures.yml Data exposures · seeds Raw CSV seeds
Components 15
Ingestion
- Raw CSV seeds
seeds - Bronze data
Silver transformations
- Staging views
models/staging - Intermediate views
models/intermediate
Gold analytics
- Customer SCD2
snapshots/scd_customers.sql - Orders fact
models/marts/core/fct_orders.sql - Order items fact
models/marts/core/fct_order_items.sql - Product dimension
models/marts/core/dim_products.sql - Data exposures
models/marts/core/_exposures.yml
Runtime and quality
- Data quality tests
tests - Run hooks
dbt_project.yml - Schema naming
macros/generate_schema_name.sql - dbt Databricks job
resources/dbt_shopflow_job.yml
Other components
- Analytics user
- Databricks
Connections 18
- Analytics user → dbt Databricks job: runs job
README.md - Analytics user → Raw CSV seeds: loads data
README.md - Raw CSV seeds → Bronze data: loads into
README.md - Bronze data → Staging views: stages raw data
README.md - Staging views → Intermediate views: transforms further
- Intermediate views → Orders fact: builds marts
- Intermediate views → Order items fact: builds marts
- Intermediate views → Product dimension: builds marts
- Raw CSV seeds → Customer SCD2: snapshots customers
README.md - dbt Databricks job → Databricks: runs dbt
README.md - Orders fact → Data quality tests: tests results
README.md - Order items fact → Data quality tests: tests results
README.md - Customer SCD2 → Data exposures: documents consumers
- Orders fact → Data exposures: documents consumers
README.md - Order items fact → Data exposures: documents consumers
README.md - Product dimension → Data exposures: documents consumers
README.md - dbt Databricks job → Run hooks: runs hooks
README.md - dbt Databricks job → Schema naming: uses naming