# khprateek/supply-chain-inventory-optimization-hub architecture

> Architecture diagram of the GitHub repository khprateek/supply-chain-inventory-optimization-hub, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/khprateek/supply-chain-inventory-optimization-hub
- Repository: https://github.com/khprateek/supply-chain-inventory-optimization-hub

## Overview

This repository is an end-to-end supply-chain analytics platform: it generates operational datasets, transforms them into a dimensional warehouse, and exposes measures and dashboards for planners and executives. The principal documented workflow is daily Airflow orchestration from raw-file arrival through BigQuery and dbt to Power BI refresh. A separate PySpark processing path and local Polars marts are also represented. The graph follows the README and sampled files; several dashboard and warehouse capabilities are documented but their runtime implementation was not sampled.

Main entry points: [`orchestration/airflow/dags/dag_supply_chain_daily_elt.py`](https://github.com/khprateek/supply-chain-inventory-optimization-hub/blob/main/orchestration/airflow/dags/dag_supply_chain_daily_elt.py) (Daily ELT DAG), [`python/orchestrator.py`](https://github.com/khprateek/supply-chain-inventory-optimization-hub/blob/main/python/orchestrator.py) (Dataset orchestrator), [`powerbi/model_definitions/model_schema.json`](https://github.com/khprateek/supply-chain-inventory-optimization-hub/blob/main/powerbi/model_definitions/model_schema.json) (Power BI semantic model), [`dbt/models/staging`](https://github.com/khprateek/supply-chain-inventory-optimization-hub/tree/main/dbt/models/staging) (Staging models)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_generation["Data generation"]
  node_generator["Dataset orchestrator<br/>[orchestrator.py]"]
  node_dimensions["Conformed dimensions"]
  node_operational_data["Operational datasets"]
  node_raw_files["Raw Parquet files"]
  node_state[("Incremental state")]
end

subgraph group_warehouse["Warehouse modeling"]
  node_bigquery_raw[("BigQuery raw<br/>[sources.yml]")]
  node_dbt_staging["Staging models"]
  node_dbt_intermediate["Intermediate metrics"]
  node_dbt_marts["Dimensional marts"]
  node_scd2["Product SCD2"]
  node_quality["Data quality tests"]
  node_bigquery[("Analytical warehouse")]
  node_pyspark_stage["PySpark staging<br/>[01_staging.py]"]
  node_pyspark_facts["PySpark fact builds<br/>[03_fact_tables.py]"]
  node_bq_io["Spark BigQuery I/O<br/>[bigquery_io.py]"]
  node_polars_marts["Local Polars marts<br/>[01_build_marts.py]"]
end

subgraph group_orchestration["Pipeline orchestration"]
  node_airflow["Daily ELT DAG"]
  node_gcs[("GCS landing")]
end

subgraph group_analytics["Analytics and reporting"]
  node_semantic["Power BI semantic model<br/>[model_schema.json]"]
  node_measures["DAX measure libraries"]
  node_security["Dynamic row security<br/>[rls_design.md]"]
  node_dashboards["Report pages"]
  node_powerbi_service["Power BI service"]
end

node_planner(("Supply-chain users"))

node_generator -->|"generates"| node_dimensions
node_generator -->|"generates"| node_operational_data
node_generator -->|"reads and updates"| node_state
node_generator -->|"writes"| node_raw_files
node_airflow -->|"checks arrival"| node_gcs
node_gcs -->|"signals readiness"| node_airflow
node_airflow -->|"syncs partitions"| node_bigquery_raw
node_bigquery_raw -->|"provides sources"| node_dbt_staging
node_dbt_staging -->|"feeds models"| node_dbt_intermediate
node_dbt_intermediate -->|"feeds models"| node_dbt_marts
node_dbt_marts -->|"materializes"| node_bigquery
node_dbt_staging -->|"snapshots products"| node_scd2
node_airflow -->|"runs tests"| node_quality
node_quality -->|"validates"| node_dbt_marts
node_airflow -->|"requests refresh"| node_powerbi_service
node_bigquery -->|"provides warehouse data"| node_semantic
node_measures -->|"defines calculations"| node_semantic
node_security -->|"specifies access rules"| node_semantic
node_semantic -->|"supports reports"| node_dashboards
node_dashboards -->|"presents analytics"| node_planner
node_pyspark_stage -->|"stages inputs for"| node_pyspark_facts
node_pyspark_facts -->|"uses read/write helpers"| node_bq_io
node_bq_io -->|"reads and writes"| node_bigquery
node_polars_marts -.->|"produces local marts"| node_planner

click node_generator "https://github.com/khprateek/supply-chain-inventory-optimization-hub/blob/main/python/orchestrator.py"
click node_dimensions "https://github.com/khprateek/supply-chain-inventory-optimization-hub/blob/main/python/generate_dimensions.py"
click node_operational_data "https://github.com/khprateek/supply-chain-inventory-optimization-hub/tree/main/python"
click node_raw_files "https://github.com/khprateek/supply-chain-inventory-optimization-hub/tree/main/data/raw"
click node_state "https://github.com/khprateek/supply-chain-inventory-optimization-hub/blob/main/python/incremental_state.py"
click node_airflow "https://github.com/khprateek/supply-chain-inventory-optimization-hub/blob/main/orchestration/airflow/dags/dag_supply_chain_daily_elt.py"
click node_bigquery_raw "https://github.com/khprateek/supply-chain-inventory-optimization-hub/blob/main/dbt/models/sources.yml"
click node_dbt_staging "https://github.com/khprateek/supply-chain-inventory-optimization-hub/tree/main/dbt/models/staging"
click node_dbt_intermediate "https://github.com/khprateek/supply-chain-inventory-optimization-hub/tree/main/dbt/models/intermediate"
click node_dbt_marts "https://github.com/khprateek/supply-chain-inventory-optimization-hub/tree/main/dbt/models/marts"
click node_scd2 "https://github.com/khprateek/supply-chain-inventory-optimization-hub/blob/main/dbt/snapshots/snap_dim_product_scd2.sql"
click node_quality "https://github.com/khprateek/supply-chain-inventory-optimization-hub/tree/main/dbt/tests"
click node_pyspark_stage "https://github.com/khprateek/supply-chain-inventory-optimization-hub/blob/main/pyspark_jobs/complex_transforms/01_staging.py"
click node_pyspark_facts "https://github.com/khprateek/supply-chain-inventory-optimization-hub/blob/main/pyspark_jobs/complex_transforms/03_fact_tables.py"
click node_bq_io "https://github.com/khprateek/supply-chain-inventory-optimization-hub/blob/main/pyspark_jobs/utils/bigquery_io.py"
click node_polars_marts "https://github.com/khprateek/supply-chain-inventory-optimization-hub/blob/main/polars_jobs/01_build_marts.py"
click node_semantic "https://github.com/khprateek/supply-chain-inventory-optimization-hub/blob/main/powerbi/model_definitions/model_schema.json"
click node_measures "https://github.com/khprateek/supply-chain-inventory-optimization-hub/tree/main/powerbi/measures"
click node_security "https://github.com/khprateek/supply-chain-inventory-optimization-hub/blob/main/docs/powerbi/rls_design.md"
click node_dashboards "https://github.com/khprateek/supply-chain-inventory-optimization-hub/tree/main/docs/powerbi"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_generator,node_dimensions,node_operational_data,node_raw_files,node_state,node_planner toneBlue
class node_bigquery_raw,node_dbt_staging,node_dbt_intermediate,node_dbt_marts,node_scd2,node_quality,node_bigquery,node_pyspark_stage,node_pyspark_facts,node_bq_io,node_polars_marts toneAmber
class node_airflow,node_gcs toneMint
class node_semantic,node_measures,node_security,node_dashboards,node_powerbi_service toneRose
```

## Components

### Data generation

- [Dataset orchestrator](https://github.com/khprateek/supply-chain-inventory-optimization-hub/blob/main/python/orchestrator.py): `python/orchestrator.py`
- [Conformed dimensions](https://github.com/khprateek/supply-chain-inventory-optimization-hub/blob/main/python/generate_dimensions.py): `python/generate_dimensions.py`
- [Operational datasets](https://github.com/khprateek/supply-chain-inventory-optimization-hub/tree/main/python): `python`
- [Raw Parquet files](https://github.com/khprateek/supply-chain-inventory-optimization-hub/tree/main/data/raw): `data/raw`
- [Incremental state](https://github.com/khprateek/supply-chain-inventory-optimization-hub/blob/main/python/incremental_state.py): `python/incremental_state.py`

### Warehouse modeling

- [BigQuery raw](https://github.com/khprateek/supply-chain-inventory-optimization-hub/blob/main/dbt/models/sources.yml): `dbt/models/sources.yml`
- [Staging models](https://github.com/khprateek/supply-chain-inventory-optimization-hub/tree/main/dbt/models/staging): `dbt/models/staging`
- [Intermediate metrics](https://github.com/khprateek/supply-chain-inventory-optimization-hub/tree/main/dbt/models/intermediate): `dbt/models/intermediate`
- [Dimensional marts](https://github.com/khprateek/supply-chain-inventory-optimization-hub/tree/main/dbt/models/marts): `dbt/models/marts`
- [Product SCD2](https://github.com/khprateek/supply-chain-inventory-optimization-hub/blob/main/dbt/snapshots/snap_dim_product_scd2.sql): `dbt/snapshots/snap_dim_product_scd2.sql`
- [Data quality tests](https://github.com/khprateek/supply-chain-inventory-optimization-hub/tree/main/dbt/tests): `dbt/tests`
- **Analytical warehouse**
- [PySpark staging](https://github.com/khprateek/supply-chain-inventory-optimization-hub/blob/main/pyspark_jobs/complex_transforms/01_staging.py): `pyspark_jobs/complex_transforms/01_staging.py`
- [PySpark fact builds](https://github.com/khprateek/supply-chain-inventory-optimization-hub/blob/main/pyspark_jobs/complex_transforms/03_fact_tables.py): `pyspark_jobs/complex_transforms/03_fact_tables.py`
- [Spark BigQuery I/O](https://github.com/khprateek/supply-chain-inventory-optimization-hub/blob/main/pyspark_jobs/utils/bigquery_io.py): `pyspark_jobs/utils/bigquery_io.py`
- [Local Polars marts](https://github.com/khprateek/supply-chain-inventory-optimization-hub/blob/main/polars_jobs/01_build_marts.py): `polars_jobs/01_build_marts.py`

### Pipeline orchestration

- [Daily ELT DAG](https://github.com/khprateek/supply-chain-inventory-optimization-hub/blob/main/orchestration/airflow/dags/dag_supply_chain_daily_elt.py): `orchestration/airflow/dags/dag_supply_chain_daily_elt.py`
- **GCS landing**

### Analytics and reporting

- [Power BI semantic model](https://github.com/khprateek/supply-chain-inventory-optimization-hub/blob/main/powerbi/model_definitions/model_schema.json): `powerbi/model_definitions/model_schema.json`
- [DAX measure libraries](https://github.com/khprateek/supply-chain-inventory-optimization-hub/tree/main/powerbi/measures): `powerbi/measures`
- [Dynamic row security](https://github.com/khprateek/supply-chain-inventory-optimization-hub/blob/main/docs/powerbi/rls_design.md): `docs/powerbi/rls_design.md`
- [Report pages](https://github.com/khprateek/supply-chain-inventory-optimization-hub/tree/main/docs/powerbi): `docs/powerbi`
- **Power BI service**

### Other components

- **Supply-chain users**

## Connections

- Dataset orchestrator → Conformed dimensions: generates
- Dataset orchestrator → Operational datasets: generates
- Dataset orchestrator → Incremental state: reads and updates
- Dataset orchestrator → Raw Parquet files: writes
- Daily ELT DAG → GCS landing: checks arrival
- GCS landing → Daily ELT DAG: signals readiness
- Daily ELT DAG → BigQuery raw: syncs partitions
- BigQuery raw → Staging models: provides sources
- Staging models → Intermediate metrics: feeds models
- Intermediate metrics → Dimensional marts: feeds models
- Dimensional marts → Analytical warehouse: materializes
- Staging models → Product SCD2: snapshots products
- Daily ELT DAG → Data quality tests: runs tests
- Data quality tests → Dimensional marts: validates
- Daily ELT DAG → Power BI service: requests refresh
- Analytical warehouse → Power BI semantic model: provides warehouse data
- DAX measure libraries → Power BI semantic model: defines calculations
- Dynamic row security → Power BI semantic model: specifies access rules
- Power BI semantic model → Report pages: supports reports
- Report pages → Supply-chain users: presents analytics
- PySpark staging → PySpark fact builds: stages inputs for
- PySpark fact builds → Spark BigQuery I/O: uses read/write helpers
- Spark BigQuery I/O → Analytical warehouse: reads and writes
- Local Polars marts → Supply-chain users: produces local marts