# irajput215/zomato-ai-data-platform architecture

> Architecture diagram of the GitHub repository irajput215/zomato-ai-data-platform, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/irajput215/zomato-ai-data-platform
- Repository: https://github.com/irajput215/zomato-ai-data-platform

## Overview

This is a batch analytics platform for Zomato-style food-delivery data. Its main workflow generates CSVs, uploads them to S3, loads Snowflake RAW tables, transforms them with dbt into staging and marts, and runs as a daily Airflow DAG. Separate AI capabilities enrich reviews, support review-grounded chat and warehouse text-to-SQL, and display operational marts. Several dbt and DAG details are documented but their implementation files were not excerpted; those links cite the README where documented.

Main entry points: [`snowflake/01_setup.sql`](https://github.com/irajput215/zomato-ai-data-platform/blob/main/snowflake/01_setup.sql) (Snowflake warehouse), [`airflow/dags/zomato_batch.py`](https://github.com/irajput215/zomato-ai-data-platform/blob/main/airflow/dags/zomato_batch.py) (Daily batch DAG), [`ai/enrich_reviews.py`](https://github.com/irajput215/zomato-ai-data-platform/blob/main/ai/enrich_reviews.py) (Review enrichment), [`ai/common.py`](https://github.com/irajput215/zomato-ai-data-platform/blob/main/ai/common.py) (AI clients)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Data preparation"]
  node_dimension_generator["Dimension generator"]
  node_fact_generator["Fact generator<br/>[generate_facts.py]"]
  node_generator_config["Generator config<br/>[config.py]"]
  node_s3_uploader["S3 uploader<br/>[upload_to_s3.py]"]
  node_s3_verifier["S3 verifier<br/>[verify_s3.py]"]
  node_s3_lake[("S3 data lake")]
end

subgraph group_warehouse["Warehouse pipeline"]
  node_snowflake_load["Snowflake RAW load<br/>[05_copy_into.sql]"]
  node_staging["dbt staging"]
  node_marts["dbt marts"]
  node_snapshot["Restaurant SCD2 snapshot"]
  node_warehouse_store[("Snowflake warehouse<br/>[01_setup.sql]")]
end

subgraph group_orchestration["Orchestration"]
  node_daily_dag["Daily batch DAG<br/>[zomato_batch.py]"]
end

subgraph group_ai["AI analytics"]
  node_ai_common["AI clients<br/>[common.py]"]
  node_enrichment["Review enrichment<br/>[enrich_reviews.py]"]
  node_rag["Review RAG chat<br/>[rag_chat.py]"]
  node_text_sql["Text-to-SQL<br/>[text_to_sql.py]"]
  node_dashboard["Operations dashboard<br/>[dashboard.py]"]
end

node_openai{{"OpenAI"}}

node_dimension_generator -->|"uses settings"| node_generator_config
node_fact_generator -->|"uses settings"| node_generator_config
node_fact_generator -->|"reads dimensions"| node_dimension_generator
node_s3_uploader -->|"uploads CSVs"| node_s3_lake
node_s3_verifier -->|"checks objects"| node_s3_lake
node_s3_lake -->|"loads staged CSVs"| node_snowflake_load
node_snowflake_load -->|"writes RAW"| node_warehouse_store
node_warehouse_store -->|"transforms sources"| node_staging
node_staging -->|"builds models"| node_marts
node_marts -->|"tracks history"| node_snapshot
node_snapshot -.->|"stores history"| node_warehouse_store
node_daily_dag -->|"runs COPY INTO"| node_snowflake_load
node_daily_dag -->|"runs dbt build"| node_staging
node_daily_dag -->|"runs snapshot"| node_snapshot
node_daily_dag -->|"runs enrichment"| node_enrichment
node_daily_dag -->|"builds AI mart"| node_marts
node_enrichment -->|"uses clients"| node_ai_common
node_rag -->|"uses clients"| node_ai_common
node_text_sql -->|"uses clients"| node_ai_common
node_dashboard -->|"uses clients"| node_ai_common
node_enrichment -->|"reads and writes reviews"| node_warehouse_store
node_rag -->|"reads review data"| node_warehouse_store
node_text_sql -->|"queries schema and data"| node_warehouse_store
node_dashboard -->|"reads marts"| node_warehouse_store
node_enrichment -->|"enriches reviews"| node_openai
node_rag -->|"embeds and answers"| node_openai
node_text_sql -->|"generates SQL"| node_openai

click node_dimension_generator "https://github.com/irajput215/zomato-ai-data-platform/blob/main/data/generator/generate_dimensions.py"
click node_fact_generator "https://github.com/irajput215/zomato-ai-data-platform/blob/main/data/generator/generate_facts.py"
click node_generator_config "https://github.com/irajput215/zomato-ai-data-platform/blob/main/data/generator/config.py"
click node_s3_uploader "https://github.com/irajput215/zomato-ai-data-platform/blob/main/ingestion/upload_to_s3.py"
click node_s3_verifier "https://github.com/irajput215/zomato-ai-data-platform/blob/main/ingestion/verify_s3.py"
click node_snowflake_load "https://github.com/irajput215/zomato-ai-data-platform/blob/main/snowflake/05_copy_into.sql"
click node_staging "https://github.com/irajput215/zomato-ai-data-platform/tree/main/zomato/models/staging"
click node_marts "https://github.com/irajput215/zomato-ai-data-platform/tree/main/zomato/models/marts"
click node_snapshot "https://github.com/irajput215/zomato-ai-data-platform/blob/main/zomato/snapshots/dim_restaurants_snapshot.sql"
click node_warehouse_store "https://github.com/irajput215/zomato-ai-data-platform/blob/main/snowflake/01_setup.sql"
click node_daily_dag "https://github.com/irajput215/zomato-ai-data-platform/blob/main/airflow/dags/zomato_batch.py"
click node_ai_common "https://github.com/irajput215/zomato-ai-data-platform/blob/main/ai/common.py"
click node_enrichment "https://github.com/irajput215/zomato-ai-data-platform/blob/main/ai/enrich_reviews.py"
click node_rag "https://github.com/irajput215/zomato-ai-data-platform/blob/main/ai/rag_chat.py"
click node_text_sql "https://github.com/irajput215/zomato-ai-data-platform/blob/main/ai/text_to_sql.py"
click node_dashboard "https://github.com/irajput215/zomato-ai-data-platform/blob/main/ai/dashboard.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_dimension_generator,node_fact_generator,node_generator_config,node_s3_uploader,node_s3_verifier,node_s3_lake toneBlue
class node_snowflake_load,node_staging,node_marts,node_snapshot,node_warehouse_store toneAmber
class node_daily_dag toneMint
class node_ai_common,node_enrichment,node_rag,node_text_sql,node_dashboard toneRose
class node_openai toneIndigo
```

## Components

### Data preparation

- [Dimension generator](https://github.com/irajput215/zomato-ai-data-platform/blob/main/data/generator/generate_dimensions.py): `data/generator/generate_dimensions.py`
- [Fact generator](https://github.com/irajput215/zomato-ai-data-platform/blob/main/data/generator/generate_facts.py): `data/generator/generate_facts.py`
- [Generator config](https://github.com/irajput215/zomato-ai-data-platform/blob/main/data/generator/config.py): `data/generator/config.py`
- [S3 uploader](https://github.com/irajput215/zomato-ai-data-platform/blob/main/ingestion/upload_to_s3.py): `ingestion/upload_to_s3.py`
- [S3 verifier](https://github.com/irajput215/zomato-ai-data-platform/blob/main/ingestion/verify_s3.py): `ingestion/verify_s3.py`
- **S3 data lake**

### Warehouse pipeline

- [Snowflake RAW load](https://github.com/irajput215/zomato-ai-data-platform/blob/main/snowflake/05_copy_into.sql): `snowflake/05_copy_into.sql`
- [dbt staging](https://github.com/irajput215/zomato-ai-data-platform/tree/main/zomato/models/staging): `zomato/models/staging`
- [dbt marts](https://github.com/irajput215/zomato-ai-data-platform/tree/main/zomato/models/marts): `zomato/models/marts`
- [Restaurant SCD2 snapshot](https://github.com/irajput215/zomato-ai-data-platform/blob/main/zomato/snapshots/dim_restaurants_snapshot.sql): `zomato/snapshots/dim_restaurants_snapshot.sql`
- [Snowflake warehouse](https://github.com/irajput215/zomato-ai-data-platform/blob/main/snowflake/01_setup.sql): `snowflake/01_setup.sql`

### Orchestration

- [Daily batch DAG](https://github.com/irajput215/zomato-ai-data-platform/blob/main/airflow/dags/zomato_batch.py): `airflow/dags/zomato_batch.py`

### AI analytics

- [AI clients](https://github.com/irajput215/zomato-ai-data-platform/blob/main/ai/common.py): `ai/common.py`
- [Review enrichment](https://github.com/irajput215/zomato-ai-data-platform/blob/main/ai/enrich_reviews.py): `ai/enrich_reviews.py`
- [Review RAG chat](https://github.com/irajput215/zomato-ai-data-platform/blob/main/ai/rag_chat.py): `ai/rag_chat.py`
- [Text-to-SQL](https://github.com/irajput215/zomato-ai-data-platform/blob/main/ai/text_to_sql.py): `ai/text_to_sql.py`
- [Operations dashboard](https://github.com/irajput215/zomato-ai-data-platform/blob/main/ai/dashboard.py): `ai/dashboard.py`

### Other components

- **OpenAI**

## Connections

- Dimension generator → Generator config: uses settings (evidence: `data/generator/generate_dimensions.py`)
- Fact generator → Generator config: uses settings (evidence: `data/generator/generate_facts.py`)
- Fact generator → Dimension generator: reads dimensions (evidence: `data/generator/generate_facts.py`)
- S3 uploader → S3 data lake: uploads CSVs (evidence: `ingestion/upload_to_s3.py`)
- S3 verifier → S3 data lake: checks objects (evidence: `ingestion/verify_s3.py`)
- S3 data lake → Snowflake RAW load: loads staged CSVs (evidence: `README.md`)
- Snowflake RAW load → Snowflake warehouse: writes RAW (evidence: `README.md`)
- Snowflake warehouse → dbt staging: transforms sources (evidence: `README.md`)
- dbt staging → dbt marts: builds models (evidence: `README.md`)
- dbt marts → Restaurant SCD2 snapshot: tracks history (evidence: `README.md`)
- Restaurant SCD2 snapshot → Snowflake warehouse: stores history
- Daily batch DAG → Snowflake RAW load: runs COPY INTO (evidence: `airflow/dags/zomato_batch.py`)
- Daily batch DAG → dbt staging: runs dbt build (evidence: `README.md`)
- Daily batch DAG → Restaurant SCD2 snapshot: runs snapshot (evidence: `README.md`)
- Daily batch DAG → Review enrichment: runs enrichment (evidence: `README.md`)
- Daily batch DAG → dbt marts: builds AI mart (evidence: `README.md`)
- Review enrichment → AI clients: uses clients (evidence: `ai/enrich_reviews.py`)
- Review RAG chat → AI clients: uses clients (evidence: `ai/rag_chat.py`)
- Text-to-SQL → AI clients: uses clients (evidence: `ai/text_to_sql.py`)
- Operations dashboard → AI clients: uses clients (evidence: `ai/dashboard.py`)
- Review enrichment → Snowflake warehouse: reads and writes reviews (evidence: `ai/enrich_reviews.py`)
- Review RAG chat → Snowflake warehouse: reads review data (evidence: `ai/rag_chat.py`)
- Text-to-SQL → Snowflake warehouse: queries schema and data (evidence: `ai/text_to_sql.py`)
- Operations dashboard → Snowflake warehouse: reads marts (evidence: `ai/dashboard.py`)
- Review enrichment → OpenAI: enriches reviews (evidence: `README.md`)
- Review RAG chat → OpenAI: embeds and answers (evidence: `README.md`)
- Text-to-SQL → OpenAI: generates SQL (evidence: `ai/text_to_sql.py`)
