How irajput215/zomato-ai-data-platform works
This is a batch analytics platform for Zomato-style food-delivery data. Its main workflow generates CSVs, uploads them to S3, loads Snowflake RAW tables, transforms them with dbt into staging and marts, and runs as a daily Airflow DAG. Separate AI capabilities enrich reviews, support review-grounded chat and warehouse text-to-SQL, and display operational marts. Several dbt and DAG details are documented but their implementation files were not excerpted; those links cite the README where documented.
Main entry points: 01_setup.sql Snowflake warehouse · zomato_batch.py Daily batch DAG · enrich_reviews.py Review enrichment · common.py AI clients
Components 18
Data preparation
- Dimension generator
data/generator/generate_dimensions.py - Fact generator
data/generator/generate_facts.py - Generator config
data/generator/config.py - S3 uploader
ingestion/upload_to_s3.py - S3 verifier
ingestion/verify_s3.py - S3 data lake
Warehouse pipeline
- Snowflake RAW load
snowflake/05_copy_into.sql - dbt staging
zomato/models/staging - dbt marts
zomato/models/marts - Restaurant SCD2 snapshot
zomato/snapshots/dim_restaurants_snapshot.sql - Snowflake warehouse
snowflake/01_setup.sql
Orchestration
- Daily batch DAG
airflow/dags/zomato_batch.py
AI analytics
- AI clients
ai/common.py - Review enrichment
ai/enrich_reviews.py - Review RAG chat
ai/rag_chat.py - Text-to-SQL
ai/text_to_sql.py - Operations dashboard
ai/dashboard.py
Other components
- OpenAI
Connections 27
- Dimension generator → Generator config: uses settings
data/generator/generate_dimensions.py - Fact generator → Generator config: uses settings
data/generator/generate_facts.py - Fact generator → Dimension generator: reads dimensions
data/generator/generate_facts.py - S3 uploader → S3 data lake: uploads CSVs
ingestion/upload_to_s3.py - S3 verifier → S3 data lake: checks objects
ingestion/verify_s3.py - S3 data lake → Snowflake RAW load: loads staged CSVs
README.md - Snowflake RAW load → Snowflake warehouse: writes RAW
README.md - Snowflake warehouse → dbt staging: transforms sources
README.md - dbt staging → dbt marts: builds models
README.md - dbt marts → Restaurant SCD2 snapshot: tracks history
README.md - Restaurant SCD2 snapshot → Snowflake warehouse: stores history
- Daily batch DAG → Snowflake RAW load: runs COPY INTO
airflow/dags/zomato_batch.py - Daily batch DAG → dbt staging: runs dbt build
README.md - Daily batch DAG → Restaurant SCD2 snapshot: runs snapshot
README.md - Daily batch DAG → Review enrichment: runs enrichment
README.md - Daily batch DAG → dbt marts: builds AI mart
README.md - Review enrichment → AI clients: uses clients
ai/enrich_reviews.py - Review RAG chat → AI clients: uses clients
ai/rag_chat.py - Text-to-SQL → AI clients: uses clients
ai/text_to_sql.py - Operations dashboard → AI clients: uses clients
ai/dashboard.py - Review enrichment → Snowflake warehouse: reads and writes reviews
ai/enrich_reviews.py - Review RAG chat → Snowflake warehouse: reads review data
ai/rag_chat.py - Text-to-SQL → Snowflake warehouse: queries schema and data
ai/text_to_sql.py - Operations dashboard → Snowflake warehouse: reads marts
ai/dashboard.py - Review enrichment → OpenAI: enriches reviews
README.md - Review RAG chat → OpenAI: embeds and answers
README.md - Text-to-SQL → OpenAI: generates SQL
ai/text_to_sql.py