How shrutimokhashi34/banking_data_platform works
This is an end-to-end synthetic banking data platform: it generates domain datasets, ingests daily Azure batches, transforms them through Databricks’ medallion layers, publishes aggregates, and serves analytics through dashboards and a role-aware multi-agent assistant. The graph preserves the documented data stages and the app’s API, authorization, agent, and query layers. Some implementation files were not sampled, so those subsystem relationships are shown only where the README documents them; app internals are otherwise limited to indexed references.
Main entry points: main.py Application API · pipeline_tools.py Pipeline audit tools · daily_ingestion_dag.py Daily orchestration · silver architechture scripts Silver domain checks
Components 22
Synthetic data
- Domain generators
python/generators - Banking models
python/models
Data processing
- Azure raw data
- Daily orchestration
airflow/dags/daily_ingestion_dag.py - Load validation
airflow/ingestion scripts/validation.py - Snowflake staging
- Silver domain checks
databricks/silver/silver architechture scripts - Gold model
databricks/gold/Gold Schema - dim fact tables creation.py - Snowflake aggregates
- Aggregate migration
databricks/Databricks Gold to Snowflake AGG Migration.py
Assistant and analytics
- BigQuery analytics
- Data Studio dashboards
- ADK agent chain
google adk agent chain and app/app/agent.py - Analytics tools
google adk agent chain and app/app/tools - Pipeline agent
google adk agent chain and app/app/agents/data_engineer_agent.py - Pipeline audit tools
google adk agent chain and app/app/tools/pipeline_tools.py
Application access
- Chat interface
google adk agent chain and app/app/frontend/src/BankingChatbotApp.jsx - Application API
google adk agent chain and app/app/api/main.py - Authentication
google adk agent chain and app/app/auth/authentication.py - Authorization
google adk agent chain and app/app/auth/authorization.py
Other components
- Source systems
- Banking user
Connections 19
- Source systems → Azure raw data: land raw data
- Azure raw data → Daily orchestration: schedule loads
- Daily orchestration → Snowflake staging: load staging
- Snowflake staging → Silver domain checks: process layers
- Silver domain checks → Gold model: curate data
- Gold model → Aggregate migration: read tables
databricks/Databricks Gold to Snowflake AGG Migration.py - Aggregate migration → Snowflake aggregates: publish aggregates
databricks/Databricks Gold to Snowflake AGG Migration.py - Snowflake aggregates → BigQuery analytics: serve analytics
- BigQuery analytics → Data Studio dashboards: serve insights
- Banking user → Chat interface: ask questions
- Chat interface → Application API: send requests
- Application API → Authentication: authenticate identity
google adk agent chain and app/app/api/main.py - Application API → ADK agent chain: run conversation
google adk agent chain and app/app/api/main.py - Application API → Analytics tools: execute dashboard queries
google adk agent chain and app/app/api/main.py - ADK agent chain → Pipeline agent: delegate pipeline questions
- Pipeline agent → Pipeline audit tools: call audit tools
google adk agent chain and app/app/agents/data_engineer_agent.py - Pipeline audit tools → Authorization: check access
google adk agent chain and app/app/tools/pipeline_tools.py - Pipeline audit tools → BigQuery analytics: query audit data
google adk agent chain and app/app/tools/pipeline_tools.py - Domain generators → Banking models: create records
python/generators/copy/banking_generator.py