How databricks-solutions/data-intelligence-for-smarter-community works
This repository is a Databricks hands-on accelerator for turning synthetic telco/IoT telemetry and UBC geodata into smart-community analytics and a campus-map app. The documented workflow moves through Bronze generation, Silver cleanup, Gold serving tables, AI/BI dashboards, Genie Q&A, and app deployment. The reference app serves map and telemetry data and can ask Genie questions. Notebook-to-notebook links follow the README’s stated lab order; app internals are shown only where excerpts or the source index support them.
Main entry points: app.py FastAPI app · 02 - Hands On - Clean & Transform.py Clean to Silver · sql.py SQL execution · 00 - Course Overview.py Course overview
Components 24
Workshop workflow
- Course overview
workshop/00 - Course Overview.py - Workspace setup
workshop/Includes/Classroom-Setup-Common.py - Profile data
workshop/01 - Hands On - Explore & Profile the Data.py - Clean to Silver
workshop/02 - Hands On - Clean & Transform.py - Deploy app
workshop/06 - Hands On - Deploy the App.py
Data preparation
- Generate Bronze
workshop/Includes/smart_campus_generators.py - Bronze tables
- UBC geodata
workshop/Includes/geojson - Silver tables
- Gold serving tables
workshop/04 - Hands On - Build Serving Tables.py
Analytics and Q&A
- AI/BI dashboard
workshop/03 - Hands On - Visualize with AI-BI.py - Genie space setup
workshop/05 - Hands On - Create a Genie Space.py
Reference app
- FastAPI app
workshop/Includes/reference_code/app/app.py - Dataset queries
workshop/Includes/reference_code/app/server/datasets.py - Map geometry
workshop/Includes/reference_code/app/server/geometry.py - Genie client
workshop/Includes/reference_code/app/server/genie.py - SQL execution
workshop/Includes/reference_code/app/server/sql.py - App configuration
workshop/Includes/reference_code/app/server/config.py - Campus map app
Databricks services
- Unity Catalog
- SQL warehouse
- Genie service
Other components
- Workshop user
- Workspace user
Connections 29
- Workshop user → Course overview: starts course
README.md - Course overview → Workspace setup: provisions workspace
README.md - Workspace setup → Generate Bronze: invokes generation
workshop/Includes/Classroom-Setup-Common.py - Generate Bronze → Bronze tables: writes tables
workshop/Includes/smart_campus_generators.py - Course overview → Profile data: begins labs
README.md - Profile data → Bronze tables: profiles data
README.md - Profile data → Clean to Silver: continues to
README.md - Clean to Silver → UBC geodata: loads geodata
workshop/02 - Hands On - Clean & Transform.py - Clean to Silver → Bronze tables: reads raw tables
workshop/02 - Hands On - Clean & Transform.py - Clean to Silver → Silver tables: writes cleaned tables
workshop/02 - Hands On - Clean & Transform.py - Clean to Silver → AI/BI dashboard: continues to
README.md - AI/BI dashboard → Gold serving tables: continues to
README.md - Gold serving tables → Genie space setup: continues to
README.md - Genie space setup → Deploy app: provides Genie ID
README.md - Deploy app → FastAPI app: deploys reference app
workshop/06 - Hands On - Deploy the App.py - Deploy app → Campus map app: publishes app
README.md - Campus map app → Workspace user: serves map and Q&A
README.md - FastAPI app → Dataset queries: routes data requests
workshop/Includes/reference_code/app/app.py - FastAPI app → Map geometry: gets map polygons
workshop/Includes/reference_code/app/app.py - FastAPI app → Genie client: forwards questions
workshop/Includes/reference_code/app/app.py - Dataset queries → SQL execution: runs queries
workshop/Includes/reference_code/app/server/datasets.py - Map geometry → SQL execution: reads table properties
workshop/Includes/reference_code/app/server/geometry.py - SQL execution → App configuration: gets connection settings
workshop/Includes/reference_code/app/server/sql.py - Genie client → App configuration: gets workspace settings
workshop/Includes/reference_code/app/server/genie.py - SQL execution → SQL warehouse: executes SQL
workshop/Includes/reference_code/app/server/sql.py - SQL warehouse → Unity Catalog: queries tables
- Genie client → Genie service: calls conversations API
workshop/Includes/reference_code/app/server/genie.py - FastAPI app → Silver tables: uses app data
README.md - FastAPI app → Genie service: asks Genie