How databeli/databricks_course works
This repository is a collection of Databricks notebook examples, not one integrated application. Its principal recurring workflow is data ingestion, transformation across lakehouse layers, and analytics; separate examples demonstrate notebook use, jobs, streaming, AI, privacy, and data modeling. Source excerpts confirm selected implementations, while unsampled projects are represented only at the directory level; the empty README provides no additional architecture evidence.
Main entry points: 02_Notebooks Agentic AI notebooks · 01_Fetch_List_of_Tables.py Table listing · my_transformation.py Declarative pipeline · 01_Notebook_01_Main.py Notebook features
Components 24
Notebook workflows
- Notebook features
01_Notebook_Features_Overview/01_Notebook_01_Main.py - Notebook helpers
01_Notebook_Features_Overview/01_Notebook_02_Helper.py - Notebook execution
02_Notebook_Run_from_Notebook/02_Notebook_01_Main.py - Job setup
05_Jobs/01_Set_up_Notebooks/01_Setup_List_of_Tables.py - Table listing
05_Jobs/02_Job_Notebooks/01_Fetch_List_of_Tables.py - Table ingestion
05_Jobs/02_Job_Notebooks/02_Ingest_Data.py - Job transformation
05_Jobs/02_Job_Notebooks/03_Transform_Data.py
Data pipelines
- Auto Loader
06_Auto_Loader/01_Auto_Loader.py - Bank source to bronze
13_Banking_Project/02_Source_to_Silver/03_Source_to_Bronze.py - Bank bronze to silver
13_Banking_Project/02_Source_to_Silver/04_Bronze_to_Silver.py - Bank gold driver
13_Banking_Project/03_Silver_to_Gold/01_Silver_to_Gold_Driver.py - Declarative pipeline
11_Lakeflow_Declarative_Pipeline/my_transformation.py
Analytics projects
- Retail ingestion
17_RetailQ_Project/02_Notebooks/01_blob_to_bronze.py - Retail silver transforms
17_RetailQ_Project/03_ETL_Pipeline/bronze_to_silver - Retail sales facts
17_RetailQ_Project/03_ETL_Pipeline/silver_to_gold/fact_sales.py - Retail metrics
17_RetailQ_Project/02_Notebooks/04_Metric View.py
Specialized examples
- PII handling
14_PII_Data_Handling - SCD data models
19_Data_Modelling/02_Code_Notebooks - Agentic AI notebooks
18_Agentic_AI/02_Notebooks
Other components
- Notebook user
- Catalog tables
- File inputs
- Vector search index
- Databricks AI service
Connections 24
- Notebook user → Notebook features: uses
- Notebook features → Notebook helpers: runs via
- Notebook user → Notebook execution: runs
- Notebook execution → Table listing: invokes
- Job setup → Table listing: prepares
- Table listing → Catalog tables: reads
- Table ingestion → Catalog tables: reads and writes
- Job transformation → Catalog tables: transforms
- File inputs → Auto Loader: streams files
- Auto Loader → Catalog tables: writes Delta
- Bank source to bronze → Catalog tables: reads and writes
- Bank bronze to silver → Catalog tables: reads and writes
- Bank gold driver → Catalog tables: produces aggregates
- Retail ingestion → Catalog tables: writes bronze
- Retail silver transforms → Catalog tables: reads bronze; writes silver
- Retail sales facts → Catalog tables: reads silver; writes gold
- Retail metrics → Catalog tables: defines metrics on
- Declarative pipeline → File inputs: reads streaming data
- Declarative pipeline → Catalog tables: defines tables and views
- PII handling → Catalog tables: transforms PII
- SCD data models → Catalog tables: models table history
- Notebook user → Agentic AI notebooks: queries documents
- Agentic AI notebooks → Vector search index: searches
- Agentic AI notebooks → Databricks AI service: requests completion