How duyocd/data-warehouse-project works
Project Type and Purpose
• This is a data warehouse & analytics project demonstrating a modern Medallion Architecture (Bronze-Silver-Gold) built on SQL Server.
• It ingests raw CSV files from two source systems (ERP, CRM), applies successive ETL transformations, models data into a star schema, and produces business-ready analytics.
File Structure
• Top-level directories:
• datasets/ holds raw CSVs for ERP and CRM.
• scripts/ contains three subfolders: bronze/, silver/, gold/—each housing DDL and ETL stored procedures.
• tests/ holds SQL quality-check scripts for silver and gold layers.
• Supporting files: README.md, LICENSE, .gitignore, requirements.txt.
Extract Key Architectural Insights from the README
• The pipeline follows Bronze (raw ingestion) → Silver (cleansing & normalization) → Gold (star schema modeling) layers.
• Focuses on SQL Server as the execution engine, with CSV ingestion, transformation stored procedures, and analytics queries.
• Clear Labels: Name each component (e.g., “Bronze-DB”, “Silver-ETL”, “Customer_Dim”).
• Gears for ETL jobs
• Files for CSVs
• Shields for quality checks
• Charts for analytics