How anandjha90/end-to-end-data-analytics-project_banking works
Project type and purpose
• This is an end-to-end data analytics pipeline for “Czechoslovakia Banking Financial Data Analysis.”
• It ingests raw banking CSVs, performs data cleaning and transformation via SQL scripts (including automated stored procedures), and delivers insights through a Power BI dashboard.
File structure
• Top-level directories:
• AUTOMATING_STORED_PROCEDURE_THROUGH_TASKS: SQL scripts and tests for scheduling stored-procedure execution
• Datasets_Raw / Datasets_Cleaned: raw CSV sources vs. cleaned CSV outputs
• SQL_SCRIPT: transformation scripts, ERD model, and Snowflake documentation
• README.md, Problem_statement.pdf, Data Manipulation docx, Dashboard_PBI.pdf
• Key configuration/build: the .mwb ERD file, Snowflake continuous loading docx, czec_bank_json_policies.txt (likely policy specs).
README for additional insights
• Eight source tables (Account, Card, Client, Disposition, District, Loan, Order, Transaction).
• Objectives: data cleaning, exploratory analysis, predictive modelling, answer business questions.
• Outputs: cleaned datasets, SQL-based transformation, Power BI dashboard.
a. Components
1. Data Sources
– Raw CSV files (Datasets_Raw/*)
2. Ingestion & Transformation Layer
– SQL_SCRIPT directory: ETL scripts, stored procedures
– AUTOMATING_STORED_PROCEDURE_THROUGH_TASKS: scheduling/automation engine
3. Data Storage
– Relational database (Snowflake or MySQL as per .mwb & docs)
– Cleaned CSVs as staging (Datasets_Cleaned/*)
4. Analytical Layer
– Ad-hoc SQL queries (automation_query.sql, TASK_CREATION_SCRIPT.sql)
5. Visualization
– Power BI dashboard (Dashboard_PBI.pdf)
6. Documentation & Metadata
– ER.pdf, Problem_statement.pdf, czec_bank_json_policies.txt
b. Relationships & data flows
• Raw CSVs → Ingestion scripts → Staging (Cleaned CSVs)
• Staging → Stored procedures/automated tasks → Database tables
• Database → Ad-hoc queries → BI semantic model
• BI model → Power BI dashboard
• ERD & metadata feed back into design and governance
c. Architectural patterns & design principles
– ETL pipeline pattern: Extract (raw CSV) → Transform (SQL scripts) → Load (database)
– Modular automation: separate directory for automating stored-procedure tasks
– Layered approach: source, staging, curated, presentation
– Schema-on-read vs. schema-on-write: raw CSVs are schemaless until ingested
d. Relevant technologies & frameworks
– SQL (MySQL Workbench .mwb ERD)
– Snowflake (continuous data loading docx)
– Power BI for dashboard
– Shell or task scheduler for automation scripts
Detailed capture
• Don’t over-generalize: include the “AUTOMATING_STORED_PROCEDURE_THROUGH_TASKS” component explicitly.
• Both raw CSV folders and cleaned CSV folders.
• Call out any external services or policies (e.g., JSON policy file for compliance).