How rujab21/data-visualization-and-analysis works
Project type and purpose
• Type: A portfolio of data analytics & BI projects (no single monolith).
• Purpose: Demonstrate end-to-end data pipelines—from raw sources to insights—using SQL, Python (EDA), and Power BI dashboards.
File structure
• Top-level: Multiple project folders (e.g., “E-Commerce Performance Dashboard…,” “End to End Project Powerbi…,” “ExploratoryDataAnalysis(EDA),” etc.).
• Inside each folder:
– Data source files: .sql scripts, .xlsx, .csv
– Processing artifacts: Jupyter notebooks (.ipynb) for EDA
– Dashboard files: Power BI project files (.pbix)
– README.md with project context
• No backend code or real‐time services—this is a batch/desktop BI solution.
README
• Tools listed: SQL (MySQL, SQL Server), Power BI, Tableau, Excel, Python
• Shows each project reads from relational or flat‐file sources, transforms data, then builds dashboards.
a. Main components
1. Data Sources
• Relational DBs (MySQL, SQL Server)
• Flat files (Excel, CSV)
2. ETL / Processing Layer
• SQL scripts (.sql)
• Python notebooks (Jupyter) for cleansing & profiling
3. Data Staging / Workspace
• Local CSV/Excel or in-memory DataFrames
4. Visualization Layer
• Power BI Desktop reports (.pbix)
• (Optional) Tableau dashboards
5. Repository / Version Control
• Git repository holding code, scripts, PBIX files
b. Relationships & interactions
– ETL Layer outputs cleansed/staged datasets → Visualization Layer
– Visualization Layer produces dashboards consumed by end users
– All artifacts are committed to the Git repo
c. Architectural patterns & principles
– Batch ETL pipelines (not real-time)
– Directory‐per‐project enforces modularity
d. Key technologies
– SQL engines: MySQL & SQL Server
– Python & pandas in Jupyter notebooks
– Power BI Desktop (.pbix)