# rujab21/data-visualization-and-analysis architecture

> Architecture diagram of the GitHub repository rujab21/data-visualization-and-analysis, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rujab21/data-visualization-and-analysis
- Repository: https://github.com/rujab21/data-visualization-and-analysis

## Overview

##### Project type and purpose
- Type: A portfolio of data analytics & BI projects (no single monolith).
- Purpose: Demonstrate end-to-end data pipelines—from raw sources to insights—using SQL, Python (EDA), and Power BI dashboards.

##### File structure
- Top-level: Multiple project folders (e.g., “E-Commerce Performance Dashboard…,” “End to End Project Powerbi…,” “ExploratoryDataAnalysis(EDA),” etc.).
- Inside each folder:
– Data source files: .sql scripts, .xlsx, .csv
– Processing artifacts: Jupyter notebooks (.ipynb) for EDA
– Dashboard files: Power BI project files (.pbix)
– README.md with project context
- No backend code or real‐time services—this is a batch/desktop BI solution.

##### README
- Tools listed: SQL (MySQL, SQL Server), Power BI, Tableau, Excel, Python
- Shows each project reads from relational or flat‐file sources, transforms data, then builds dashboards.

a. Main components
1. Data Sources
- Relational DBs (MySQL, SQL Server)
- Flat files (Excel, CSV)
2. ETL / Processing Layer
- SQL scripts (.sql)
- Python notebooks (Jupyter) for cleansing & profiling
3. Data Staging / Workspace
- Local CSV/Excel or in-memory DataFrames
4. Visualization Layer
- Power BI Desktop reports (.pbix)
- (Optional) Tableau dashboards
5. Repository / Version Control
- Git repository holding code, scripts, PBIX files
b. Relationships & interactions
– ETL Layer outputs cleansed/staged datasets → Visualization Layer
– Visualization Layer produces dashboards consumed by end users
– All artifacts are committed to the Git repo
c. Architectural patterns & principles
– Batch ETL pipelines (not real-time)
– Directory‐per‐project enforces modularity
d. Key technologies
– SQL engines: MySQL & SQL Server
– Python & pandas in Jupyter notebooks
– Power BI Desktop (.pbix)

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Data Sources Layer
    subgraph "Data Sources" 
        direction TB
        MySQL["MySQL DB\n(Sales Data)"]:::db
        SQLServer["SQL Server DB\n(HR Data)"]:::db
        FlightXLS["flight_price.xlsx"]:::file
        WineRedCSV["winequality-red.csv"]:::file
        WineWhiteCSV["winequality-white.csv"]:::file
    end

    %% ETL / Processing Layer
    subgraph "ETL / Processing" 
        direction TB
        SQLScript1["mysql.sql"]:::script
        SQLScript2["SQLQuery4.sql"]:::script
        Notebook1["EDA Flight Price\n(dataset).ipynb"]:::notebook
        Notebook2["Redwine EDQ.ipynb"]:::notebook
        Notebook3["Whitewine.ipynb"]:::notebook
    end

    %% Data Staging / Workspace
    subgraph "Data Staging / Workspace"
        direction TB
        StagedXLS["output.xlsx"]:::file
    end

    %% Visualization Layer
    subgraph "Visualization"
        direction TB
        PBI_EComm["madhavdashboard.pbix"]:::dashboard
        PBI_End2End["pbi proj 42.pbix"]:::dashboard
        PBI_HR["HR Analytics Dashboard.pbix"]:::dashboard
        PBI_Insurance["project2.pbix"]:::dashboard
        PBI_Utility["Utility dashboard.pbix"]:::dashboard
    end

    %% Repository / Version Control
    GitRepo["Git Repository\n(README.md files)"]:::repo

    %% Connections
    MySQL -->|reads| SQLScript1
    MySQL -->|reads| SQLScript2
    SQLServer -->|reads| SQLScript2
    FlightXLS -->|inputs| Notebook1
    FlightXLS -->|inputs| Notebook1
    WineRedCSV -->|inputs| Notebook2
    WineWhiteCSV -->|inputs| Notebook3

    SQLScript1 -->|outputs| StagedXLS
    SQLScript2 -->|outputs| StagedXLS
    Notebook1 -->|outputs| StagedXLS
    Notebook2 -->|outputs| StagedXLS
    Notebook3 -->|outputs| StagedXLS

    StagedXLS -->|feeds| PBI_EComm
    StagedXLS -->|feeds| PBI_End2End
    StagedXLS -->|feeds| PBI_HR
    StagedXLS -->|feeds| PBI_Insurance
    StagedXLS -->|feeds| PBI_Utility

    %% Git connections
    MySQL -->|versioned in| GitRepo
    SQLServer -->|versioned in| GitRepo
    FlightXLS -->|versioned in| GitRepo
    WineRedCSV -->|versioned in| GitRepo
    WineWhiteCSV -->|versioned in| GitRepo
    SQLScript1 -->|versioned in| GitRepo
    SQLScript2 -->|versioned in| GitRepo
    Notebook1 -->|versioned in| GitRepo
    Notebook2 -->|versioned in| GitRepo
    Notebook3 -->|versioned in| GitRepo
    StagedXLS -->|versioned in| GitRepo
    PBI_EComm -->|versioned in| GitRepo
    PBI_End2End -->|versioned in| GitRepo
    PBI_HR -->|versioned in| GitRepo
    PBI_Insurance -->|versioned in| GitRepo
    PBI_Utility -->|versioned in| GitRepo

    %% Click Events
    click SQLScript1 "https://github.com/rujab21/data-visualization-and-analysis/blob/main/End to End Project Powerbi(Datasource : MYsql and SQL server)/mysql.sql"
    click SQLScript2 "https://github.com/rujab21/data-visualization-and-analysis/blob/main/End to End Project Powerbi(Datasource : MYsql and SQL server)/SQLQuery4.sql"
    click FlightXLS "https://github.com/rujab21/data-visualization-and-analysis/blob/main/ExploratoryDataAnalysis(EDA)/FlightPrice dataset EDA/flight_price.xlsx"
    click WineRedCSV "https://github.com/rujab21/data-visualization-and-analysis/blob/main/ExploratoryDataAnalysis(EDA)/RED/White-WineEDA/winequality-red.csv"
    click WineWhiteCSV "https://github.com/rujab21/data-visualization-and-analysis/blob/main/ExploratoryDataAnalysis(EDA)/RED/White-WineEDA/winequality-white.csv"
    click Notebook1 "https://github.com/rujab21/data-visualization-and-analysis/blob/main/ExploratoryDataAnalysis(EDA)/FlightPrice dataset EDA/EDA Flight Price dataset.ipynb"
    click Notebook2 "https://github.com/rujab21/data-visualization-and-analysis/blob/main/ExploratoryDataAnalysis(EDA)/RED/White-WineEDA/Redwine EDQ.ipynb"
    click Notebook3 "https://github.com/rujab21/data-visualization-and-analysis/blob/main/ExploratoryDataAnalysis(EDA)/RED/White-WineEDA/Whitewine.ipynb"
    click StagedXLS "https://github.com/rujab21/data-visualization-and-analysis/blob/main/ExploratoryDataAnalysis(EDA)/FlightPrice dataset EDA/output.xlsx"
    click PBI_EComm "https://github.com/rujab21/data-visualization-and-analysis/blob/main/E-Commerce Performance Dashboard – Sales, Profit & Customer Analytics/madhavdashboard.pbix"
    click PBI_End2End "https://github.com/rujab21/data-visualization-and-analysis/blob/main/End to End Project Powerbi(Datasource : MYsql and SQL server)/pbi proj 42.pbix"
    click PBI_HR "https://github.com/rujab21/data-visualization-and-analysis/blob/main/HR Analytics Dashboard – Employee Attrition & Workforce Insights/HR Analytics Dashboard – Employee Attrition & Workforce Insights.pbix"
    click PBI_Insurance "https://github.com/rujab21/data-visualization-and-analysis/blob/main/Insurance Data Analytics Dashboard – Power BI & SQL/project2.pbix"
    click PBI_Utility "https://github.com/rujab21/data-visualization-and-analysis/blob/main/Utility Consumption & Production Analysis – Power BI Project/Utility dashboard.pbix"
    click GitRepo "https://github.com/rujab21/data-visualization-and-analysis/blob/main/README.md"

    %% Styles
    classDef db fill:#FFF0B3,stroke:#F2C94C,stroke-width:2px;
    classDef file fill:#E8F0FE,stroke:#4285F4,stroke-width:2px;
    classDef script fill:#E6FFE6,stroke:#27AE60,stroke-width:2px;
    classDef notebook fill:#FFF1E6,stroke:#F2994A,stroke-width:2px;
    classDef dashboard fill:#F5E8FF,stroke:#9B51E0,stroke-width:2px;
    classDef repo fill:#ECECEC,stroke:#BDBDBD,stroke-width:2px;
```
