# magaramol/loan-risk-classification-and-insights-etl-and-eda-pipeline architecture

> Architecture diagram of the GitHub repository magaramol/loan-risk-classification-and-insights-etl-and-eda-pipeline, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/magaramol/loan-risk-classification-and-insights-etl-and-eda-pipeline
- Repository: https://github.com/magaramol/loan-risk-classification-and-insights-etl-and-eda-pipeline

## Overview

##### Project Type and Purpose
- This is an end-to-end ETL (Extract, Transform, Load) and Exploratory Data Analysis (EDA) pipeline built in a Jupyter Notebook.
- Its goal is to ingest raw loan data, apply cleaning and feature engineering, classify borrowers by risk and tenure, recommend communication channels, produce visualizations, and persist results into a MySQL database.

##### Key Architectural Components
a. Data Source
- CSV file (`dpd_zero.csv`) stored locally or in cloud storage
- Potential for extension: additional file formats (Parquet, JSON), API endpoints
b. ETL Engine
- Python modules/functions orchestrated in a Jupyter Notebook
- Subcomponents:
– Extraction: pandas read_csv
– Transformation: cleaning, risk/tenure classification, segmentation, feature engineering
– Loading: SQLAlchemy + mysql.connector writing into MySQL
c. Analytics & Visualization
- Seaborn and Matplotlib components producing PNGs in the IMG folder
- Exploratory dashboards or notebooks for inspecting distributions
d. Storage Layer
- MySQL database: schema creation, table `dpd`, using SQLAlchemy engine
- External dependency: MySQL server (hosted or managed service)
e. Execution Environment
- Jupyter Notebook as the orchestration layer
- Python runtime with dependencies: pandas, seaborn, matplotlib, mysql-connector-python, pymysql, SQLAlchemy

##### Relationships & Data Flow
1. User/Notebook Kernel triggers pipeline
2. Extraction reads CSV → DataFrame
3. Transformation applies cleaning functions and feature engineering → enriched DataFrame
4a. Load step persists DataFrame into MySQL (via connector + SQLAlchemy)
4b. Visualization step generates charts → saved in IMG/ folder
5. Consumer (data analyst, BI tool) queries MySQL or inspects saved charts

##### Architectural Patterns & Design Principles
- **Layered Architecture**: separate extraction, transformation, loading, visualization layers
- **Modular Functions**: classify_risk, classify_tenure, recommend_channel, feature engineering helpers
- **Single Source of Truth**: all cleaned and engineered data persisted in MySQL
- **Incremental Extension Points**: adding new features, data sources, or visualization modules

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Orchestration Layer
    subgraph "Orchestration Host"
        Orchestration["Jupyter Notebook\n(Loan_Risk_Classification_and_Insights_ETL_and_EDA_Pipeline.ipynb)"]:::orchestration
    end

    %% Data Source
    CSVSource["dpd_zero.csv"]:::storage

    %% ETL Stages
    Extraction["Extraction Module\n(pandas.read_csv)"]:::processing
    Transformation["Transformation Module\n(cleaning, risk & tenure classification,\nfeature engineering)"]:::processing
    Loading["Loading Module\n(SQLAlchemy + mysql.connector)"]:::processing
    Visualization["Visualization Module\n(seaborn, matplotlib)"]:::viz

    %% Storage and Outputs
    DB[(MySQL Database)]:::storage
    IMGOutputs["IMG/ (PNG files)"]:::viz

    %% Consumers and Documentation
    Consumer["Data Analyst / BI Tool"]:::external
    Docs["README.md"]:::external

    %% Data Flow
    CSVSource -->|"raw CSV"| Extraction
    Orchestration -->|"triggers pipeline"| Extraction
    Extraction -->|"pandas DataFrame"| Transformation
    Transformation -->|"enriched DataFrame"| Loading
    Transformation -->|"DataFrame for charts"| Visualization
    Loading -->|"SQL insert"| DB
    Visualization -->|"PNG files"| IMGOutputs
    DB -->|"query results"| Consumer
    IMGOutputs -->|"view charts"| Consumer
    Docs -->|"project instructions"| Orchestration

    %% Click Events
    click Orchestration "https://github.com/magaramol/loan-risk-classification-and-insights-etl-and-eda-pipeline/blob/main/Loan_Risk_Classification_and_Insights_ETL_and_EDA_Pipeline.ipynb"
    click IMGOutputs "https://github.com/magaramol/loan-risk-classification-and-insights-etl-and-eda-pipeline/tree/main/IMG/"
    click Docs "https://github.com/magaramol/loan-risk-classification-and-insights-etl-and-eda-pipeline/blob/main/README.md"

    %% Styles
    classDef orchestration fill:#e0f7fa,stroke:#006064,color:#004d40;
    classDef processing fill:#e8f5e9,stroke:#1b5e20,color:#2e7d32;
    classDef storage fill:#e3f2fd,stroke:#0d47a1,color:#1565c0;
    classDef viz fill:#fff3e0,stroke:#e65100,color:#bf360c;
    classDef external fill:#eceff1,stroke:#37474f,color:#263238;
```
