# tergelitu/ml-models-for-credit-scoring-benchmark architecture

> Architecture diagram of the GitHub repository tergelitu/ml-models-for-credit-scoring-benchmark, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tergelitu/ml-models-for-credit-scoring-benchmark
- Repository: https://github.com/tergelitu/ml-models-for-credit-scoring-benchmark

## Overview

##### Project Type & Purpose
- This is an ML benchmarking pipeline (not a multi-service web app). It’s packaged as a Jupyter notebook (`result.ipynb`) that orchestrates the end-to-end credit-scoring workflow: data ingestion → preprocessing → feature engineering → model training & tuning → evaluation & visualization.

##### Key Files & Directories
- result.ipynb: orchestration of all steps
- requirements.txt: Python libraries (pandas, scikit-learn, XGBoost, LightGBM, CatBoost, matplotlib, seaborn, etc.)
- data/ (mentioned in README): raw CSVs from Kaggle
- outcome/: generated plots, flowcharts, ROC curves, 3D visualizations

##### Main Components
a. Data Ingestion
- External service: Kaggle dataset (downloaded to local `data/` folder)
- Component: Data Loader (pandas)
b. Data Preprocessing & Cleaning
- Null handling, outlier removal, encoding
- Tools: pandas, NumPy
c. Feature Engineering
- Aggregation of auxiliary tables, custom ratios, time-based features
- Output: single merged DataFrame keyed by SK_ID_CURR
d. Sampling & Feature Selection
- Stratified sampling
- Feature importance ranking (RF, XGBoost, LGBM, CatBoost, GB)
- Correlation filtering, leakage detection
e. Model Training & Evaluation
- Parallel training of multiple models
- ROC AUC computation & comparison
- Visualization: ROC curves
f. Hyperparameter Tuning
- GridSearchCV (scikit-learn) or built-in tuning of LightGBM/CatBoost/GB
g. Score Binning & 3D Intersection
- K-means binning into deciles
- 3D plot of bin intersections across three model probabilities
h. Acceptance Region Analysis
- Contiguous region with bad-rate threshold
- 3D cutoff area visualization
i. Outputs
- Plots and flowcharts saved in `outcome/`

##### Relationships & Data Flow
- Data flows linearly from ingestion → preprocessing → feature engineering → sampling → training → evaluation → visualization → storage.
- Hyperparameter tuning loops back into the training component.
- Visual outputs feed into the outcome folder.

##### Architectural Patterns & Principles
- Pipeline pattern: each stage transforms the dataset, outputs feed into next stage.
- Modular stages (each can be extracted into functions/modules if codebase grows).
- Single orchestrator (notebook) – consider separating into scripts or a DAG for production.

##### Technologies & Frameworks
- Python ecosystem: pandas, NumPy for ETL; scikit-learn for modeling & grid search; XGBoost, LightGBM, CatBoost for specialized boosting; matplotlib/Seaborn for plots; Jupyter for orchestration.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Storage and Documentation
    RawData["Raw Data\n(data/*.csv)"]:::storage
    Dependencies["Dependencies\n(requirements.txt)"]:::storage
    click Dependencies "https://github.com/tergelitu/ml-models-for-credit-scoring-benchmark/blob/main/requirements.txt"
    Docs["Documentation\n(README.md)"]:::storage
    click Docs "https://github.com/tergelitu/ml-models-for-credit-scoring-benchmark/blob/main/README.md"
    Outcome["Outcome Storage\n(outcome/)"]:::storage
    click Outcome "https://github.com/tergelitu/ml-models-for-credit-scoring-benchmark/tree/main/outcome/"
    
    %% Orchestrator
    Notebook["Orchestrator\n(result.ipynb)"]:::dataStage
    click Notebook "https://github.com/tergelitu/ml-models-for-credit-scoring-benchmark/blob/main/result.ipynb"
    
    %% Pipeline Stages
    DataIngest["Data Ingestion\n(pandas)"]:::dataStage
    Preprocess["Preprocessing & Cleaning\n(pandas, NumPy)"]:::dataStage
    FeatureEng["Feature Engineering\n(pandas)"]:::dataStage
    Sampling["Sampling & Feature Selection\n(stratified, RF, XGB, LGBM)"]:::dataStage
    Training["Model Training\n(XGBoost, LGBM, CatBoost)"]:::modelStage
    Tuning["Hyperparameter Tuning\n(GridSearchCV, built-in)"]:::modelStage
    Evaluation["Evaluation & Comparison\n(ROC AUC, ensemble)"]:::modelStage
    Binning["Score Binning & 3D Plot\n(K-means)"]:::vizStage
    Acceptance["Acceptance Region Analysis\n(3D cutoff)"]:::vizStage

    %% Orchestration Flow
    Dependencies --> Notebook --> DataIngest
    RawData --> DataIngest --> Preprocess --> FeatureEng --> Sampling --> Training
    Training --> Tuning --> Training
    Training --> Evaluation --> Binning --> Acceptance --> Outcome

    %% Classes
    classDef dataStage fill:#ADD8E6,stroke:#000;
    classDef modelStage fill:#FFA500,stroke:#000;
    classDef vizStage fill:#90EE90,stroke:#000;
    classDef storage fill:#D3D3D3,stroke:#000;
```
