# luu-quang/data_mining_and_machine_learning_usth architecture

> Architecture diagram of the GitHub repository luu-quang/data_mining_and_machine_learning_usth, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/luu-quang/data_mining_and_machine_learning_usth
- Repository: https://github.com/luu-quang/data_mining_and_machine_learning_usth

## Overview

This is a batch banking-product recommendation workflow: customer CSV data is converted, cleaned, transformed into temporal product-addition targets, and used to rank new-product recommendations with out-of-time MAP@5 evaluation. The README also describes EDA, feature engineering, optional modeling approaches, and optimization; the available scripts concretely implement conversion, cleaning, PCA exploration, and a random-forest pipeline. Edges reflect only script calls and documented workflow order; the scripts are standalone and use hard-coded file paths, so links between stages are documented rather than direct module calls.

Main entry points: [`01_eda_and_cleaning.py`](https://github.com/luu-quang/data_mining_and_machine_learning_usth/blob/main/01_eda_and_cleaning.py) (Cleaning and imputation), [`00_convert_format.py`](https://github.com/luu-quang/data_mining_and_machine_learning_usth/blob/main/00_convert_format.py) (Format conversion), [`03_finalize_model.py`](https://github.com/luu-quang/data_mining_and_machine_learning_usth/blob/main/03_finalize_model.py) (Temporal features and targets), [`02_feature_extraction_pca.py`](https://github.com/luu-quang/data_mining_and_machine_learning_usth/blob/main/02_feature_extraction_pca.py) (PCA exploration)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ingest["Data preparation"]
  node_format_conversion["Format conversion"]
  node_cleaning["Cleaning and imputation"]
  node_optimized_data[("Optimized Parquet data")]
  node_cleaned_data[("Cleaned Parquet data")]
end

subgraph group_features["Feature engineering"]
  node_eda["Behavior exploration"]
  node_pca_visualization["PCA exploration"]
  node_temporal_features["Temporal features and targets"]
  node_pca_pipeline["Scaled PCA transform"]
end

subgraph group_model["Recommendation modeling"]
  node_target_models["Product-specific forests"]
  node_recommendations["Ranked recommendations"]
end

subgraph group_evaluation["Evaluation"]
  node_map_evaluation["MAP@5 evaluation"]
end

node_customer_data(("Bank customer data"))

node_customer_data -->|"loads CSV"| node_format_conversion
node_format_conversion -->|"writes Parquet"| node_optimized_data
node_optimized_data -.->|"reads Parquet"| node_cleaning
node_cleaning -->|"writes Parquet"| node_cleaned_data
node_cleaned_data -->|"loads datasets"| node_temporal_features
node_cleaning -.->|"supports exploration"| node_eda
node_cleaned_data -->|"loads training data"| node_pca_visualization
node_temporal_features -->|"prepares split features"| node_pca_pipeline
node_pca_pipeline -->|"provides components"| node_target_models
node_target_models -->|"scores and ranks"| node_recommendations
node_recommendations -->|"compared with purchases"| node_map_evaluation

click node_format_conversion "https://github.com/luu-quang/data_mining_and_machine_learning_usth/blob/main/00_convert_format.py"
click node_cleaning "https://github.com/luu-quang/data_mining_and_machine_learning_usth/blob/main/01_eda_and_cleaning.py"
click node_eda "https://github.com/luu-quang/data_mining_and_machine_learning_usth/blob/main/01_eda_and_cleaning.py"
click node_pca_visualization "https://github.com/luu-quang/data_mining_and_machine_learning_usth/blob/main/02_feature_extraction_pca.py"
click node_temporal_features "https://github.com/luu-quang/data_mining_and_machine_learning_usth/blob/main/03_finalize_model.py"
click node_pca_pipeline "https://github.com/luu-quang/data_mining_and_machine_learning_usth/blob/main/03_finalize_model.py"
click node_target_models "https://github.com/luu-quang/data_mining_and_machine_learning_usth/blob/main/03_finalize_model.py"
click node_recommendations "https://github.com/luu-quang/data_mining_and_machine_learning_usth/blob/main/03_finalize_model.py"
click node_map_evaluation "https://github.com/luu-quang/data_mining_and_machine_learning_usth/blob/main/03_finalize_model.py"
click node_optimized_data "https://github.com/luu-quang/data_mining_and_machine_learning_usth/blob/main/00_convert_format.py"
click node_cleaned_data "https://github.com/luu-quang/data_mining_and_machine_learning_usth/blob/main/01_eda_and_cleaning.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_format_conversion,node_cleaning,node_optimized_data,node_cleaned_data toneBlue
class node_eda,node_pca_visualization,node_temporal_features,node_pca_pipeline toneAmber
class node_target_models,node_recommendations toneMint
class node_map_evaluation toneRose
class node_customer_data toneIndigo
```

## Components

### Data preparation

- [Format conversion](https://github.com/luu-quang/data_mining_and_machine_learning_usth/blob/main/00_convert_format.py): `00_convert_format.py`
- [Cleaning and imputation](https://github.com/luu-quang/data_mining_and_machine_learning_usth/blob/main/01_eda_and_cleaning.py): `01_eda_and_cleaning.py`
- [Optimized Parquet data](https://github.com/luu-quang/data_mining_and_machine_learning_usth/blob/main/00_convert_format.py): `00_convert_format.py`
- [Cleaned Parquet data](https://github.com/luu-quang/data_mining_and_machine_learning_usth/blob/main/01_eda_and_cleaning.py): `01_eda_and_cleaning.py`

### Feature engineering

- [Behavior exploration](https://github.com/luu-quang/data_mining_and_machine_learning_usth/blob/main/01_eda_and_cleaning.py): `01_eda_and_cleaning.py`
- [PCA exploration](https://github.com/luu-quang/data_mining_and_machine_learning_usth/blob/main/02_feature_extraction_pca.py): `02_feature_extraction_pca.py`
- [Temporal features and targets](https://github.com/luu-quang/data_mining_and_machine_learning_usth/blob/main/03_finalize_model.py): `03_finalize_model.py`
- [Scaled PCA transform](https://github.com/luu-quang/data_mining_and_machine_learning_usth/blob/main/03_finalize_model.py): `03_finalize_model.py`

### Recommendation modeling

- [Product-specific forests](https://github.com/luu-quang/data_mining_and_machine_learning_usth/blob/main/03_finalize_model.py): `03_finalize_model.py`
- [Ranked recommendations](https://github.com/luu-quang/data_mining_and_machine_learning_usth/blob/main/03_finalize_model.py): `03_finalize_model.py`

### Evaluation

- [MAP@5 evaluation](https://github.com/luu-quang/data_mining_and_machine_learning_usth/blob/main/03_finalize_model.py): `03_finalize_model.py`

### Other components

- **Bank customer data**

## Connections

- Bank customer data → Format conversion: loads CSV (evidence: `00_convert_format.py`)
- Format conversion → Optimized Parquet data: writes Parquet (evidence: `00_convert_format.py`)
- Optimized Parquet data → Cleaning and imputation: reads Parquet (evidence: `README.md`)
- Cleaning and imputation → Cleaned Parquet data: writes Parquet (evidence: `01_eda_and_cleaning.py`)
- Cleaned Parquet data → Temporal features and targets: loads datasets (evidence: `03_finalize_model.py`)
- Cleaning and imputation → Behavior exploration: supports exploration (evidence: `README.md`)
- Cleaned Parquet data → PCA exploration: loads training data (evidence: `02_feature_extraction_pca.py`)
- Temporal features and targets → Scaled PCA transform: prepares split features (evidence: `03_finalize_model.py`)
- Scaled PCA transform → Product-specific forests: provides components (evidence: `03_finalize_model.py`)
- Product-specific forests → Ranked recommendations: scores and ranks (evidence: `03_finalize_model.py`)
- Ranked recommendations → MAP@5 evaluation: compared with purchases (evidence: `03_finalize_model.py`)
