# kimseungwoo0407/lg_aimers_4 architecture

> Architecture diagram of the GitHub repository kimseungwoo0407/lg_aimers_4, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kimseungwoo0407/lg_aimers_4
- Repository: https://github.com/kimseungwoo0407/lg_aimers_4

## Overview

##### Project Type and Purpose
- This is a data-science/Machine-Learning pipeline notebook (LG_AIMERS.ipynb) that ingests customer data (train.csv), performs preprocessing and feature engineering, trains a predictive model (LightGBM, scikit-learn), and generates submission.csv.

##### File Structure
- Top-level:
- LG_AIMERS.ipynb – orchestrates the entire pipeline
- train.csv – labeled training data
- submission.csv – unlabeled test data for predictions

##### Main Components in the Architecture
a. Data Ingestion
- Components: File System (train.csv, submission.csv)
- Responsibility: Load raw CSVs via pandas
b. Data Preprocessing
- Subcomponents:
- Missing‐Value Handler (numeric median imputation, categorical mode imputation)
- Mapping Module (maps corporate code → country → continent)
- Responsibility: Clean and normalize raw data
c. Feature Engineering
- Subcomponents:
- Frequency Feature Generator (e.g., customer_idx_freq)
- Ratio Feature Generator (e.g., historical_conversion_rate_per_customer)
- Derived Columns (e.g., response_corporate_1)
- Responsibility: Enrich dataset with predictive signals
d. Modeling
- Subcomponents:
- Model Trainer (LightGBM, scikit-learn pipelines)
- Hyperparameter Tuning Stub (future GridSearch/BayesOpt)
- Responsibility: Fit predictive model, evaluate performance
e. Output & Submission
- Components:
- Prediction Engine (applies model to submission.csv)
- Submission Writer (exports submission.csv)

##### Relationships & Data Flows
train.csv → Data Preprocessing → Feature Engineering → Model Trainer → Model Artifact
submission.csv → Data Preprocessing → Feature Engineering → Prediction Engine → submission.csv
- That preprocessing and feature engineering modules are shared by both train and test paths.

##### Architectural Patterns & Principles
- Pipeline/ETL pattern: sequential tasks on data
- Modularity: separate preprocessing, feature engineering, modeling
- Reusability: preprocess and feature steps applied to both train and test

##### Technologies & Frameworks
- pandas, numpy (data manipulation)
- scikit-learn (model pipelines, metrics)
- LightGBM (GBDT learner)
- gensim (if any NLP features in future extensions)
- Jupyter Notebook (orchestration environment)

1. Sketch the four high‐level lanes (Data Ingestion, Preprocessing, Feature Engineering, Modeling & Output).
4. Next, the Feature Engineering block, subdivided into Frequency Features and Ratio Features.
5. Then, the Modeling block, showing LightGBM and scikit-learn pipelines.
6. Finally, output to submission.csv.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Orchestration
    Notebook["LG_AIMERS.ipynb"]:::external
    click Notebook "https://github.com/kimseungwoo0407/lg_aimers_4/blob/main/LG_AIMERS.ipynb"

    %% Data Ingestion Layer
    subgraph "Data Ingestion" 
        direction TB
        TrainCSV["train.csv"]:::data
        SubmissionCSV_Input["submission.csv"]:::data
    end

    %% Preprocessing Layer
    subgraph "Data Preprocessing"
        direction TB
        MVH["Missing-Value Handler"]:::process
        MM["Mapping Module"]:::process
    end

    %% Feature Engineering Layer
    subgraph "Feature Engineering"
        direction TB
        FFG["Frequency Feature Generator"]:::process
        RFG["Ratio Feature Generator"]:::process
        DC["Derived Columns"]:::process
    end

    %% Modeling Layer
    subgraph "Modeling"
        direction TB
        MT["Model Trainer (LightGBM & scikit-learn)"]:::model
        HT["Hyperparameter Tuning Stub"]:::model
    end

    %% Output & Submission Layer
    subgraph "Output & Submission"
        direction TB
        PE["Prediction Engine"]:::process
        SW["Submission Writer"]:::process
        SubmissionCSV_Output["submission.csv"]:::data
    end

    %% Documentation
    Docs["README.md"]:::external
    click Docs "https://github.com/kimseungwoo0407/lg_aimers_4/blob/main/README.md"

    %% Data Flow Connections
    Notebook -->|orchestrates| TrainCSV
    Notebook -->|orchestrates| SubmissionCSV_Input

    TrainCSV -->|raw data| MVH
    SubmissionCSV_Input -->|raw data| MVH

    MVH -->|cleaned data| MM
    MM -->|mapped data| FFG

    FFG -->|features| RFG
    RFG -->|features| DC

    DC -->|"train features"| MT
    MT -->|"model artifact"| HT

    SubmissionCSV_Input -->|raw data| MVH
    DC -->|"test features"| PE
    PE -->|"predictions"| SW
    SW --> SubmissionCSV_Output

    %% Legend
    subgraph "Legend"
        direction TB
        A["Data Layers"]:::data
        B["Processing Modules"]:::process
        C["Modeling Components"]:::model
        D["External/Orchestration"]:::external
    end

    classDef data fill:#cfe2f3,stroke:#0969da,color:#0969da;
    classDef process fill:#d1e7dd,stroke:#0f5132,color:#0f5132;
    classDef model fill:#fde2c5,stroke:#d97706,color:#9a3412;
    classDef external fill:#e2e3e5,stroke:#6c757d,color:#495057;
```
