# thalesh7991/insiders-customers architecture

> Architecture diagram of the GitHub repository thalesh7991/insiders-customers, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/thalesh7991/insiders-customers
- Repository: https://github.com/thalesh7991/insiders-customers

## Overview

##### Project type and purpose
- This is a data-science/ML pipeline focused on customer clustering (“Insiders Customers”).
- Primary outputs: cluster models (KMeans, GMM, HC), EDA reports, and a deployed model in AWS.

##### File structure
- Top‐level:
- `.ipynb_checkpoints` – auto‐save of notebooks
- `notebooks/` – sequential Jupyter notebooks for each step (planning → filtering → feature engineering → modeling → deployment)
- `reports/` – exported snapshots of key notebooks for stakeholders
- `references/` – reference HTML outputs
- `README.md` – description of business problem, metrics, and deployment target
- No distinct “src” or “backend/frontend” folders: the entire logic is embedded in the notebooks, so the pipeline itself is the central architectural asset.

##### Extract architecture insights from the README
- Ten step solution strategy: data description → filtering → feature engineering → EDA → feature selection → data prep → modeling → cluster analysis → cluster EDA → AWS deploy
- Deployment on AWS (implies model artifact storage, hosting, and possibly an endpoint or batch scoring)

##### Main components & interactions (to represent
A. Data Sources
- Raw transaction data (CSV or database)
- (Optional) Metadata for filtering (stock codes list)
B. Data Processing Environment
- Jupyter Notebook Kernel (compute container or local)
- Sequential pipeline stages:
1. Data Description & Cleaning
2. Data Filtering
3. Feature Engineering & Selection
4. Exploratory Data Analysis
C. Model Training
- Algorithms: KMeans, GMM, Hierarchical Clustering
- Compute environment (same notebook kernel or separate training instance)
- Output: trained model artifacts
D. Model Evaluation
- Silhouette Score analysis in notebooks
- Selection of best clustering configuration
E. Model Registry & Storage
- S3 bucket or AWS Model Registry for artifacts
F. Deployment Environment
- AWS service (e.g., SageMaker Endpoint, Lambda + API Gateway, or batch job)
- Endpoint for scoring new customer data
G. Reporting/Visualization
- Exported HTML notebooks in `reports/` and `references/`

##### Architectural patterns & design principles
- Linear pipeline (“staircase”) pattern: each notebook is a discrete stage
- Separation of concerns:
- Data ingestion/cleaning
- Feature engineering
- Modeling/evaluation
- Deployment
- Reproducibility: all steps captured in notebooks that can be rerun end-to-end
- Modular deployment: model artifacts decoupled from code environment, deployed on managed AWS service

##### Final checklist
- All components have clear, unambiguous labels.
- Call out critical design choices (e.g., stateless notebooks vs. persistent model registry).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Sources
    DataSources["Raw Data (CSV/DB)"]:::storage
    MetaData["Metadata (Stock Codes)"]:::storage

    %% Jupyter Notebook Pipeline
    subgraph "Jupyter Notebook Pipeline"
        DescClean["Data Description & Cleaning<br/>(Pandas)"]:::processing
        Filter["Data Filtering<br/>(Pandas)"]:::processing
        FeatEng["Feature Engineering & Selection<br/>(Pandas, Scikit-Learn)"]:::processing
        EDA["Exploratory Data Analysis<br/>(Matplotlib, Seaborn)"]:::processing
        ModelTrain["Model Training<br/>(KMeans, GMM, HC)"]:::processing
        EvalPerf["Model Evaluation<br/>(Silhouette Score)"]:::processing
        EvalResult["Model Result Selection"]:::processing
        Deploy1["Deployment Prep<br/>(Serialization)"]:::processing
        Deploy2["AWS Integration"]:::processing
    end

    %% Storage & Reproducibility
    Checkpoints[".ipynb_checkpoints<br/>(Reproducibility)"]:::storage
    Reports["Exported Reports"]:::storage
    References["Reference HTML Outputs"]:::storage

    %% AWS Environment
    subgraph "AWS Cloud"
        S3["Model Registry & Storage (S3)"]:::storage
        SageMaker["SageMaker Endpoint"]:::aws
        Lambda["Lambda + API Gateway"]:::aws
    end

    %% Consumers
    Dashboard["Marketing Dashboard<br/>/ Loyalty App"]:::external

    %% Data Flow
    DataSources --> DescClean
    MetaData --> DescClean
    DescClean --> Filter
    Filter --> FeatEng
    FeatEng --> EDA
    EDA --> Reports
    EDA --> References
    EDA --> ModelTrain
    ModelTrain --> EvalPerf
    EvalPerf --> ModelTrain
    EvalPerf --> EvalResult
    EvalResult --> Deploy1
    Deploy1 --> Deploy2
    Deploy2 --> S3
    S3 --> SageMaker
    S3 --> Lambda
    SageMaker --> Dashboard
    Lambda --> Dashboard

    %% Click Events
    click DescClean "https://github.com/thalesh7991/insiders-customers/blob/main/notebooks/c0.0-mdfl-planning.ipynb"
    click FeatEng "https://github.com/thalesh7991/insiders-customers/blob/main/notebooks/c0.4-mdfl-feature-engineering-II.ipynb"
    click ModelTrain "https://github.com/thalesh7991/insiders-customers/blob/main/notebooks/c0.5-mdfl-clustering-modeling.ipynb"
    click EvalPerf "https://github.com/thalesh7991/insiders-customers/blob/main/notebooks/c0.6-mdfl-model-performance.ipynb"
    click EvalResult "https://github.com/thalesh7991/insiders-customers/blob/main/notebooks/c0.7-mdfl-model-result.ipynb"
    click EDA "https://github.com/thalesh7991/insiders-customers/blob/main/notebooks/c0.8-mdfl-eda.ipynb"
    click Deploy1 "https://github.com/thalesh7991/insiders-customers/blob/main/notebooks/c0.9-mdfl-deploy.ipynb"
    click Deploy2 "https://github.com/thalesh7991/insiders-customers/blob/main/notebooks/c10-mdfl-deploy.ipynb"
    click Reports "https://github.com/thalesh7991/insiders-customers/tree/main/reports/"
    click References "https://github.com/thalesh7991/insiders-customers/tree/main/references/"
    click Checkpoints "https://github.com/thalesh7991/insiders-customers/tree/main/.ipynb_checkpoints/"
    click DataSources "https://github.com/thalesh7991/insiders-customers/blob/main/README.md"

    %% Styles
    classDef processing fill:#D0E8FF,stroke:#0366d6,color:#0366d6
    classDef storage fill:#C8E6C9,stroke:#2e7d32,color:#2e7d32
    classDef aws fill:#FFE0B2,stroke:#ef6c00,color:#ef6c00
    classDef external fill:#f0f0f0,stroke:#999999,color:#555555,font-style:italic
```
