How thalesh7991/insiders-customers works
Project type and purpose
• This is a data-science/ML pipeline focused on customer clustering (“Insiders Customers”).
• Primary outputs: cluster models (KMeans, GMM, HC), EDA reports, and a deployed model in AWS.
File structure
• Top‐level:
• .ipynb_checkpoints – auto‐save of notebooks
• notebooks/ – sequential Jupyter notebooks for each step (planning → filtering → feature engineering → modeling → deployment)
• reports/ – exported snapshots of key notebooks for stakeholders
• references/ – reference HTML outputs
• README.md – description of business problem, metrics, and deployment target
• No distinct “src” or “backend/frontend” folders: the entire logic is embedded in the notebooks, so the pipeline itself is the central architectural asset.
Extract architecture insights from the README
• Ten step solution strategy: data description → filtering → feature engineering → EDA → feature selection → data prep → modeling → cluster analysis → cluster EDA → AWS deploy
• Deployment on AWS (implies model artifact storage, hosting, and possibly an endpoint or batch scoring)
Main components & interactions (to represent
A. Data Sources
• Raw transaction data (CSV or database)
• (Optional) Metadata for filtering (stock codes list)
B. Data Processing Environment
• Jupyter Notebook Kernel (compute container or local)
• Sequential pipeline stages:
1. Data Description & Cleaning
2. Data Filtering
3. Feature Engineering & Selection
4. Exploratory Data Analysis
C. Model Training
• Algorithms: KMeans, GMM, Hierarchical Clustering
• Compute environment (same notebook kernel or separate training instance)
• Output: trained model artifacts
D. Model Evaluation
• Silhouette Score analysis in notebooks
• Selection of best clustering configuration
E. Model Registry & Storage
• S3 bucket or AWS Model Registry for artifacts
F. Deployment Environment
• AWS service (e.g., SageMaker Endpoint, Lambda + API Gateway, or batch job)
• Endpoint for scoring new customer data
G. Reporting/Visualization
• Exported HTML notebooks in reports/ and references/
Architectural patterns & design principles
• Linear pipeline (“staircase”) pattern: each notebook is a discrete stage
• Separation of concerns:
• Data ingestion/cleaning
• Feature engineering
• Modeling/evaluation
• Deployment
• Reproducibility: all steps captured in notebooks that can be rerun end-to-end
• Modular deployment: model artifacts decoupled from code environment, deployed on managed AWS service
Final checklist
• All components have clear, unambiguous labels.
• Call out critical design choices (e.g., stateless notebooks vs. persistent model registry).