# alaindelong2k/end-to-end-machine-learning-project architecture

> Architecture diagram of the GitHub repository alaindelong2k/end-to-end-machine-learning-project, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/alaindelong2k/end-to-end-machine-learning-project
- Repository: https://github.com/alaindelong2k/end-to-end-machine-learning-project

## Overview

##### Project type and purpose
- This is an end-to-end machine learning web application (“Student Math Score Predictor”) built in Python. It bundles an ML model training/serving pipeline (data ingestion, transformation, model training, prediction) together with a Streamlit-based user interface for real-time score prediction.

##### File structure
- Top-level entries:
- app.py – Streamlit application entry point (frontend + serving logic)
- artifacts/ – serialized inputs (CSV) and outputs (trained model.pkl, preprocessor.pkl, train/test splits)
- src/ – production code implementing the pipeline components (data_ingestion, data_transformation, model_trainer, predict_pipeline, train_pipeline), utilities, logger, exception handlers
- notebook/ – Jupyter notebooks and parallel source tree for EDA and exploratory pipeline code
- requirements.txt, setup.py – dependency and packaging metadata
- Separation of “pipeline” (train_pipeline.py, predict_pipeline.py) and “components” (ingestion/transformation/trainer), conforming to a layered pipeline architecture

##### Extract additional insights from the README
- Data source: Kaggle “Students Performance in Exams” CSV
- Models experimented with: Linear Regression, KNN, Decision Tree, XGBoost
- Hyperparameter tuning via Grid/Random Search
- Deployed as a Streamlit web app (hosted on Hugging Face Spaces)
- User inputs demographic + prior scores, UI triggers prediction and displays math score

- User Interface (Streamlit app in app.py)
- Prediction Service (predict_pipeline loading preprocessor.pkl & model.pkl)
- Training Pipeline (train_pipeline invoking data_ingestion → data_transformation → model_trainer → artifacts)
- Data Store (raw CSV dataset, train/test CSVs in artifacts/)
- Artifact Repository (serialized objects: model.pkl, preprocessor.pkl)
- Logging & Monitoring (logger.py, exception.py)
- External Services (Kaggle dataset source, hosting platform – e.g., Hugging Face Spaces)
b. Relationships and data flow:
1. Training flow (offline): raw data → DataIngestion → DataTransformation → ModelTrainer → serialized artifacts
2. Serving flow (online): user inputs → Streamlit app → PredictPipeline → loads preprocessor + model → transforms inputs → outputs prediction → UI display
3. Logging & exception paths intercept both flows for audit and error handling
c. Architectural patterns and design principles:
- Layered pipeline (clear separation between ingestion, transformation, and modeling)
- Modular components for testability and reuse (src/components)
- Single-page web UI pattern (Streamlit) with synchronous request-response for prediction
- Artifact versioning for reproducibility
d. Key technologies and frameworks:
- Python, pandas, scikit-learn (data/scaling)
- CatBoost (model training + evaluation)
- Streamlit (web interface)
- Logging module (custom logger.py)
- Packaging via setup.py, dependency control via requirements.txt

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Data Source
    Kaggle["Kaggle Dataset Source"]:::external

    %% Offline Training Pipeline
    subgraph "Offline Training"
        DI["DataIngestion Component"]:::codeModules
        DT["DataTransformation Component"]:::codeModules
        MT["ModelTrainer Component"]:::codeModules
        TP["Training Pipeline"]:::codeModules
        AS["Artifacts Store"]:::storage
        CB["CatBoost Training Metadata"]:::storage
        LG["Logging Module"]:::codeModules
        EX["Exception Handling Module"]:::codeModules
        UT["Utility Functions"]:::codeModules
    end

    %% Online Serving Pipeline
    subgraph "Online Serving"
        UI["Streamlit UI"]:::ui
        PP["PredictPipeline"]:::codeModules
        AS2["Artifacts Store"]:::storage
        LG2["Logging Module"]:::codeModules
        EX2["Exception Handling Module"]:::codeModules
    end

    %% External Hosting
    HF["Hugging Face Spaces"]:::external

    %% Connections - Offline
    Kaggle -->|ingests raw CSV| DI
    DI -->|outputs cleaned data| DT
    DT -->|features for model| MT
    MT -->|trained model| TP
    TP -->|saves artifacts| AS
    TP -->|saves metadata| CB
    DI --> LG
    DT --> LG
    MT --> LG
    DI --> EX
    DT --> EX
    MT --> EX
    UT --> DI
    UT --> DT
    UT --> MT

    %% Connections - Online
    UI -->|user inputs| PP
    PP -->|load preprocessor & model| AS2
    AS2 -->|provides artifacts| PP
    PP -->|prediction result| UI
    UI --> HF
    PP --> LG2
    PP --> EX2

    %% Click Events
    click UI "https://github.com/alaindelong2k/end-to-end-machine-learning-project/blob/main/app.py"
    click PP "https://github.com/alaindelong2k/end-to-end-machine-learning-project/blob/main/src/pipeline/predict_pipeline.py"
    click TP "https://github.com/alaindelong2k/end-to-end-machine-learning-project/blob/main/src/pipeline/train_pipeline.py"
    click DI "https://github.com/alaindelong2k/end-to-end-machine-learning-project/blob/main/src/components/data_ingestion.py"
    click DT "https://github.com/alaindelong2k/end-to-end-machine-learning-project/blob/main/src/components/data_transformation.py"
    click MT "https://github.com/alaindelong2k/end-to-end-machine-learning-project/blob/main/src/components/model_trainer.py"
    click AS "https://github.com/alaindelong2k/end-to-end-machine-learning-project/tree/main/artifacts/"
    click AS2 "https://github.com/alaindelong2k/end-to-end-machine-learning-project/tree/main/artifacts/"
    click LG "https://github.com/alaindelong2k/end-to-end-machine-learning-project/blob/main/src/logger.py"
    click LG2 "https://github.com/alaindelong2k/end-to-end-machine-learning-project/blob/main/src/logger.py"
    click EX "https://github.com/alaindelong2k/end-to-end-machine-learning-project/blob/main/src/exception.py"
    click EX2 "https://github.com/alaindelong2k/end-to-end-machine-learning-project/blob/main/src/exception.py"
    click UT "https://github.com/alaindelong2k/end-to-end-machine-learning-project/blob/main/src/utils.py"
    click CB "https://github.com/alaindelong2k/end-to-end-machine-learning-project/tree/main/catboost_info/"

    %% Styles
    classDef codeModules fill:#D6EAF8,stroke:#1B4F72,stroke-width:1px;
    classDef storage fill:#ABEBC6,stroke:#186A3B,stroke-width:1px,stroke-dasharray: 5 5,shape:cylinder;
    classDef ui fill:#FAD7A0,stroke:#B9770E,stroke-width:1px,shape:parallelogram;
    classDef external fill:#E5E7E9,stroke:#515A5A,stroke-width:1px,shape:cloud;
```
