# adilmirza975/ml_projects architecture

> Architecture diagram of the GitHub repository adilmirza975/ml_projects, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/adilmirza975/ml_projects
- Repository: https://github.com/adilmirza975/ml_projects

## Overview

1. **Project type and purpose**
   - This is an end-to-end **machine learning application** for student performance prediction.
   - The repository combines an **offline training workflow** with an **online inference web app**.

2. **Two main execution paths**
   - **Training path:** prepares data, trains a model, and writes serialized artifacts.
   - **Serving path:** loads the saved preprocessing/model artifacts and exposes predictions through a web UI.
   - This separation is reflected in `src/pipeline/train_pipeline.py` vs. `src/pipeline/predict_pipeline.py`.

3. **Core ML pipeline structure**
   - The ML logic is organized under `src/components/`:
     - `data_ingestion.py` handles loading and splitting data.
     - `data_transformation.py` handles feature preprocessing.
     - `model_trainer.py` trains and evaluates the estimator.
   - These are orchestrated by the pipeline layer, which keeps the components loosely coupled.

4. **Data flow through training**
   - Raw dataset originates from the notebook data and/or ingested CSVs.
   - Training produces intermediate datasets and final serialized objects in `artifacts/`.
   - The presence of `train.csv`, `test.csv`, `data.csv`, `preprocessor.pkl`, and `model.pkl` shows a classic batch ML workflow with persisted preprocessing and model state.

5. **Model artifact boundary**
   - `artifacts/` is the key boundary between training and inference.
   - The web app does not retrain; it depends on the persisted `preprocessor.pkl` and `model.pkl`.
   - This makes the system deployable as a stateless prediction service after training.

6. **Prediction/serving layer**
   - `src/pipeline/predict_pipeline.py` is the inference-side pipeline.
   - It likely converts user input into model-ready features using the saved preprocessor, then produces predictions from the saved model.
   - This is the bridge between the HTML form and the ML artifacts.

7. **Web application layer**
   - `app.py` and `application.py` suggest a **Flask-style** entry point structure.
   - `templates/` contains the HTML views (`home.html`, `index.html`), and `static/css/style.css` provides UI styling.
   - The web layer is primarily responsible for collecting user inputs and displaying prediction results, not training logic.

8. **Deployment target**
   - `.ebextensions/python.config` indicates the project is configured for **AWS Elastic Beanstalk** deployment.
   - This means the app is intended to run as a managed Python web service with environment configuration handled externally.

9. **Supporting engineering utilities**
   - `src/logger.py` and `src/exception.py` provide cross-cutting concerns for observability and error handling.
   - `src/utils.py` likely contains shared helper functions used by multiple pipeline stages.
   - These modules support maintainability across both training and serving code paths.

10. **Project packaging and dependencies**
   - `setup.py` and `requirements.txt` define packaging and Python dependencies.
   - `src/__init.py` and package submodules organize the code as an importable application rather than a notebook-only prototype.

11. **Notebook layer**
   - `notebook/EDA STUDENT PERFORMANCE.ipynb` and `notebook/MODEL TRAINER.ipynb` show the exploratory and experimental workflow used before formalizing the pipeline.
   - These notebooks are not part of runtime execution, but they likely informed the final component design and model choice.

12. **Training diagnostics**
   - `catboost_info/` and similar notebook-local training outputs indicate CatBoost was used at some point or during experimentation.
   - These directories are training diagnostics rather than runtime dependencies, so they belong to the model development side of the architecture.

13. **High-level architecture summary**
   - The repository is a **three-layer ML system**:
     - **Data/model development:** notebooks and training scripts
     - **Core ML backend:** ingestion → transformation → model training
     - **Delivery layer:** Flask web UI with saved artifacts for inference
   - The central design principle is that **training is offline, prediction is online, and the artifact directory is the contract between them**.

Main entry points: [`src/pipeline/train_pipeline.py`](https://github.com/adilmirza975/ml_projects/blob/main/src/pipeline/train_pipeline.py) (Train Pipeline), [`app.py`](https://github.com/adilmirza975/ml_projects/blob/main/app.py) (App), [`src/pipeline/predict_pipeline.py`](https://github.com/adilmirza975/ml_projects/blob/main/src/pipeline/predict_pipeline.py) (Predict Pipeline), [`src/components/data_ingestion.py`](https://github.com/adilmirza975/ml_projects/blob/main/src/components/data_ingestion.py) (Ingestion)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_group_dev["Development"]
  node_node_notebook_eda["EDA Notebook<br/>analysis notebook"]
  node_node_notebook_train["Trainer Notebook<br/>experiment notebook"]
  node_node_dataset["Sample Data<br/>source dataset<br/>[stud.csv]"]
end

subgraph group_group_core["Core ML"]
  node_node_train_pipeline["Train Pipeline<br/>orchestrator<br/>[train_pipeline.py]"]
  node_node_predict_pipeline["Predict Pipeline<br/>inference orchestrator"]
  node_node_ingestion["Ingestion<br/>data loading<br/>[data_ingestion.py]"]
  node_node_transform["Transformation<br/>feature preprocessing"]
  node_node_trainer["Model Trainer<br/>model training<br/>[model_trainer.py]"]
  node_node_utils["Utils<br/>shared helpers<br/>[utils.py]"]
  node_node_logger["Logger<br/>observability<br/>[logger.py]"]
  node_node_exception["Exceptions<br/>error handling<br/>[exception.py]"]
end

subgraph group_group_delivery["Delivery"]
  node_node_app["App<br/>web entrypoint<br/>[app.py]"]
  node_node_application["WSGI App<br/>server entrypoint<br/>[application.py]"]
  node_node_templates["Templates<br/>html views<br/>[index.html]"]
  node_node_home["Home Page<br/>html view<br/>[home.html]"]
  node_node_css["Stylesheet<br/>ui styling<br/>[style.css]"]
end

subgraph group_group_ops["Deployment"]
  node_node_eb["EB Config<br/>deployment config<br/>[python.config]"]
  node_node_setup["Setup<br/>packaging<br/>[setup.py]"]
  node_node_requirements["Deps<br/>dependencies<br/>[requirements.txt]"]
end

subgraph group_group_artifacts["Artifacts"]
  node_node_artifacts[("Artifacts<br/>persisted model state<br/>[model.pkl]")]
  node_node_preprocessor[("Preprocessor<br/>serialized transformer<br/>[preprocessor.pkl]")]
  node_node_model[("Model<br/>serialized estimator<br/>[model.pkl]")]
  node_node_train_csv["Train CSV<br/>training split<br/>[train.csv]"]
  node_node_test_csv["Test CSV<br/>validation split<br/>[test.csv]"]
  node_node_data_csv["Data CSV<br/>ingested dataset<br/>[data.csv]"]
end

node_node_readme["README<br/>project overview<br/>[README.md]"]

node_node_notebook_eda -->|"explores"| node_node_dataset
node_node_notebook_train -->|"trains on"| node_node_dataset
node_node_train_pipeline -->|"orchestrates"| node_node_ingestion
node_node_train_pipeline -->|"orchestrates"| node_node_transform
node_node_train_pipeline -->|"orchestrates"| node_node_trainer
node_node_ingestion -->|"writes"| node_node_data_csv
node_node_ingestion -->|"splits"| node_node_train_csv
node_node_ingestion -->|"splits"| node_node_test_csv
node_node_transform -->|"serializes"| node_node_preprocessor
node_node_trainer -->|"serializes"| node_node_model
node_node_train_pipeline -->|"publishes"| node_node_artifacts
node_node_predict_pipeline -->|"loads"| node_node_preprocessor
node_node_predict_pipeline -->|"loads"| node_node_model
node_node_app -->|"calls"| node_node_predict_pipeline
node_node_app -->|"renders"| node_node_templates
node_node_templates -->|"includes"| node_node_home
node_node_app -->|"serves"| node_node_css
node_node_application -->|"boots"| node_node_app
node_node_app -->|"logs"| node_node_logger
node_node_train_pipeline -->|"logs"| node_node_logger
node_node_predict_pipeline -->|"handles"| node_node_exception
node_node_train_pipeline -->|"uses"| node_node_utils
node_node_predict_pipeline -->|"uses"| node_node_utils
node_node_eb -->|"deploys"| node_node_app
node_node_setup -->|"packages"| node_node_requirements

click node_node_readme "https://github.com/adilmirza975/ml_projects/blob/main/README.md"
click node_node_notebook_eda "https://github.com/adilmirza975/ml_projects/blob/main/notebook/EDA STUDENT PERFORMANCE.ipynb"
click node_node_notebook_train "https://github.com/adilmirza975/ml_projects/blob/main/notebook/MODEL TRAINER.ipynb"
click node_node_dataset "https://github.com/adilmirza975/ml_projects/blob/main/notebook/data/stud.csv"
click node_node_train_pipeline "https://github.com/adilmirza975/ml_projects/blob/main/src/pipeline/train_pipeline.py"
click node_node_predict_pipeline "https://github.com/adilmirza975/ml_projects/blob/main/src/pipeline/predict_pipeline.py"
click node_node_ingestion "https://github.com/adilmirza975/ml_projects/blob/main/src/components/data_ingestion.py"
click node_node_transform "https://github.com/adilmirza975/ml_projects/blob/main/src/components/data_transformation.py"
click node_node_trainer "https://github.com/adilmirza975/ml_projects/blob/main/src/components/model_trainer.py"
click node_node_utils "https://github.com/adilmirza975/ml_projects/blob/main/src/utils.py"
click node_node_logger "https://github.com/adilmirza975/ml_projects/blob/main/src/logger.py"
click node_node_exception "https://github.com/adilmirza975/ml_projects/blob/main/src/exception.py"
click node_node_app "https://github.com/adilmirza975/ml_projects/blob/main/app.py"
click node_node_application "https://github.com/adilmirza975/ml_projects/blob/main/application.py"
click node_node_templates "https://github.com/adilmirza975/ml_projects/blob/main/templates/index.html"
click node_node_home "https://github.com/adilmirza975/ml_projects/blob/main/templates/home.html"
click node_node_css "https://github.com/adilmirza975/ml_projects/blob/main/static/css/style.css"
click node_node_artifacts "https://github.com/adilmirza975/ml_projects/blob/main/artifacts/model.pkl"
click node_node_preprocessor "https://github.com/adilmirza975/ml_projects/blob/main/artifacts/preprocessor.pkl"
click node_node_model "https://github.com/adilmirza975/ml_projects/blob/main/artifacts/model.pkl"
click node_node_train_csv "https://github.com/adilmirza975/ml_projects/blob/main/artifacts/train.csv"
click node_node_test_csv "https://github.com/adilmirza975/ml_projects/blob/main/artifacts/test.csv"
click node_node_data_csv "https://github.com/adilmirza975/ml_projects/blob/main/artifacts/data.csv"
click node_node_eb "https://github.com/adilmirza975/ml_projects/blob/main/.ebextensions/python.config"
click node_node_setup "https://github.com/adilmirza975/ml_projects/blob/main/setup.py"
click node_node_requirements "https://github.com/adilmirza975/ml_projects/blob/main/requirements.txt"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_node_notebook_eda,node_node_notebook_train,node_node_dataset toneBlue
class node_node_train_pipeline,node_node_predict_pipeline,node_node_ingestion,node_node_transform,node_node_trainer,node_node_utils,node_node_logger,node_node_exception toneAmber
class node_node_app,node_node_application,node_node_templates,node_node_home,node_node_css toneMint
class node_node_eb,node_node_setup,node_node_requirements toneRose
class node_node_artifacts,node_node_preprocessor,node_node_model,node_node_train_csv,node_node_test_csv,node_node_data_csv toneIndigo
class node_node_readme toneNeutral
```

## Components

### Development

Notebooks and experiment outputs used to explore data and prototype the model.

- [EDA Notebook](https://github.com/adilmirza975/ml_projects/blob/HEAD/notebook/EDA%20STUDENT%20PERFORMANCE.ipynb): analysis notebook · Explores the student performance dataset and informs feature/model choices. · `notebook/EDA STUDENT PERFORMANCE.ipynb`
- [Trainer Notebook](https://github.com/adilmirza975/ml_projects/blob/HEAD/notebook/MODEL%20TRAINER.ipynb): experiment notebook · Notebook-based model development and training workflow. · `notebook/MODEL TRAINER.ipynb`
- [Sample Data](https://github.com/adilmirza975/ml_projects/blob/main/notebook/data/stud.csv): source dataset · Notebook dataset used for exploration and experimentation. · `notebook/data/stud.csv`

### Core ML

Reusable training and inference pipeline code.

- [Train Pipeline](https://github.com/adilmirza975/ml_projects/blob/main/src/pipeline/train_pipeline.py): orchestrator · Coordinates the offline training workflow from raw data to saved artifacts. · `src/pipeline/train_pipeline.py`
- [Predict Pipeline](https://github.com/adilmirza975/ml_projects/blob/main/src/pipeline/predict_pipeline.py): inference orchestrator · Loads saved preprocessing and model artifacts to serve predictions. · `src/pipeline/predict_pipeline.py`
- [Ingestion](https://github.com/adilmirza975/ml_projects/blob/main/src/components/data_ingestion.py): data loading · Reads raw data and prepares train/test splits. · `src/components/data_ingestion.py`
- [Transformation](https://github.com/adilmirza975/ml_projects/blob/main/src/components/data_transformation.py): feature preprocessing · Builds and applies preprocessing for model-ready features. · `src/components/data_transformation.py`
- [Model Trainer](https://github.com/adilmirza975/ml_projects/blob/main/src/components/model_trainer.py): model training · Trains and evaluates the estimator, then persists the model artifact. · `src/components/model_trainer.py`
- [Utils](https://github.com/adilmirza975/ml_projects/blob/main/src/utils.py): shared helpers · Common helper functions reused across pipeline stages. · `src/utils.py`
- [Logger](https://github.com/adilmirza975/ml_projects/blob/main/src/logger.py): observability · Central logging support for the application and pipelines. · `src/logger.py`
- [Exceptions](https://github.com/adilmirza975/ml_projects/blob/main/src/exception.py): error handling · Shared exception and error reporting utilities. · `src/exception.py`

### Delivery

Web app entry points, templates, and static assets for serving predictions.

- [App](https://github.com/adilmirza975/ml_projects/blob/main/app.py): web entrypoint · Primary web application entry point for request handling. · `app.py`
- [WSGI App](https://github.com/adilmirza975/ml_projects/blob/main/application.py): server entrypoint · Application bootstrap/launch module for the Flask-style service. · `application.py`
- [Templates](https://github.com/adilmirza975/ml_projects/blob/main/templates/index.html): html views · Rendered pages for the prediction UI. · `templates/index.html`
- [Home Page](https://github.com/adilmirza975/ml_projects/blob/main/templates/home.html): html view · Landing or form page for collecting user inputs. · `templates/home.html`
- [Stylesheet](https://github.com/adilmirza975/ml_projects/blob/main/static/css/style.css): ui styling · Presentation styling for the web interface. · `static/css/style.css`

### Deployment

Environment and packaging files for running the application.

- [EB Config](https://github.com/adilmirza975/ml_projects/blob/main/.ebextensions/python.config): deployment config · Elastic Beanstalk runtime configuration for Python deployment. · `.ebextensions/python.config`
- [Setup](https://github.com/adilmirza975/ml_projects/blob/main/setup.py): packaging · Package metadata and install configuration. · `setup.py`
- [Deps](https://github.com/adilmirza975/ml_projects/blob/main/requirements.txt): dependencies · Python dependency list for training and serving. · `requirements.txt`

### Artifacts

Persisted datasets and serialized objects shared between training and inference.

- [Artifacts](https://github.com/adilmirza975/ml_projects/blob/main/artifacts/model.pkl): persisted model state · Shared boundary between offline training and online inference. · `artifacts/model.pkl`
- [Preprocessor](https://github.com/adilmirza975/ml_projects/blob/main/artifacts/preprocessor.pkl): serialized transformer · Saved preprocessing pipeline used during inference. · `artifacts/preprocessor.pkl`
- [Model](https://github.com/adilmirza975/ml_projects/blob/main/artifacts/model.pkl): serialized estimator · Trained prediction model loaded by the serving path. · `artifacts/model.pkl`
- [Train CSV](https://github.com/adilmirza975/ml_projects/blob/main/artifacts/train.csv): training split · Persisted training split produced by ingestion. · `artifacts/train.csv`
- [Test CSV](https://github.com/adilmirza975/ml_projects/blob/main/artifacts/test.csv): validation split · Persisted test split produced by ingestion. · `artifacts/test.csv`
- [Data CSV](https://github.com/adilmirza975/ml_projects/blob/main/artifacts/data.csv): ingested dataset · Persisted canonical dataset snapshot used in the pipeline. · `artifacts/data.csv`

### Other components

- [README](https://github.com/adilmirza975/ml_projects/blob/main/README.md): project overview · Top-level project documentation and usage context. · `README.md`

## Connections

- EDA Notebook → Sample Data: explores · EDA uses the sample dataset for analysis.
- Trainer Notebook → Sample Data: trains on · Notebook training experiments consume the prepared dataset.
- Train Pipeline → Ingestion: orchestrates · Training pipeline starts with data ingestion.
- Train Pipeline → Transformation: orchestrates · Training pipeline applies feature transformation.
- Train Pipeline → Model Trainer: orchestrates · Training pipeline invokes model training.
- Ingestion → Data CSV: writes · Ingestion persists the canonical dataset snapshot.
- Ingestion → Train CSV: splits · Ingestion produces the training split artifact.
- Ingestion → Test CSV: splits · Ingestion produces the test split artifact.
- Transformation → Preprocessor: serializes · Transformation saves the fitted preprocessing pipeline.
- Model Trainer → Model: serializes · Model trainer writes the trained estimator artifact.
- Train Pipeline → Artifacts: publishes · Training pipeline outputs its results to the artifact boundary.
- Predict Pipeline → Preprocessor: loads · Inference pipeline restores the saved preprocessor.
- Predict Pipeline → Model: loads · Inference pipeline restores the trained model.
- App → Predict Pipeline: calls · Web request handling delegates prediction work to the inference pipeline.
- App → Templates: renders · Application renders HTML views for user interaction.
- Templates → Home Page: includes · Template set contains the home or input page.
- App → Stylesheet: serves · Application serves styling assets to the browser.
- WSGI App → App: boots · WSGI/application bootstrap initializes the web app.
- App → Logger: logs · Web layer uses shared logging support.
- Train Pipeline → Logger: logs · Training workflow uses shared logging support.
- Predict Pipeline → Exceptions: handles · Inference path uses shared error handling utilities.
- Train Pipeline → Utils: uses · Training workflow relies on shared helpers.
- Predict Pipeline → Utils: uses · Inference workflow relies on shared helpers.
- EB Config → App: deploys · Elastic Beanstalk configuration targets the web application.
- Setup → Deps: packages · Packaging and dependency metadata define the runtime environment.