# aitullalimon/end-to-end-ml-production-pipeline architecture

> Architecture diagram of the GitHub repository aitullalimon/end-to-end-ml-production-pipeline, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/aitullalimon/end-to-end-ml-production-pipeline
- Repository: https://github.com/aitullalimon/end-to-end-ml-production-pipeline

## Overview

This repository is a reusable classification and regression ML pipeline template. Its documented workflow runs from data ingestion and EDA through preprocessing, training, validation, model saving, and batch inference. The notebook is the documented home for training; its implementation was not sampled, so training-stage links follow the README. The CLI’s model loading, prediction, and CSV export are supported by sampled source.

Main entry points: [`01_setup_and_data.ipynb`](https://github.com/aitullalimon/end-to-end-ml-production-pipeline/blob/main/01_setup_and_data.ipynb) (ML workflow), [`src/predict_cli.py`](https://github.com/aitullalimon/end-to-end-ml-production-pipeline/blob/main/src/predict_cli.py) (Inference CLI), [`data/raw`](https://github.com/aitullalimon/end-to-end-ml-production-pipeline/tree/main/data/raw) (Raw datasets), [`src/data_loader.py`](https://github.com/aitullalimon/end-to-end-ml-production-pipeline/blob/main/src/data_loader.py) (Data loading)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_training["Model development"]
  node_data_loader["Data loading<br/>[data_loader.py]"]
  node_notebook["ML workflow"]
  node_eda["EDA reports"]
  node_preprocessing["Preprocessing"]
  node_training_stage["Model training"]
  node_validation["Cross-validation"]
  node_metrics["Evaluation metrics"]
end

subgraph group_inference["Batch inference"]
  node_model_store[("Saved models")]
  node_cli["Inference CLI<br/>[predict_cli.py]"]
end

node_user(("Notebook user"))
node_batch_user(("Batch user"))
node_datasets["Raw datasets"]
node_input_csv["Input CSV<br/>[sample_cls.csv]"]
node_prediction["Predictions"]

node_user -->|"runs"| node_notebook
node_datasets -->|"loads"| node_data_loader
node_notebook -->|"generates"| node_eda
node_notebook -->|"prepares features"| node_preprocessing
node_preprocessing -->|"transforms"| node_training_stage
node_training_stage -->|"validates"| node_validation
node_training_stage -->|"evaluates"| node_metrics
node_notebook -->|"saves models"| node_model_store
node_batch_user -->|"runs"| node_cli
node_cli -->|"reads"| node_input_csv
node_cli -->|"loads model"| node_model_store
node_cli -->|"writes CSV"| node_prediction

click node_datasets "https://github.com/aitullalimon/end-to-end-ml-production-pipeline/tree/main/data/raw"
click node_data_loader "https://github.com/aitullalimon/end-to-end-ml-production-pipeline/blob/main/src/data_loader.py"
click node_notebook "https://github.com/aitullalimon/end-to-end-ml-production-pipeline/blob/main/01_setup_and_data.ipynb"
click node_eda "https://github.com/aitullalimon/end-to-end-ml-production-pipeline/tree/main/reports/eda"
click node_preprocessing "https://github.com/aitullalimon/end-to-end-ml-production-pipeline/blob/main/01_setup_and_data.ipynb"
click node_training_stage "https://github.com/aitullalimon/end-to-end-ml-production-pipeline/blob/main/01_setup_and_data.ipynb"
click node_validation "https://github.com/aitullalimon/end-to-end-ml-production-pipeline/blob/main/01_setup_and_data.ipynb"
click node_metrics "https://github.com/aitullalimon/end-to-end-ml-production-pipeline/blob/main/01_setup_and_data.ipynb"
click node_cli "https://github.com/aitullalimon/end-to-end-ml-production-pipeline/blob/main/src/predict_cli.py"
click node_input_csv "https://github.com/aitullalimon/end-to-end-ml-production-pipeline/blob/main/data/sample_cls.csv"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_data_loader,node_notebook,node_eda,node_preprocessing,node_training_stage,node_validation,node_metrics,node_user,node_batch_user toneBlue
class node_model_store,node_cli toneAmber
class node_datasets,node_input_csv toneTeal
class node_prediction toneIndigo
```

## Components

### Model development

- [Data loading](https://github.com/aitullalimon/end-to-end-ml-production-pipeline/blob/main/src/data_loader.py): `src/data_loader.py`
- [ML workflow](https://github.com/aitullalimon/end-to-end-ml-production-pipeline/blob/main/01_setup_and_data.ipynb): `01_setup_and_data.ipynb`
- [EDA reports](https://github.com/aitullalimon/end-to-end-ml-production-pipeline/tree/main/reports/eda): `reports/eda`
- [Preprocessing](https://github.com/aitullalimon/end-to-end-ml-production-pipeline/blob/main/01_setup_and_data.ipynb): `01_setup_and_data.ipynb`
- [Model training](https://github.com/aitullalimon/end-to-end-ml-production-pipeline/blob/main/01_setup_and_data.ipynb): `01_setup_and_data.ipynb`
- [Cross-validation](https://github.com/aitullalimon/end-to-end-ml-production-pipeline/blob/main/01_setup_and_data.ipynb): `01_setup_and_data.ipynb`
- [Evaluation metrics](https://github.com/aitullalimon/end-to-end-ml-production-pipeline/blob/main/01_setup_and_data.ipynb): `01_setup_and_data.ipynb`

### Batch inference

- **Saved models**
- [Inference CLI](https://github.com/aitullalimon/end-to-end-ml-production-pipeline/blob/main/src/predict_cli.py): `src/predict_cli.py`

### Other components

- **Notebook user**
- **Batch user**
- [Raw datasets](https://github.com/aitullalimon/end-to-end-ml-production-pipeline/tree/main/data/raw): `data/raw`
- [Input CSV](https://github.com/aitullalimon/end-to-end-ml-production-pipeline/blob/main/data/sample_cls.csv): `data/sample_cls.csv`
- **Predictions**

## Connections

- Notebook user → ML workflow: runs
- Raw datasets → Data loading: loads
- ML workflow → EDA reports: generates
- ML workflow → Preprocessing: prepares features
- Preprocessing → Model training: transforms
- Model training → Cross-validation: validates
- Model training → Evaluation metrics: evaluates
- ML workflow → Saved models: saves models
- Batch user → Inference CLI: runs
- Inference CLI → Input CSV: reads
- Inference CLI → Saved models: loads model
- Inference CLI → Predictions: writes CSV