# vidhanagarwal594-oss/machine_learning architecture

> Architecture diagram of the GitHub repository vidhanagarwal594-oss/machine_learning, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/vidhanagarwal594-oss/machine_learning
- Repository: https://github.com/vidhanagarwal594-oss/machine_learning

## Overview

1. **Repository shape**
   - This is a small, notebook-driven machine learning repo.
   - The only code artifacts are two Jupyter notebooks: `model_using_pipeline1 (3).ipynb` and `production.ipynb`.
   - There is no separate application layer, API server, or package/module structure visible in the repository tree.

2. **Primary purpose**
   - The repo demonstrates a single end-to-end Scikit-learn preprocessing-and-training workflow.
   - The core architectural idea is to keep all preprocessing inside the model pipeline so that training and inference use the same transformations.

3. **Execution environment**
   - The system is built for interactive Python execution in Jupyter.
   - The main runtime dependencies are Python, NumPy, Pandas, and Scikit-learn.
   - Any persistence/deployment mentioned in the README is optional and would likely rely on `joblib` or `pickle`, but no saved-model artifacts are present in the tree.

4. **Notebook roles**
   - The notebooks appear to be the operational boundary of the project.
   - `model_using_pipeline1 (3).ipynb` likely contains the core pipeline construction and model training workflow.
   - `production.ipynb` likely represents the inference-oriented or deployment-oriented version of the same workflow, intended to reuse the trained pipeline for prediction.

5. **Data flow**
   - The documented flow is: raw dataset → train/test split → preprocessing → feature transformation → pipeline → model training → prediction.
   - The train/test split happens before fitting preprocessing, which is the key boundary that prevents leakage.
   - Training data drives fitting; test or inference data only passes through already-fitted transformations.

6. **Preprocessing subsystem**
   - Preprocessing is split by feature type using Scikit-learn primitives.
   - Numeric features are handled with missing-value imputation and scaling.
   - Ordinal categorical features use `OrdinalEncoder`.
   - Nominal categorical features use `OneHotEncoder`.
   - `ColumnTransformer` is the orchestration layer that applies the right transformation to each column group.

7. **Modeling subsystem**
   - A Scikit-learn `Pipeline` wraps preprocessing and the estimator into one composite object.
   - This makes the model the top-level deployable unit rather than separate preprocessing scripts plus model code.
   - The estimator itself is not specified in the README, so the diagram should show it as an abstract “ML model” block rather than naming a specific algorithm.

8. **Key architectural boundary**
   - The important separation is between raw tabular input and the fitted pipeline.
   - All transformations are learned only from training data and then reused unchanged for predictions.
   - This boundary is what enforces consistency between offline training and later prediction in `production.ipynb`.

9. **Operational behavior**
   - During training, the pipeline fits imputers, encoders, scaler, and model in sequence.
   - During inference, the same pipeline transforms new rows and emits predictions.
   - There is no evidence of batch orchestration, scheduling, or distributed processing; the architecture is local and single-process.

10. **Persistence and reuse**
   - The README explicitly calls out saving and deploying the pipeline with `joblib` or `pickle`.
   - That implies the intended deployable artifact is the serialized pipeline object, not individual preprocessing components.
   - No persisted artifacts are included in the repository tree, so serialization is a planned capability rather than a committed component.

11. **Main external dependency surface**
   - The architecture depends primarily on Scikit-learn’s preprocessing and composition APIs.
   - Pandas likely provides tabular data loading/manipulation, while NumPy supports numerical operations.
   - There are no visible external services, databases, message queues, or cloud integrations.

12. **How to draw the diagram**
   - Show two notebook entrypoints feeding the same logical ML workflow.
   - Inside that workflow, depict: dataset input → split → column-wise preprocessing → unified pipeline → estimator → predictions.
   - Emphasize the `ColumnTransformer` and `Pipeline` as the central composition mechanisms, with serialization as an optional end state for production use.

Main entry points: [`model_using_pipeline1 (3).ipynb`](https://github.com/vidhanagarwal594-oss/machine_learning/blob/HEAD/model_using_pipeline1%20(3).ipynb) (Training notebook), [`production.ipynb`](https://github.com/vidhanagarwal594-oss/machine_learning/blob/main/production.ipynb) (Production notebook), [`README.md`](https://github.com/vidhanagarwal594-oss/machine_learning/blob/main/README.md) (README)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_notebooks["Notebooks"]
  node_train_nb["Training notebook<br/>jupyter notebook"]
  node_prod_nb["Production notebook<br/>jupyter notebook<br/>[production.ipynb]"]
end

node_readme["README<br/>project overview<br/>[README.md]"]
node_raw_data[("Raw data<br/>tabular input")]
node_split["Train/test split<br/>data partition"]
node_num_select["Numeric cols<br/>feature selection"]
node_cat_select["Categorical cols<br/>feature selection"]
node_num_impute["Imputer<br/>numeric preprocessing"]
node_scale["Scaler<br/>numeric preprocessing"]
node_ord_enc["Ordinal encoder<br/>categorical preprocessing"]
node_onehot["One-hot encoder<br/>categorical preprocessing"]
node_cols["ColumnTransformer<br/>feature orchestration"]
node_pipeline["Pipeline<br/>model composition"]
node_model(("ML model<br/>estimator"))
node_predictions[("Predictions<br/>output")]
node_serialized["Serialized pipeline<br/>artifact"]

node_readme -.->|"documents"| node_train_nb
node_readme -.->|"documents"| node_prod_nb
node_train_nb -->|"loads"| node_raw_data
node_prod_nb -->|"loads"| node_serialized
node_raw_data -->|"split"| node_split
node_split -->|"routes numeric"| node_num_select
node_split -->|"routes categorical"| node_cat_select
node_num_select -->|"preprocess"| node_num_impute
node_num_impute -->|"transform"| node_scale
node_cat_select -->|"encode ordered"| node_ord_enc
node_cat_select -->|"encode nominal"| node_onehot
node_scale -->|"combine"| node_cols
node_ord_enc -->|"combine"| node_cols
node_onehot -->|"combine"| node_cols
node_cols -->|"feed"| node_pipeline
node_pipeline -->|"fit"| node_model
node_model -->|"predict"| node_predictions
node_pipeline -.->|"save"| node_serialized
node_train_nb -->|"builds"| node_pipeline
node_prod_nb -->|"reuses"| node_pipeline

click node_readme "https://github.com/vidhanagarwal594-oss/machine_learning/blob/main/README.md"
click node_train_nb "https://github.com/vidhanagarwal594-oss/machine_learning/blob/main/model_using_pipeline1 (3).ipynb"
click node_prod_nb "https://github.com/vidhanagarwal594-oss/machine_learning/blob/main/production.ipynb"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_train_nb,node_prod_nb toneBlue
class node_readme,node_raw_data,node_split,node_num_select,node_cat_select,node_num_impute,node_scale,node_ord_enc,node_onehot,node_cols,node_pipeline,node_model,node_predictions,node_serialized toneNeutral
```

## Components

### Notebooks

Jupyter entrypoints for training and production inference

- [Training notebook](https://github.com/vidhanagarwal594-oss/machine_learning/blob/HEAD/model_using_pipeline1%20(3).ipynb): jupyter notebook · Core end-to-end preprocessing and model training workflow · `model_using_pipeline1 (3).ipynb`
- [Production notebook](https://github.com/vidhanagarwal594-oss/machine_learning/blob/main/production.ipynb): jupyter notebook · Inference-oriented reuse of the fitted pipeline for prediction · `production.ipynb`

### Other components

- [README](https://github.com/vidhanagarwal594-oss/machine_learning/blob/main/README.md): project overview · Describes the notebook-driven pipeline workflow and deployment intent · `README.md`
- **Raw data**: tabular input · Source dataset before any split or transformation
- **Train/test split**: data partition · Separates training rows from held-out evaluation or inference data
- **Numeric cols**: feature selection · Numeric feature subset routed to numeric preprocessing
- **Categorical cols**: feature selection · Categorical feature subset routed to categorical preprocessing
- **Imputer**: numeric preprocessing · Fills missing numeric values before scaling
- **Scaler**: numeric preprocessing · Normalizes numeric features for model input
- **Ordinal encoder**: categorical preprocessing · Encodes ordered categories as numeric ranks
- **One-hot encoder**: categorical preprocessing · Expands nominal categories into indicator features
- **ColumnTransformer**: feature orchestration · Applies the correct preprocessing branch per column group
- **Pipeline**: model composition · Wraps preprocessing and estimator into one reusable object
- **ML model**: estimator · Trained predictive model used for fitting and prediction
- **Predictions**: output · Predicted values emitted from the fitted pipeline
- **Serialized pipeline**: artifact · Optional saved pipeline for reuse with joblib or pickle

## Connections

- README → Training notebook: documents · Explains the training workflow shown in the notebook
- README → Production notebook: documents · Describes production-style reuse of the fitted pipeline
- Training notebook → Raw data: loads · Notebook starts from tabular dataset input
- Production notebook → Serialized pipeline: loads · Production notebook is intended to reuse a saved pipeline
- Raw data → Train/test split: split · Training data is separated before fitting preprocessing
- Train/test split → Numeric cols: routes numeric · Numeric columns are selected from the training frame
- Train/test split → Categorical cols: routes categorical · Categorical columns are selected from the training frame
- Numeric cols → Imputer: preprocess · Numeric branch imputes missing values
- Imputer → Scaler: transform · Imputed numeric values are scaled
- Categorical cols → Ordinal encoder: encode ordered · Ordered categorical features use ordinal encoding
- Categorical cols → One-hot encoder: encode nominal · Nominal categorical features use one-hot encoding
- Scaler → ColumnTransformer: combine · Numeric features rejoin the column-wise transformer
- Ordinal encoder → ColumnTransformer: combine · Ordinal categorical features rejoin the column-wise transformer
- One-hot encoder → ColumnTransformer: combine · One-hot features rejoin the column-wise transformer
- ColumnTransformer → Pipeline: feed · ColumnTransformer output becomes model input
- Pipeline → ML model: fit · Estimator is trained as the final pipeline step
- ML model → Predictions: predict · Fitted pipeline emits predictions on new rows
- Pipeline → Serialized pipeline: save · Optional persistence of the full preprocessing-and-model bundle
- Training notebook → Pipeline: builds · Training notebook constructs the reusable pipeline object
- Production notebook → Pipeline: reuses · Production notebook applies the same fitted pipeline for inference