# patilsri585/mlops-project-vehicle-insurance architecture

> Architecture diagram of the GitHub repository patilsri585/mlops-project-vehicle-insurance, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-23). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/patilsri585/mlops-project-vehicle-insurance
- Repository: https://github.com/patilsri585/mlops-project-vehicle-insurance

## Overview

1. This repository is an end-to-end MLOps project for vehicle insurance response prediction. It has two primary execution paths: an offline training pipeline that builds and publishes a model, and an online inference path that serves predictions through a FastAPI app.

2. The main runtime entry points are `demo.py` for running the full training workflow and `app.py` for serving predictions. The app layer is lightweight; most business logic lives under `main/`, which is organized as a reusable Python package.

3. The training pipeline is staged and linear: MongoDB ingestion, schema/data validation, data transformation, model training, model evaluation, and model push. Each stage emits a persisted artifact into a timestamped `artifact/` folder, so outputs from one run are isolated from the next.

4. Data ingress starts in `main/configuration/mongo_db_connection.py` and `main/data_access/proj1_data.py`, which connect to MongoDB Atlas and extract the source dataset. The ingestion component writes raw data to a CSV feature store and splits it into train/test files before downstream processing.

5. Validation and transformation are separate boundaries. `data_validation.py` checks incoming data against `config/schema.yaml`, while `data_transformation.py` performs preprocessing such as encoding, scaling, and class-imbalance handling. The fitted preprocessing pipeline is serialized as `preprocessing.pkl`, and transformed datasets are stored as `.npy` arrays.

6. Model training is isolated in `model_trainer.py`, which consumes the transformed training arrays and creates the classifier artifact `model.pkl`. `model_evaluation.py` then compares the trained model against test data and acceptance criteria defined in configuration. This makes model promotion dependent on measured quality, not just successful training.

7. Model publishing is handled by `model_pusher.py` and the AWS helpers in `main/configuration/aws_connection.py` plus `main/cloud_storage/aws_storage.py`. The trained model artifact is uploaded to S3, which acts as the model registry / distribution point for deployment and inference.

8. The inference pipeline in `main/pipeline/prediction_pipeline.py` mirrors the training preprocessing logic to avoid training-serving skew. It accepts user input from the HTML form in `templates/vehicledata.html`, applies custom type conversions and feature alignment, loads the preprocessor and model, and returns a binary prediction.

9. Configuration is externalized in `config/model.yaml` and `config/schema.yaml`. These files define model settings, feature lists, scaling rules, and validation constraints, so the pipeline behavior is controlled by config rather than hardcoded assumptions.

10. Shared domain objects live in `main/entity/`. These dataclasses define configuration and artifact shapes, while `estimator.py` and `s3_estimator.py` wrap model loading and S3-backed access. This keeps orchestration code from depending directly on low-level storage details.

11. Operational concerns are built into the package: `main/logger/` provides rotating file logging, and `main/exception/` standardizes error reporting with traceback context. Generated logs are stored under `logs/`, while notebooks under `notebook/` support exploration and offline experimentation rather than production execution.

12. Deployment is container-based and cloud-backed. The `dockerfile` packages the app, and `.github/workflows/aws.yaml` implements CI/CD that builds the image, pushes it to AWS ECR, then deploys it to an EC2 self-hosted runner. Environment secrets include AWS credentials and the MongoDB connection string, making cloud services part of the runtime boundary.

Main entry points: [`main/pipeline/training_pipeline.py`](https://github.com/patilsri585/mlops-project-vehicle-insurance/blob/main/main/pipeline/training_pipeline.py) (Training Pipeline), [`main/pipeline/prediction_pipeline.py`](https://github.com/patilsri585/mlops-project-vehicle-insurance/blob/main/main/pipeline/prediction_pipeline.py) (Prediction Pipeline), [`main/components/data_transformation.py`](https://github.com/patilsri585/mlops-project-vehicle-insurance/blob/main/main/components/data_transformation.py) (Transformation), [`main/components/model_pusher.py`](https://github.com/patilsri585/mlops-project-vehicle-insurance/blob/main/main/components/model_pusher.py) (Pusher)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Runtimes"]
  node_demo_py["demo.py<br/>training entry<br/>[demo.py]"]
  node_app_py["app.py<br/>serve entry<br/>[app.py]"]
end

subgraph group_pipeline["Pipelines"]
  node_training_pipeline["Training Pipeline<br/>orchestration"]
  node_prediction_pipeline["Prediction Pipeline<br/>inference flow"]
end

subgraph group_components["Core Components"]
  node_data_ingestion["Ingestion<br/>source load<br/>[data_ingestion.py]"]
  node_data_validation["Validation<br/>schema check<br/>[data_validation.py]"]
  node_data_transformation["Transformation<br/>preprocessing"]
  node_model_trainer["Trainer<br/>model fit<br/>[model_trainer.py]"]
  node_model_evaluation["Evaluation<br/>quality gate"]
  node_model_pusher["Pusher<br/>publish step<br/>[model_pusher.py]"]
end

subgraph group_shared["Shared Package"]
  node_entities["Entities<br/>dataclasses<br/>[artifact_entity.py]"]
  node_estimators["Estimators<br/>model wrappers<br/>[estimator.py]"]
  node_logger_exc["Logging & Errors<br/>ops support"]
end

subgraph group_config["Configuration"]
  node_model_yaml["model.yaml<br/>model config<br/>[model.yaml]"]
  node_schema_yaml["schema.yaml<br/>data schema<br/>[schema.yaml]"]
end

subgraph group_infra["Infrastructure"]
  node_mongo_connection[("MongoDB<br/>db connection")]
  node_aws_connection["AWS Setup<br/>cloud connection<br/>[aws_connection.py]"]
  node_aws_storage[("S3 Storage<br/>model registry<br/>[aws_storage.py]")]
end

subgraph group_ui["User Interface"]
  node_vehicle_form["Vehicle Form<br/>html ui<br/>[vehicledata.html]"]
end

node_demo_py -->|"runs"| node_training_pipeline
node_app_py -->|"serves"| node_prediction_pipeline
node_training_pipeline -->|"stage 1"| node_data_ingestion
node_training_pipeline -->|"stage 2"| node_data_validation
node_training_pipeline -->|"stage 3"| node_data_transformation
node_training_pipeline -->|"stage 4"| node_model_trainer
node_training_pipeline -->|"stage 5"| node_model_evaluation
node_training_pipeline -->|"stage 6"| node_model_pusher
node_data_ingestion -->|"reads from"| node_mongo_connection
node_data_validation -->|"checks against"| node_schema_yaml
node_data_transformation -->|"uses config"| node_model_yaml
node_data_transformation -->|"emits"| node_entities
node_model_trainer -->|"emits"| node_entities
node_model_evaluation -->|"thresholds"| node_model_yaml
node_model_pusher -->|"uses"| node_aws_connection
node_model_pusher -->|"uploads to"| node_aws_storage
node_aws_storage -->|"backs"| node_estimators
node_prediction_pipeline -->|"receives"| node_vehicle_form
node_prediction_pipeline -->|"mirrors"| node_data_transformation
node_prediction_pipeline -->|"loads"| node_estimators
node_training_pipeline -->|"logs"| node_logger_exc
node_prediction_pipeline -->|"logs"| node_logger_exc

click node_demo_py "https://github.com/patilsri585/mlops-project-vehicle-insurance/blob/main/demo.py"
click node_app_py "https://github.com/patilsri585/mlops-project-vehicle-insurance/blob/main/app.py"
click node_training_pipeline "https://github.com/patilsri585/mlops-project-vehicle-insurance/blob/main/main/pipeline/training_pipeline.py"
click node_prediction_pipeline "https://github.com/patilsri585/mlops-project-vehicle-insurance/blob/main/main/pipeline/prediction_pipeline.py"
click node_data_ingestion "https://github.com/patilsri585/mlops-project-vehicle-insurance/blob/main/main/components/data_ingestion.py"
click node_data_validation "https://github.com/patilsri585/mlops-project-vehicle-insurance/blob/main/main/components/data_validation.py"
click node_data_transformation "https://github.com/patilsri585/mlops-project-vehicle-insurance/blob/main/main/components/data_transformation.py"
click node_model_trainer "https://github.com/patilsri585/mlops-project-vehicle-insurance/blob/main/main/components/model_trainer.py"
click node_model_evaluation "https://github.com/patilsri585/mlops-project-vehicle-insurance/blob/main/main/components/model_evaluation.py"
click node_model_pusher "https://github.com/patilsri585/mlops-project-vehicle-insurance/blob/main/main/components/model_pusher.py"
click node_mongo_connection "https://github.com/patilsri585/mlops-project-vehicle-insurance/blob/main/main/configuration/mongo_db_connection.py"
click node_aws_connection "https://github.com/patilsri585/mlops-project-vehicle-insurance/blob/main/main/configuration/aws_connection.py"
click node_aws_storage "https://github.com/patilsri585/mlops-project-vehicle-insurance/blob/main/main/cloud_storage/aws_storage.py"
click node_model_yaml "https://github.com/patilsri585/mlops-project-vehicle-insurance/blob/main/config/model.yaml"
click node_schema_yaml "https://github.com/patilsri585/mlops-project-vehicle-insurance/blob/main/config/schema.yaml"
click node_entities "https://github.com/patilsri585/mlops-project-vehicle-insurance/blob/main/main/entity/artifact_entity.py"
click node_estimators "https://github.com/patilsri585/mlops-project-vehicle-insurance/blob/main/main/entity/estimator.py"
click node_logger_exc "https://github.com/patilsri585/mlops-project-vehicle-insurance/tree/main/main/logger"
click node_vehicle_form "https://github.com/patilsri585/mlops-project-vehicle-insurance/blob/main/templates/vehicledata.html"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_demo_py,node_app_py,node_vehicle_form toneBlue
class node_training_pipeline,node_prediction_pipeline toneAmber
class node_data_ingestion,node_data_validation,node_data_transformation,node_model_trainer,node_model_evaluation,node_model_pusher toneMint
class node_entities,node_estimators,node_logger_exc toneRose
class node_model_yaml,node_schema_yaml toneIndigo
class node_mongo_connection,node_aws_connection,node_aws_storage toneTeal
```

## Components

### Runtimes

Main execution entry points for training and inference

- [demo.py](https://github.com/patilsri585/mlops-project-vehicle-insurance/blob/main/demo.py): training entry · Runs the end-to-end offline training workflow · `demo.py`
- [app.py](https://github.com/patilsri585/mlops-project-vehicle-insurance/blob/main/app.py): serve entry · Starts the lightweight FastAPI inference application · `app.py`

### Pipelines

Orchestrates offline training and online prediction flows

- [Training Pipeline](https://github.com/patilsri585/mlops-project-vehicle-insurance/blob/main/main/pipeline/training_pipeline.py): orchestration · Coordinates ingestion through model push · `main/pipeline/training_pipeline.py`
- [Prediction Pipeline](https://github.com/patilsri585/mlops-project-vehicle-insurance/blob/main/main/pipeline/prediction_pipeline.py): inference flow · Prepares input and generates online predictions · `main/pipeline/prediction_pipeline.py`

### Core Components

Staged MLOps processing blocks

- [Ingestion](https://github.com/patilsri585/mlops-project-vehicle-insurance/blob/main/main/components/data_ingestion.py): source load · Pulls data from MongoDB and writes feature store splits · `main/components/data_ingestion.py`
- [Validation](https://github.com/patilsri585/mlops-project-vehicle-insurance/blob/main/main/components/data_validation.py): schema check · Validates data against the configured schema · `main/components/data_validation.py`
- [Transformation](https://github.com/patilsri585/mlops-project-vehicle-insurance/blob/main/main/components/data_transformation.py): preprocessing · Encodes, scales, and balances data for training · `main/components/data_transformation.py`
- [Trainer](https://github.com/patilsri585/mlops-project-vehicle-insurance/blob/main/main/components/model_trainer.py): model fit · Trains the classifier and produces the model artifact · `main/components/model_trainer.py`
- [Evaluation](https://github.com/patilsri585/mlops-project-vehicle-insurance/blob/main/main/components/model_evaluation.py): quality gate · Compares model performance to acceptance criteria · `main/components/model_evaluation.py`
- [Pusher](https://github.com/patilsri585/mlops-project-vehicle-insurance/blob/main/main/components/model_pusher.py): publish step · Uploads the approved model artifact to S3 · `main/components/model_pusher.py`

### Shared Package

Reusable domain objects and support utilities

- [Entities](https://github.com/patilsri585/mlops-project-vehicle-insurance/blob/main/main/entity/artifact_entity.py): dataclasses · Defines config and artifact shapes shared across the app · `main/entity/artifact_entity.py`
- [Estimators](https://github.com/patilsri585/mlops-project-vehicle-insurance/blob/main/main/entity/estimator.py): model wrappers · Wraps local model loading and S3-backed access · `main/entity/estimator.py`
- [Logging & Errors](https://github.com/patilsri585/mlops-project-vehicle-insurance/tree/main/main/logger): ops support · Standardizes rotating logs and traceback-rich exceptions · `main/logger`

### Configuration

Externalized model and schema settings

- [model.yaml](https://github.com/patilsri585/mlops-project-vehicle-insurance/blob/main/config/model.yaml): model config · Holds model and preprocessing settings · `config/model.yaml`
- [schema.yaml](https://github.com/patilsri585/mlops-project-vehicle-insurance/blob/main/config/schema.yaml): data schema · Defines feature constraints and validation rules · `config/schema.yaml`

### Infrastructure

Cloud, deployment, and persistence boundaries

- [MongoDB](https://github.com/patilsri585/mlops-project-vehicle-insurance/blob/main/main/configuration/mongo_db_connection.py): db connection · Connects to MongoDB Atlas for source data access · `main/configuration/mongo_db_connection.py`
- [AWS Setup](https://github.com/patilsri585/mlops-project-vehicle-insurance/blob/main/main/configuration/aws_connection.py): cloud connection · Provides AWS credentials and service access · `main/configuration/aws_connection.py`
- [S3 Storage](https://github.com/patilsri585/mlops-project-vehicle-insurance/blob/main/main/cloud_storage/aws_storage.py): model registry · Stores and retrieves model artifacts from S3 · `main/cloud_storage/aws_storage.py`

### User Interface

Request input and presentation surfaces

- [Vehicle Form](https://github.com/patilsri585/mlops-project-vehicle-insurance/blob/main/templates/vehicledata.html): html ui · Collects user input for online prediction · `templates/vehicledata.html`

## Connections

- demo.py → Training Pipeline: runs · Launches the offline training workflow
- app.py → Prediction Pipeline: serves · Uses the inference flow for request handling
- Training Pipeline → Ingestion: stage 1 · Starts data acquisition and split creation
- Training Pipeline → Validation: stage 2 · Validates raw and split data
- Training Pipeline → Transformation: stage 3 · Builds preprocessing artifacts and arrays
- Training Pipeline → Trainer: stage 4 · Trains the classifier from transformed data
- Training Pipeline → Evaluation: stage 5 · Checks model quality before promotion
- Training Pipeline → Pusher: stage 6 · Publishes the approved model to S3
- Ingestion → MongoDB: reads from · Connects to MongoDB Atlas source data
- Validation → schema.yaml: checks against · Uses schema constraints for validation
- Transformation → model.yaml: uses config · Applies preprocessing rules from model config
- Transformation → Entities: emits · Produces transformation artifact metadata
- Trainer → Entities: emits · Produces training artifact metadata
- Evaluation → model.yaml: thresholds · Reads acceptance criteria from configuration
- Pusher → AWS Setup: uses · Uses AWS service configuration
- Pusher → S3 Storage: uploads to · Pushes the trained model into S3 storage
- S3 Storage → Estimators: backs · Provides S3-backed model access
- Prediction Pipeline → Vehicle Form: receives · Accepts user-submitted vehicle data
- Prediction Pipeline → Transformation: mirrors · Reuses preprocessing behavior to avoid skew
- Prediction Pipeline → Estimators: loads · Loads the persisted model for inference
- Training Pipeline → Logging & Errors: logs · Records progress and failures
- Prediction Pipeline → Logging & Errors: logs · Records inference errors and tracebacks