# codewithkaran-21/mlops-vehicle-insurance architecture

> Architecture diagram of the GitHub repository codewithkaran-21/mlops-vehicle-insurance, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/codewithkaran-21/mlops-vehicle-insurance
- Repository: https://github.com/codewithkaran-21/mlops-vehicle-insurance

## Overview

##### Project Type and Purpose
- This is a full-stack, end-to-end MLOps application for Vehicle Insurance Claim Prediction.
- It orchestrates data ingestion from MongoDB Atlas, ML pipeline steps (ingestion, validation, transformation, training, evaluation, pushing), model registry on AWS S3, and deployment via Docker on EC2 with CI/CD.

##### File Structure
- Top-level scripts: app.py (Flask API), demo.py, notebook folder (exploratory work).
- config/ model.yaml & schema.yaml define pipeline settings and data schemas.
- src/ contains:
- cloud_storage (AWS interactions)
- components (pipeline stages)
- configuration (connections to AWS & MongoDB)
- data_access (raw data fetchers)
- entity (data structures and artifacts)
- pipeline (orchestrators: training_pipeline, prediction_pipeline)
- utils, logger, exception for shared helpers and logging.
- Dockerfile, .github/workflows/aws.yaml, and setup files support containerization and CI/CD.

##### Insights from README
- Five logical layers: Data Layer, Pipeline Layer, Entity Layer, Utility & Logging Layer, Application Layer.
- Uses MongoDB Atlas, AWS S3, EC2/ECR, GitHub Actions, Docker, Flask.
- Emphasizes modular, scalable, production-grade MLOps design.

a. Main Components
- Data Layer: MongoDB Atlas cluster → Data Access Connector (`mongo_db_connection.py`)
- Pipeline Layer:
– Data Ingestion (json→csv, train/test split)
– Data Validation (schema checks)
– Data Transformation (encoding, scaling)
– Model Training & Selection
– Model Evaluation & Promotion
– Model Pusher (uploads to S3)
- Entity Layer: config/entity classes for artifacts and estimators
- Utility & Logging Layer: shared utilities, logger, exception
- Cloud Storage: AWS S3 bucket for model registry
- Application Layer: Flask service (`app.py`) exposing /train and /predict
- CI/CD & Deployment: GitHub Actions ➔ build Docker image ➔ push to ECR ➔ deploy to EC2

b. Relationships & Interactions
- Flask API invoking Prediction Pipeline, reading model from S3/local.

c. Architectural Patterns & Principles
- Layered architecture (Data, Pipeline, Entity, Utility, Application).
- Modular pipeline components with artifact hand-off.
- Separation of concerns: configuration, data access, business logic, infrastructure.
- Infrastructure as Code: Dockerfile and GitHub Actions workflow.

d. Key Technologies
- Python 3.10, Flask
- MongoDB Atlas
- Scikit-learn, Pandas, NumPy
- AWS S3, EC2, ECR, IAM
- Docker
- GitHub Actions CI/CD

##### Essential Elements
- Component labels matching code modules (e.g., src/components/data_validation).
- Artifact storage (local filesystem) between pipeline stages.
- External connections (environment variables for MongoDB URL, AWS credentials).

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Data Layer
    subgraph "Data Layer"
        direction TB
        MongoDB["MongoDB Atlas"]:::cloud
        DataConnector["MongoDB Connector"]:::dataLayer
        DataAccess["Data Access Module"]:::dataLayer
    end

    %% Pipeline Layer
    subgraph "Pipeline Layer"
        direction TB
        Ingestion["Data Ingestion"]:::pipeline
        Validation["Data Validation"]:::pipeline
        Transformation["Data Transformation"]:::pipeline
        Trainer["Model Trainer"]:::pipeline
        Evaluation["Model Evaluation"]:::pipeline
        Pusher["Model Pusher"]:::pipeline
    end

    %% Orchestrators
    subgraph "Orchestrators"
        direction TB
        TrainingPipeline["Training Pipeline"]:::pipeline
        PredictionPipeline["Prediction Pipeline"]:::pipeline
    end

    %% Entity Layer
    subgraph "Entity Layer"
        direction TB
        ArtifactEntity["Artifact Entity"]:::entity
        ConfigEntity["Config Entity"]:::entity
        Estimator["Estimator Interface"]:::entity
        S3Estimator["S3 Estimator Entity"]:::entity
    end

    %% Utility & Logging Layer
    subgraph "Utility & Logging Layer"
        direction TB
        Utils["Utilities"]:::utility
        Logger["Logger Module"]:::utility
        Exception["Exception Handling"]:::utility
    end

    %% Cloud Storage
    subgraph "Cloud Storage"
        direction TB
        S3Bucket["AWS S3 Bucket"]:::cloud
    end

    %% Configuration Files
    subgraph "Configuration Files"
        direction TB
        ModelYAML["model.yaml"]:::config
        SchemaYAML["schema.yaml"]:::config
    end

    %% Application Layer
    subgraph "Application Layer"
        direction TB
        FlaskAPI["Flask API (app.py)"]:::app
        TemplateEngine["HTML Template Renderer"]:::app
        HTMLTemplate["vehicledata.html"]:::app
        CSS["style.css"]:::app
    end

    %% CI/CD & Deployment
    subgraph "CI/CD & Deployment"
        direction TB
        GitHubActions["GitHub Actions Workflow"]:::infra
        DockerfileNode["Dockerfile"]:::infra
        ECR["AWS ECR"]:::cloud
        EC2["AWS EC2"]:::cloud
    end

    %% Demo & Exploration
    subgraph "Demo & Exploration"
        direction TB
        DemoScript["demo.py"]:::demo
        Notebook1["exp-notebook.ipynb"]:::demo
        Notebook2["mongoDB_demo.ipynb"]:::demo
    end

    %% Connections - Data Layer to Pipeline
    MongoDB -->|“raw JSON data”| DataConnector
    DataConnector -->|“connection”| DataAccess
    DataAccess -->|“fetched data”| Ingestion

    %% Pipeline flow
    Ingestion -->|"csv/train-test split"| Validation
    Validation -->|"validated data"| Transformation
    Transformation -->|"features"| Trainer
    Trainer -->|"trained model"| Evaluation
    Evaluation -->|"evaluation report"| Pusher
    Pusher -->|"model artifact"| S3Bucket

    %% Orchestrator relations
    TrainingPipeline -->|“runs stages”| Ingestion
    Evaluation -->|“notifies”| TrainingPipeline
    FlaskAPI -->|“invoke predict”| PredictionPipeline
    S3Bucket -->|“download model”| PredictionPipeline
    PredictionPipeline -->|“prediction result”| FlaskAPI

    %% Entity & Utility to Pipeline
    ArtifactEntity -->|“artifact definitions”| TrainingPipeline
    ConfigEntity -->|“config params”| TrainingPipeline
    Estimator -->|“model interfaces”| Trainer
    S3Estimator -->|“S3 model handler”| Pusher
    Utils -->|“helpers”| TrainingPipeline
    Logger -->|“logs”| TrainingPipeline
    Exception -->|“error handling”| TrainingPipeline

    %% Config files to orchestrator
    ModelYAML -->|“pipeline config”| TrainingPipeline
    SchemaYAML -->|“schema definitions”| Validation

    %% Application layer assets
    TemplateEngine -->|“renders”| HTMLTemplate
    TemplateEngine -->|“styles”| CSS
    FlaskAPI -->|“serves”| HTMLTemplate

    %% CI/CD workflow
    GitHubActions -->|“build”| DockerfileNode
    DockerfileNode -->|“image”| ECR
    ECR -->|“deploy”| EC2
    EC2 -->|“host service”| FlaskAPI

    %% Demo & Exploration to Data Layer
    Notebook1 -->|“offline analysis”| DataAccess
    Notebook2 -->|“DB demo”| DataConnector
    DemoScript -->|“run demo”| FlaskAPI

    %% Click Events
    click DataConnector "https://github.com/codewithkaran-21/mlops-vehicle-insurance/blob/main/src/configuration/mongo_db_connection.py"
    click DataAccess "https://github.com/codewithkaran-21/mlops-vehicle-insurance/blob/main/src/data_access/proj1_data.py"
    click Ingestion "https://github.com/codewithkaran-21/mlops-vehicle-insurance/blob/main/src/components/data_ingestion.py"
    click Validation "https://github.com/codewithkaran-21/mlops-vehicle-insurance/blob/main/src/components/data_validation.py"
    click Transformation "https://github.com/codewithkaran-21/mlops-vehicle-insurance/blob/main/src/components/data_tarnsformation.py"
    click Trainer "https://github.com/codewithkaran-21/mlops-vehicle-insurance/blob/main/src/components/model_trainer.py"
    click Evaluation "https://github.com/codewithkaran-21/mlops-vehicle-insurance/blob/main/src/components/model_evaluation.py"
    click Pusher "https://github.com/codewithkaran-21/mlops-vehicle-insurance/blob/main/src/components/model_pusher.py"
    click TrainingPipeline "https://github.com/codewithkaran-21/mlops-vehicle-insurance/blob/main/src/pipeline/training_pipeline.py"
    click PredictionPipeline "https://github.com/codewithkaran-21/mlops-vehicle-insurance/blob/main/src/pipeline/prediction_pipeline.py"
    click ArtifactEntity "https://github.com/codewithkaran-21/mlops-vehicle-insurance/blob/main/src/entity/artifact_entity.py"
    click ConfigEntity "https://github.com/codewithkaran-21/mlops-vehicle-insurance/blob/main/src/entity/config_entity.py"
    click Estimator "https://github.com/codewithkaran-21/mlops-vehicle-insurance/blob/main/src/entity/estimator.py"
    click S3Estimator "https://github.com/codewithkaran-21/mlops-vehicle-insurance/blob/main/src/entity/s3_estimator.py"
    click Utils "https://github.com/codewithkaran-21/mlops-vehicle-insurance/blob/main/src/utils/main_utils.py"
    click Logger "https://github.com/codewithkaran-21/mlops-vehicle-insurance/blob/main/src/logger/__init__.py"
    click Exception "https://github.com/codewithkaran-21/mlops-vehicle-insurance/blob/main/src/exception/__init__.py"
    click S3Bucket "https://github.com/codewithkaran-21/mlops-vehicle-insurance/blob/main/src/cloud_storage/aws_storage.py"
    click AWSConnection "https://github.com/codewithkaran-21/mlops-vehicle-insurance/blob/main/src/configuration/aws_connection.py"
    click ModelYAML "https://github.com/codewithkaran-21/mlops-vehicle-insurance/blob/main/config/model.yaml"
    click SchemaYAML "https://github.com/codewithkaran-21/mlops-vehicle-insurance/blob/main/config/schema.yaml"
    click FlaskAPI "https://github.com/codewithkaran-21/mlops-vehicle-insurance/blob/main/app.py"
    click TemplateEngine "https://github.com/codewithkaran-21/mlops-vehicle-insurance/blob/main/template.py"
    click HTMLTemplate "https://github.com/codewithkaran-21/mlops-vehicle-insurance/blob/main/templates/vehicledata.html"
    click CSS "https://github.com/codewithkaran-21/mlops-vehicle-insurance/blob/main/static/css/style.css"
    click DockerfileNode "https://github.com/codewithkaran-21/mlops-vehicle-insurance/tree/main/Dockerfile"
    click GitHubActions "https://github.com/codewithkaran-21/mlops-vehicle-insurance/blob/main/.github/workflows/aws.yaml"
    click DemoScript "https://github.com/codewithkaran-21/mlops-vehicle-insurance/blob/main/demo.py"
    click Notebook1 "https://github.com/codewithkaran-21/mlops-vehicle-insurance/blob/main/notebook/exp-notebook.ipynb"
    click Notebook2 "https://github.com/codewithkaran-21/mlops-vehicle-insurance/blob/main/notebook/mongoDB_demo.ipynb"

    %% Styles
    classDef dataLayer fill:#BBDEFB,stroke:#2196F3,stroke-width:1px;
    classDef pipeline fill:#C8E6C9,stroke:#388E3C,stroke-width:1px;
    classDef entity fill:#ECEFF1,stroke:#90A4AE,stroke-width:1px;
    classDef utility fill:#F5F5F5,stroke:#BDBDBD,stroke-width:1px;
    classDef app fill:#FFE0B2,stroke:#FB8C00,stroke-width:1px;
    classDef infra fill:#E1BEE7,stroke:#8E24AA,stroke-width:1px;
    classDef cloud fill:#B2EBF2,stroke:#006064,stroke-width:1px;
    classDef config fill:#FFF9C4,stroke:#FBC02D,stroke-width:1px;
    classDef demo fill:#E8EAF6,stroke:#3949AB,stroke-width:1px;
```
