# saad1912/vehicle_insurance_project architecture

> Architecture diagram of the GitHub repository saad1912/vehicle_insurance_project, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/saad1912/vehicle_insurance_project
- Repository: https://github.com/saad1912/vehicle_insurance_project

## Overview

##### Project Type & Purpose
- The project is an end-to-end machine learning pipeline focused on vehicle insurance data. It covers data ingestion from MongoDB Atlas, data validation & transformation, model training/evaluation, AWS integration for model management (S3), and even a prediction web interface. Additionally, there’s comprehensive CI/CD automation using Docker and GitHub Actions.
- Key technologies include Python, Docker, MongoDB Atlas, AWS (S3, IAM and possibly other services), and GitHub Actions.

##### Analyzing the File Structure
- Top-level files such as app.py, Dockerfile, README.md, and setup.py indicate an application entry point and containerization.
- The “src” directory is organized into subdirectories that clearly segment the architecture:
- “cloud_storage” for AWS interactions,
- “components” for pipeline steps (e.g., data_ingestion, data_transformation, model_trainer, model_evaluation, model_pusher),
- “configuration” for defining cloud and database connections,
- “data_access” for handling database connectivity (MongoDB),
- “entity” for classes representing configurations, artifacts, and model descriptors,
- “pipeline” for orchestrating the prediction and training pipelines,
- “utils” for miscellaneous helper functions.
- There are separate directories for static files and HTML templates, indicating the existence of a frontend part for the prediction pipeline.

##### Key Components
a. Data Engineering & Storage:
- Data Access Layer: Component (src/data_access) that connects the application to MongoDB.

b. Machine Learning Pipeline:
- Data Ingestion Component: Responsible for fetching data from MongoDB into the pipeline.
- Data Validation & Transformation Components: Handle schema validation (based on config/schema.yaml) and transforming data before training.
- Model Trainer: Uses the transformed data to create a machine learning model.
- Model Evaluation: Evaluates performance against a threshold to check if the new model supersedes the current one.
- Model Pusher: Uploads the approved, trained model to storage – specifically interacting with AWS S3.

c. AWS Integration:
- AWS S3 Storage: Used for model registry and management.
- AWS Connection/Configuration: Modules that manage AWS credentials and S3 bucket interactions (e.g., in src/cloud_storage/aws_storage.py and entity/s3_estimator.py).

d. Prediction Pipeline & Frontend:
- Web Application (app.py): Acts as an entry point for prediction requests.
- Static Assets & Templates: “static” folder for CSS and “templates” folder for HTML views.

e. DevOps & CI/CD:
- Docker: Containerization elements include Dockerfile and .dockerignore to package the application.
- GitHub Actions: CI/CD pipelines (under .github/workflows) automatically build, test, and deploy the application.
- EC2 Deployment & GitHub Runner: Deployment components for the live environment.

f. Utilities & Support:
- Logging & Exception Handling: Custom logger modules and exception processing (in src/logger and src/exception).
- Configuration Files: YAML files and configuration modules that centralize settings for the pipeline, DB, and AWS.

##### Relationships & Data Flow
- Data then flows from Ingestion to Validation and Transformation.
- From there, the pipeline splits into training and evaluation stages.
- After evaluation, a successful model is pushed to AWS S3 via the Model Pusher.
- On the front end, the Prediction Pipeline (app.py) retrieves models and handles user requests for predictions.
- An outer loop for CI/CD shows automated build and deployment (Docker + GitHub Actions) that touches the entire pipeline.
- Additionally, auxiliary elements like logging and exception handling loosely tie into every component for error monitoring.

##### Architectural Patterns & Design Principles
- Layered/Microservices-Like Decomposition: The project is modularized heavily into pipeline stages.
- Separation of Concerns: Each directory (components, configuration, entity, etc.) encapsulates a specific responsibility.
- Cloud-Native & Containerized: With Docker and AWS integration, the system design supports scalable and deployable microservices.
- CI/CD Integration: Ensures automated testing and deployment practices are integrated into the architecture.

## Diagram (Mermaid)

```mermaid
graph TD
    %% Data Engineering
    subgraph "Data Engineering"
        A["MongoDB Atlas"]:::database
        B["Data Access Layer"]:::backend
        C["Data Ingestion Component"]:::backend
    end

    %% Machine Learning Pipeline
    subgraph "Machine Learning Pipeline"
        D["Data Validation Component"]:::backend
        E["Data Transformation Component"]:::backend
        F["Model Trainer"]:::backend
        G["Model Evaluation"]:::backend
        H["Model Pusher"]:::backend
    end

    %% Cloud Integration
    subgraph "Cloud Integration"
        J["AWS Connection & Configuration"]:::cloud
        I["AWS S3 Storage"]:::cloud
    end

    %% Prediction & Frontend
    subgraph "Prediction & Frontend"
        K["Prediction Web Interface"]:::frontend
        L["Static Assets & Templates"]:::frontend
    end

    %% DevOps & CI/CD
    subgraph "DevOps & CI/CD"
        M["CI/CD Pipeline"]:::devops
    end

    %% Orchestration & Utilities
    subgraph "Orchestration & Utilities"
        N["Pipeline Orchestration"]:::backend
        O["Logging & Exception Handling"]:::utility
        P["Utility Functions"]:::utility
        Q["Configuration Files"]:::backend
    end

    %% Connections
    A --> B
    B --> C
    C --> D
    D --> E
    E --> F
    F --> G
    G --> H
    H --> I
    J --> I
    I --> K
    K --> L
    Q --> D
    M --> N
    N --> C
    N --> G
    N --> K
    O -.-> D
    O -.-> F
    P -.-> F

    %% Click Events
    click B "https://github.com/saad1912/vehicle_insurance_project/blob/main/src/data_access/proj1.py"
    click C "https://github.com/saad1912/vehicle_insurance_project/blob/main/src/components/data_ingestion.py"
    click D "https://github.com/saad1912/vehicle_insurance_project/blob/main/src/components/data_validation.py"
    click E "https://github.com/saad1912/vehicle_insurance_project/blob/main/src/components/data_transformation.py"
    click F "https://github.com/saad1912/vehicle_insurance_project/blob/main/src/components/model_trainer.py"
    click G "https://github.com/saad1912/vehicle_insurance_project/blob/main/src/components/model_evaluation.py"
    click H "https://github.com/saad1912/vehicle_insurance_project/blob/main/src/components/model_pusher.py"
    click J "https://github.com/saad1912/vehicle_insurance_project/blob/main/src/configuration/aws_connection.py"
    click K "https://github.com/saad1912/vehicle_insurance_project/blob/main/app.py"
    click L "https://github.com/saad1912/vehicle_insurance_project/blob/main/templates/vehicledata.html"
    click M "https://github.com/saad1912/vehicle_insurance_project/tree/main/Dockerfile"
    click N "https://github.com/saad1912/vehicle_insurance_project/tree/main/src/pipeline"
    click O "https://github.com/saad1912/vehicle_insurance_project/blob/main/src/logger/__init__.py"
    click P "https://github.com/saad1912/vehicle_insurance_project/blob/main/src/utils/main_utils.py"
    click Q "https://github.com/saad1912/vehicle_insurance_project/blob/main/config/model.yaml"

    %% Styles
    classDef database fill:#fce94f,stroke:#555,stroke-width:2px,color:#000;
    classDef backend fill:#8ae234,stroke:#555,stroke-width:2px,color:#000;
    classDef frontend fill:#729fcf,stroke:#555,stroke-width:2px,color:#fff;
    classDef devops fill:#fcaf3e,stroke:#555,stroke-width:2px,color:#000;
    classDef cloud fill:#ad7fa8,stroke:#555,stroke-width:2px,color:#fff;
    classDef utility fill:#e9b96e,stroke:#555,stroke-width:2px,color:#000;
```
