How harshitraiii2005/vehichle-insurance-prediction works
Project Type and Purpose
• This is an MLOps pipeline for building, evaluating, and serving a vehicle‐insurance prediction model.
• It combines data engineering (MongoDB ingestion, validation, transformation), model training/evaluation, cloud storage (AWS S3), a FastAPI prediction service (app.py + templates/static), and CI/CD (Docker + GitHub Actions + AWS EC2/ECR).
File Structure
• Top-level: Dockerfile, GitHub Actions workflow (.github/workflows/aws.yaml), config/schema.yaml, app.py (FastAPI service), demo.py (quick smoke test), notebooks for EDA.
• src/cloud_storage: AWS S3 logic
• src/components: pipeline steps (data_ingestion → validation → transformation → trainer → evaluator → pusher)
• src/configuration: connection objects (MongoDB, AWS)
• src/data_access: raw data fetch (vehicle_data.py)
• src/entity: configuration and artifact definitions
• src/pipline: orchestrates training and prediction pipelines
• src/utils, src/logger, src/exception: cross-cutting utilities
• templates/ + static/: UI assets for the prediction web interface
README Insights
• 18‐step workflow: from environment setup → MongoDB data push → ETL steps → model training/evaluation → S3 model push → FastAPI app → Dockerize → CI/CD to EC2/ECR.
• Emphasis on modular “components” for each ML stage and on cloud integration.
Main Components & Interactions
a. Data Sources
• CSV files (notebooks/data.csv, data.csv)
• Jupyter notebooks (for EDA)
b. Data Storage
• MongoDB Atlas cluster
c. ETL Pipeline
• Data Ingestion (src/components/data_ingestion.py) reads from MongoDB via src/data_access/vehicle_data.py
• Data Validation (src/components/data_validation.py; schema in config/schema.yaml)
• Data Transformation (src/components/data_transformation.py; uses entity/estimator.py)
d. Model Pipeline
• Model Trainer (src/components/model_trainer.py)
• Model Evaluation (src/components/model_evaluation.py)
• Model Pusher (src/components/model_pusher.py) → AWS S3 (src/cloud_storage/aws_storage.py)
e. Serving Layer
• FastAPI app (app.py + templates.py) loads best model from S3 and serves predictions via templates/vehicledata.html + static/css/style.css
f. CI/CD & Infrastructure
• Dockerfile builds container image
• GitHub Actions (aws.yaml) runs tests, builds image, pushes to ECR
• EC2 instance (self-hosted runner) pulls image and runs container on port 5080
Architectural Patterns & Design Principles
• Modular ETL & model‐ops “components” connected by clear interfaces (entity definitions, configuration objects)
• Layered separation: ingestion → processing → training → deployment
• Infrastructure as code (Docker + GitHub Actions + AWS resources)
• Single responsibility modules for logging, exception handling, configuration