patilsri585/mlops-project-vehicle-insurance

1. This repository is an end-to-end MLOps project for vehicle insurance response prediction. It has two primary execution paths: an offline training pipeline that builds and publishes a model, and an online inference path that serves predictions through a FastAPI app.

2. The main runtime entry points are demo.py for running the full training workflow and app.py for serving predictions. The app layer is lightweight; most business logic lives under main/, which is organized as a reusable Python package.

3. The training pipeline is staged and linear: MongoDB ingestion, schema/data validation, data transformation, model training, model evaluation, and model push. Each stage emits a persisted artifact into a timestamped artifact/ folder, so outputs from one run are isolated from the next.

4. Data ingress starts in main/configuration/mongo_db_connection.py and main/data_access/proj1_data.py, which connect to MongoDB Atlas and extract the source dataset. The ingestion component writes raw data to a CSV feature store and splits it into train/test files before downstream processing.

5. Validation and transformation are separate boundaries. data_validation.py checks incoming data against config/schema.yaml, while data_transformation.py performs preprocessing such as encoding, scaling, and class-imbalance handling. The fitted preprocessing pipeline is serialized as preprocessing.pkl, and transformed datasets are stored as .npy arrays.

6. Model training is isolated in model_trainer.py, which consumes the transformed training arrays and creates the classifier artifact model.pkl. model_evaluation.py then compares the trained model against test data and acceptance criteria defined in configuration. This makes model promotion dependent on measured quality, not just successful training.

7. Model publishing is handled by model_pusher.py and the AWS helpers in main/configuration/aws_connection.py plus main/cloud_storage/aws_storage.py. The trained model artifact is uploaded to S3, which acts as the model registry / distribution point for deployment and inference.

8. The inference pipeline in main/pipeline/prediction_pipeline.py mirrors the training preprocessing logic to avoid training-serving skew. It accepts user input from the HTML form in templates/vehicledata.html, applies custom type conversions and feature alignment, loads the preprocessor and model, and returns a binary prediction.

9. Configuration is externalized in config/model.yaml and config/schema.yaml. These files define model settings, feature lists, scaling rules, and validation constraints, so the pipeline behavior is controlled by config rather than hardcoded assumptions.

10. Shared domain objects live in main/entity/. These dataclasses define configuration and artifact shapes, while estimator.py and s3_estimator.py wrap model loading and S3-backed access. This keeps orchestration code from depending directly on low-level storage details.

11. Operational concerns are built into the package: main/logger/ provides rotating file logging, and main/exception/ standardizes error reporting with traceback context. Generated logs are stored under logs/, while notebooks under notebook/ support exploration and offline experimentation rather than production execution.

12. Deployment is container-based and cloud-backed. The dockerfile packages the app, and .github/workflows/aws.yaml implements CI/CD that builds the image, pushes it to AWS ECR, then deploys it to an EC2 self-hosted runner. Environment secrets include AWS credentials and the MongoDB connection string, making cloud services part of the runtime boundary.