How krishnaik06/mlproject works
Project type and purpose
• This is an end-to-end machine learning web application built in Python.
• It provides both model training (offline) and prediction (online) capabilities, wrapped in a Flask app and intended for deployment via AWS Elastic Beanstalk.
File structure
• Top-level:
• app.py – Flask entry point, serves HTML templates and prediction endpoints.
• artifacts/ – stores raw & processed data, trained model, preprocessor, test/train CSVs.
• catboost_info/ – logs and metadata from CatBoost training.
• notebook/ – Jupyter notebooks for EDA and model training (reproducibility).
• requirements.txt, setup.py – dependency and packaging config.
• .ebextensions/ – AWS EB deployment config.
• src/ – core Python package.
• components/ – modules for data_ingestion, data_transformation, model_trainer.
• pipeline/ – orchestrators for train_pipeline and predict_pipeline.
• exception.py, logger.py, utils.py – support code for error handling and logging.
README
• Describes the project as an end-to-end ML workflow.
• Key operations: ingest → transform → train → save artifacts → predict.
Main components & interactions
a. Frontend (User):
• Browser or UI client
• HTML templates: home.html (input form), index.html (results)
b. API / Web Server:
• app.py (Flask)
• Routes:
• “/” – render input form
• “/predict” – accept features, invoke predict_pipeline
c. Prediction Pipeline (src/pipeline/predict_pipeline.py):
• Load preprocessor.pkl and model.pkl from artifacts
• Transform incoming data, produce prediction
d. Training Pipeline (src/pipeline/train_pipeline.py) – offline:
• DataIngestion → DataTransformation → ModelTrainer
• Save model.pkl, preprocessor.pkl, update catboost_info logs
e. Artifacts Store:
• Local disk folder “artifacts/” (could be S3 in a future microservices expansion)
f. Logging & Exception Handling:
• src/logger.py and src/exception.py capture runtime events, errors
g. Deployment:
• AWS Elastic Beanstalk
• .ebextensions/python.config configures Python runtime
Architectural patterns & design principles
• Modular “pipeline” pattern – clear separation of data ingestion, transformation, training, prediction.
• MVC-lite for web layer – templates (View), app.py (Controller), components & pipelines (Model).
• Artifact versioning – ML artifacts are serialized and stored for reproducibility.
• Single-process monolith suitable for small-scale deployment; can evolve into microservices (e.g. separate training service).
Detailed checklist for the principal software engineer
• Separate the system into at least these components: UI, API, ML‐processing pipelines, data/artifacts, and deployment infrastructure.
• Both training and inference workflows.
• Files/modules responsible for each step (e.g., src/components/model_trainer.py).
• External dependencies: GitHub Actions, AWS EB, Python packages from requirements.txt.