How techiescamp/mlops-for-devops works
This repository is a hands-on MLOps learning project, not one unified runtime: it demonstrates local employee-attrition modeling and serving, then enterprise alternatives for data versioning, feature management, orchestration, training, experiment tracking, deployment, and monitoring. The principal user path is browser input through the Flask frontend to the FastAPI prediction service and back. The README documents broader enterprise capabilities, but several implementations were not sampled, so their internal wiring is intentionally omitted.
Main entry points: app.py Prediction frontend · dataset-pipeline.py DVC data pipeline · pipeline.py Kubeflow pipeline · employee_attrition.csv Attrition dataset
Components 27
Local ML workflow
- Attrition dataset
phase-1-local-dev/datasets/employee_attrition.csv - Data ingestion
phase-1-local-dev/src/data_preparation/01_ingestion.py - Data validation
phase-1-local-dev/src/data_preparation/02_validation.py - Data exploration
phase-1-local-dev/src/data_preparation/03_eda.py - Data cleaning
phase-1-local-dev/src/data_preparation/04_cleaning.py - Feature engineering
phase-1-local-dev/src/data_preparation/05_feature_engg.py - Preprocessing
phase-1-local-dev/src/data_preparation/06_preprocessing.py - Model training
phase-1-local-dev/src/model_training/01_training.py - Model tuning
phase-1-local-dev/src/model_training/04_tuning.py
Prediction serving
- Prediction frontend
phase-1-local-dev/frontend/app.py - Inference API
phase-1-local-dev/inference/src/app.py - Model predictor
phase-1-local-dev/inference/src/predictor.py - Serialized model
phase-1-local-dev/inference/artifacts/model.pkl
Enterprise workflows
- DVC data pipeline
phase-2-enterprise-setup/airflow/dags/dataset-pipeline.py - Feature conversion
phase-2-enterprise-setup/feast/csv_to_parquet.py - Feature definitions
phase-2-enterprise-setup/feast/feature_definitions.py - Kubeflow pipeline
phase-2-enterprise-setup/kubeflow-pipeline/pipeline.py - Pipeline components
phase-2-enterprise-setup/kubeflow-pipeline/components - Distributed training
phase-2-enterprise-setup/kubeflow-trainer/sdk_train.py - Tracked training
phase-2-enterprise-setup/mlflow/train_and_log_model.py
Platform operations
- Model monitoring
phase-2-enterprise-setup/evidently-ai/model-monitoring.py
Other components
- Prediction user
- Web browser
- DVC remote storage
- Git remote
- MLflow tracking server
- Evidently workspace
Connections 13
- Attrition dataset → Data ingestion: reads CSV
- Web browser → Prediction frontend: submits features
- Prediction frontend → Inference API: posts prediction
- Inference API → Model predictor: calls predictor
- Model predictor → Serialized model: loads model
- Prediction frontend → Web browser: returns result
- DVC data pipeline → DVC remote storage: pushes data
- DVC data pipeline → Git remote: commits version
- Feature conversion → Attrition dataset: reads CSV
- Kubeflow pipeline → Pipeline components: invokes stages
- Kubeflow pipeline → MLflow tracking server: none
- Tracked training → MLflow tracking server: logs run
- Model monitoring → Evidently workspace: pushes snapshots