How anand-lab-172/air-quality-prediction-with-mlflow works
Project Type & Purpose
• This is a machine learning project with an emphasis on MLOps practices. Its primary purpose is to build, evaluate, and manage multiple ML models for air quality prediction while using MLflow to orchestrate and monitor the experiments.
• The README highlights the steps: data loading/preprocessing, model training with various classifiers, metrics/confusion matrix logging, and MLflow-based model tracking and deployment.
Analyzing the File Structure
• Top-Level Components:
• README.md – Provides the project overview, steps, and instructions.
• air_quality_pred.py – Likely the main script that handles data loading, preprocessing (including applying SMOTE), training models, and integrating with MLflow.
• mlflow-testing.ipynb – Jupyter Notebook used for experimentation, which likely illustrates the EDA and model experiments.
• CSV Files (pollution_dataset.csv, predicted_air_quality.csv, model_results.csv) – The dataset, output predictions, and possibly result logs from model evaluations.
• schedule_mlflow.bat – A Windows batch file for scheduling (or automatically triggering) the MLflow processes, such as starting the MLflow server or job scheduling.
• Notice that, although the README refers to a script named "MLflow.py", the available file (“air_quality_pred.py”) likely serves the same purpose.
System Components
a. Data Layer:
• "pollution_dataset.csv" as the input data source.
• Preprocessing component (include operations like handling imbalances with SMOTE).
b. Model Training & Experimentation:
• The main training script (air_quality_pred.py) and the Jupyter notebook (mlflow-testing.ipynb) as points where the models are trained.
• Multiple classifiers (e.g., Logistic Regression, Decision Tree, Random Forest, etc.) as sub-components of the training module.
c. Evaluation & Metrics Logging:
• Metrics capture (accuracy, precision, recall, F1 score, log loss) and confusion matrix visualization components.
• "model_results.csv" for capturing/logging results.
d. MLflow Integration:
• MLflow Server/UI that tracks experiments, logs model versions, parameters, and metrics.
e. Deployment & Packaging:
• Notational components for model deployment (suggesting that packaging with conda or Docker and MLflow’s deployment features is available).
f. Scheduler:
• The automated task scheduler represented by the schedule_mlflow.bat file indicating how the MLflow server or experiment tracking jobs are triggered.
Relationships & Interactions
• From Preprocessing to “Model Training & Experimentation” (air_quality_pred.py & mlflow-testing.ipynb).
Architectural Patterns & Design Principles
• The architecture demonstrates a pipeline pattern common in ML systems:
• Input (data ingestion) → Processing (preprocessing) → Model training → Evaluation → MLflow logging/deployment.
• MLOps principles are applied by decoupling experiment tracking, model versioning, and deployment from the core model training code, making the system modular and auditable.