How ibraheem-al-hafith/open_food_mlops works
Open Food MLOps classifies food products into NOVA groups from nutritional data. Its main workflow ingests Open Food Facts data, builds features, trains and evaluates candidate models, applies champion-selection gates, and registers the winner for FastAPI serving. Serving records predictions for later batch, drift, and performance monitoring, with metrics sent to Prometheus infrastructure. The graph preserves documented capabilities whose implementation was not sampled; edges without direct source evidence are omitted. Links cited to the README reflect its documented architecture, not independent inspection of those implementations.
Main entry points: orchestrator.py Experiment orchestration · prediction_store.py Prediction store · app.py FastAPI serving · data_ingestor.py Data ingestion
Components 31
Data preparation
- Data ingestion
src/open_food_mlops/data/data_ingestor.py - Ingestion flow
pipelines/data_ingestion_flow.py - Dataset splitting
src/open_food_mlops/data/splitting.py
Training and models
- Feature pipeline
src/open_food_mlops/features/base.py - Feature builder
src/open_food_mlops/features/builder.py - Experiment orchestration
src/open_food_mlops/experiments/orchestrator.py - Model contract
src/open_food_mlops/models/base.py - Model registry
src/open_food_mlops/models/registry.py - Search-space specs
src/open_food_mlops/models/specs.py - Optuna tuning
src/open_food_mlops/models/tuning/optuna_tuner.py - Model evaluation
src/open_food_mlops/evaluation/evaluator.py - Champion selection
src/open_food_mlops/experiments/selection.py - MLflow tracking
src/open_food_mlops/tracking/mlflow_tracker.py
Prediction serving
- FastAPI serving
serving/app.py - API schemas
serving/schemas.py - Serving metrics
serving/metrics.py - Prediction store
serving/prediction_store.py - Batch inference
pipelines/batch_inference_flow.py
Monitoring
- Reference data
pipelines/create_reference_data.py - Reference predictions
pipelines/create_reference_predictions.py - Feature drift reports
monitoring/evidently_report.py - Prediction drift
pipelines/prediction_drift.py - Performance evaluation
pipelines/evaluate_performance.py - Monitoring flow
pipelines/monitoring_flow.py
Shared configuration
- Feature schema
src/open_food_mlops/config/features.py - Application settings
src/open_food_mlops/config/settings.py
Other components
- Open Food Facts
- Training CLI
main.py - MLflow registry
- Pushgateway
- Prometheus and Grafana
Connections 33
- Open Food Facts → Data ingestion: supplies data
README.md - Ingestion flow → Data ingestion: runs ingestion
pipelines/data_ingestion_flow.py - Data ingestion → Feature schema: uses schema
src/open_food_mlops/data/data_ingestor.py - Training CLI → Experiment orchestration: starts training
main.py - Experiment orchestration → Dataset splitting: splits data
src/open_food_mlops/experiments/orchestrator.py - Experiment orchestration → Feature builder: builds features
src/open_food_mlops/experiments/orchestrator.py - Feature builder → Feature pipeline: uses transformers
src/open_food_mlops/features/builder.py - Experiment orchestration → Model registry: resolves models
src/open_food_mlops/experiments/orchestrator.py - Experiment orchestration → Optuna tuning: tunes candidates
src/open_food_mlops/experiments/orchestrator.py - Optuna tuning → Search-space specs: uses search spaces
src/open_food_mlops/models/tuning/optuna_tuner.py - Experiment orchestration → Model evaluation: evaluates candidates
src/open_food_mlops/experiments/orchestrator.py - Experiment orchestration → Champion selection: selects champion
src/open_food_mlops/experiments/orchestrator.py - Experiment orchestration → MLflow tracking: tracks runs
src/open_food_mlops/experiments/orchestrator.py - MLflow tracking → MLflow registry: logs and registers
README.md - Model registry → Model contract: registers implementations
src/open_food_mlops/models/registry.py - FastAPI serving → MLflow registry: loads production model
README.md - FastAPI serving → API schemas: validates requests
serving/app.py - FastAPI serving → Serving metrics: records metrics
serving/app.py - FastAPI serving → Prediction store: stores predictions
serving/app.py - Batch inference → MLflow registry: loads champion
pipelines/batch_inference_flow.py - Batch inference → Prediction store: writes predictions
README.md - Reference predictions → Batch inference: runs predictions
pipelines/create_reference_predictions.py - Reference data → Data ingestion: uses ingested data
pipelines/reference_flow.py - Feature drift reports → Prediction store: reads predictions
monitoring/evidently_report.py - Prediction drift → Prediction store: reads predictions
pipelines/prediction_drift.py - Performance evaluation → Prediction store: reads predictions
pipelines/evaluate_performance.py - Monitoring flow → Feature drift reports: runs report
pipelines/monitoring_flow.py - Monitoring flow → Prediction drift: runs drift check
pipelines/monitoring_flow.py - Monitoring flow → Performance evaluation: runs evaluation
pipelines/monitoring_flow.py - Feature drift reports → Pushgateway: pushes drift metrics
monitoring/evidently_report.py - Pushgateway → Prometheus and Grafana: scraped and visualized
README.md - Application settings → MLflow registry: configures tracking
src/open_food_mlops/config/settings.py - Application settings → Pushgateway: configures endpoint
src/open_food_mlops/config/settings.py