# ibraheem-al-hafith/open_food_mlops architecture

> Architecture diagram of the GitHub repository ibraheem-al-hafith/open_food_mlops, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ibraheem-al-hafith/open_food_mlops
- Repository: https://github.com/ibraheem-al-hafith/open_food_mlops

## Overview

Open Food MLOps classifies food products into NOVA groups from nutritional data. Its main workflow ingests Open Food Facts data, builds features, trains and evaluates candidate models, applies champion-selection gates, and registers the winner for FastAPI serving. Serving records predictions for later batch, drift, and performance monitoring, with metrics sent to Prometheus infrastructure. The graph preserves documented capabilities whose implementation was not sampled; edges without direct source evidence are omitted. Links cited to the README reflect its documented architecture, not independent inspection of those implementations.

Main entry points: [`src/open_food_mlops/experiments/orchestrator.py`](https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/src/open_food_mlops/experiments/orchestrator.py) (Experiment orchestration), [`serving/prediction_store.py`](https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/serving/prediction_store.py) (Prediction store), [`serving/app.py`](https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/serving/app.py) (FastAPI serving), [`src/open_food_mlops/data/data_ingestor.py`](https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/src/open_food_mlops/data/data_ingestor.py) (Data ingestion)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Data preparation"]
  node_ingestion["Data ingestion<br/>[data_ingestor.py]"]
  node_ingest_flow["Ingestion flow"]
  node_data_split["Dataset splitting<br/>[splitting.py]"]
end

subgraph group_training["Training and models"]
  node_feature_pipeline["Feature pipeline<br/>[base.py]"]
  node_feature_builder["Feature builder<br/>[builder.py]"]
  node_experiment["Experiment orchestration<br/>[orchestrator.py]"]
  node_model_contract["Model contract<br/>[base.py]"]
  node_model_registry["Model registry<br/>[registry.py]"]
  node_model_specs["Search-space specs<br/>[specs.py]"]
  node_tuning["Optuna tuning<br/>[optuna_tuner.py]"]
  node_evaluation["Model evaluation<br/>[evaluator.py]"]
  node_selection["Champion selection<br/>[selection.py]"]
  node_tracking["MLflow tracking<br/>[mlflow_tracker.py]"]
end

subgraph group_serving["Prediction serving"]
  node_api["FastAPI serving<br/>[app.py]"]
  node_api_schema["API schemas<br/>[schemas.py]"]
  node_metrics["Serving metrics<br/>[metrics.py]"]
  node_prediction_store[("Prediction store")]
  node_batch["Batch inference"]
end

subgraph group_monitoring["Monitoring"]
  node_reference["Reference data"]
  node_reference_predictions["Reference predictions"]
  node_drift_report["Feature drift reports"]
  node_prediction_drift["Prediction drift"]
  node_performance["Performance evaluation"]
  node_monitor_flow["Monitoring flow<br/>[monitoring_flow.py]"]
end

subgraph group_config["Shared configuration"]
  node_feature_schema["Feature schema<br/>[features.py]"]
  node_settings["Application settings<br/>[settings.py]"]
end

node_off(("Open Food Facts"))
node_cli["Training CLI<br/>[main.py]"]
node_mlflow[("MLflow registry")]
node_pushgateway["Pushgateway"]
node_prometheus["Prometheus and Grafana"]

node_off -->|"supplies data"| node_ingestion
node_ingest_flow -->|"runs ingestion"| node_ingestion
node_ingestion -->|"uses schema"| node_feature_schema
node_cli -->|"starts training"| node_experiment
node_experiment -->|"splits data"| node_data_split
node_experiment -->|"builds features"| node_feature_builder
node_feature_builder -->|"uses transformers"| node_feature_pipeline
node_experiment -->|"resolves models"| node_model_registry
node_experiment -->|"tunes candidates"| node_tuning
node_tuning -->|"uses search spaces"| node_model_specs
node_experiment -->|"evaluates candidates"| node_evaluation
node_experiment -->|"selects champion"| node_selection
node_experiment -->|"tracks runs"| node_tracking
node_tracking -->|"logs and registers"| node_mlflow
node_model_registry -->|"registers implementations"| node_model_contract
node_api -->|"loads production model"| node_mlflow
node_api -->|"validates requests"| node_api_schema
node_api -->|"records metrics"| node_metrics
node_api -->|"stores predictions"| node_prediction_store
node_batch -->|"loads champion"| node_mlflow
node_batch -->|"writes predictions"| node_prediction_store
node_reference_predictions -->|"runs predictions"| node_batch
node_reference -->|"uses ingested data"| node_ingestion
node_drift_report -->|"reads predictions"| node_prediction_store
node_prediction_drift -->|"reads predictions"| node_prediction_store
node_performance -->|"reads predictions"| node_prediction_store
node_monitor_flow -->|"runs report"| node_drift_report
node_monitor_flow -->|"runs drift check"| node_prediction_drift
node_monitor_flow -->|"runs evaluation"| node_performance
node_drift_report -->|"pushes drift metrics"| node_pushgateway
node_pushgateway -->|"scraped and visualized"| node_prometheus
node_settings -->|"configures tracking"| node_mlflow
node_settings -->|"configures endpoint"| node_pushgateway

click node_ingestion "https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/src/open_food_mlops/data/data_ingestor.py"
click node_ingest_flow "https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/pipelines/data_ingestion_flow.py"
click node_feature_schema "https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/src/open_food_mlops/config/features.py"
click node_feature_pipeline "https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/src/open_food_mlops/features/base.py"
click node_feature_builder "https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/src/open_food_mlops/features/builder.py"
click node_data_split "https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/src/open_food_mlops/data/splitting.py"
click node_cli "https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/main.py"
click node_experiment "https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/src/open_food_mlops/experiments/orchestrator.py"
click node_model_contract "https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/src/open_food_mlops/models/base.py"
click node_model_registry "https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/src/open_food_mlops/models/registry.py"
click node_model_specs "https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/src/open_food_mlops/models/specs.py"
click node_tuning "https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/src/open_food_mlops/models/tuning/optuna_tuner.py"
click node_evaluation "https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/src/open_food_mlops/evaluation/evaluator.py"
click node_selection "https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/src/open_food_mlops/experiments/selection.py"
click node_tracking "https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/src/open_food_mlops/tracking/mlflow_tracker.py"
click node_api "https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/serving/app.py"
click node_api_schema "https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/serving/schemas.py"
click node_metrics "https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/serving/metrics.py"
click node_prediction_store "https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/serving/prediction_store.py"
click node_batch "https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/pipelines/batch_inference_flow.py"
click node_reference "https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/pipelines/create_reference_data.py"
click node_reference_predictions "https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/pipelines/create_reference_predictions.py"
click node_drift_report "https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/monitoring/evidently_report.py"
click node_prediction_drift "https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/pipelines/prediction_drift.py"
click node_performance "https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/pipelines/evaluate_performance.py"
click node_monitor_flow "https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/pipelines/monitoring_flow.py"
click node_settings "https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/src/open_food_mlops/config/settings.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_ingestion,node_ingest_flow,node_data_split toneBlue
class node_feature_pipeline,node_feature_builder,node_experiment,node_model_contract,node_model_registry,node_model_specs,node_tuning,node_evaluation,node_selection,node_tracking,node_mlflow toneAmber
class node_api,node_api_schema,node_metrics,node_prediction_store,node_batch toneMint
class node_reference,node_reference_predictions,node_drift_report,node_prediction_drift,node_performance,node_monitor_flow toneRose
class node_feature_schema,node_settings,node_off,node_pushgateway,node_prometheus toneIndigo
class node_cli toneTeal
```

## Components

### Data preparation

- [Data ingestion](https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/src/open_food_mlops/data/data_ingestor.py): `src/open_food_mlops/data/data_ingestor.py`
- [Ingestion flow](https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/pipelines/data_ingestion_flow.py): `pipelines/data_ingestion_flow.py`
- [Dataset splitting](https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/src/open_food_mlops/data/splitting.py): `src/open_food_mlops/data/splitting.py`

### Training and models

- [Feature pipeline](https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/src/open_food_mlops/features/base.py): `src/open_food_mlops/features/base.py`
- [Feature builder](https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/src/open_food_mlops/features/builder.py): `src/open_food_mlops/features/builder.py`
- [Experiment orchestration](https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/src/open_food_mlops/experiments/orchestrator.py): `src/open_food_mlops/experiments/orchestrator.py`
- [Model contract](https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/src/open_food_mlops/models/base.py): `src/open_food_mlops/models/base.py`
- [Model registry](https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/src/open_food_mlops/models/registry.py): `src/open_food_mlops/models/registry.py`
- [Search-space specs](https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/src/open_food_mlops/models/specs.py): `src/open_food_mlops/models/specs.py`
- [Optuna tuning](https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/src/open_food_mlops/models/tuning/optuna_tuner.py): `src/open_food_mlops/models/tuning/optuna_tuner.py`
- [Model evaluation](https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/src/open_food_mlops/evaluation/evaluator.py): `src/open_food_mlops/evaluation/evaluator.py`
- [Champion selection](https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/src/open_food_mlops/experiments/selection.py): `src/open_food_mlops/experiments/selection.py`
- [MLflow tracking](https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/src/open_food_mlops/tracking/mlflow_tracker.py): `src/open_food_mlops/tracking/mlflow_tracker.py`

### Prediction serving

- [FastAPI serving](https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/serving/app.py): `serving/app.py`
- [API schemas](https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/serving/schemas.py): `serving/schemas.py`
- [Serving metrics](https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/serving/metrics.py): `serving/metrics.py`
- [Prediction store](https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/serving/prediction_store.py): `serving/prediction_store.py`
- [Batch inference](https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/pipelines/batch_inference_flow.py): `pipelines/batch_inference_flow.py`

### Monitoring

- [Reference data](https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/pipelines/create_reference_data.py): `pipelines/create_reference_data.py`
- [Reference predictions](https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/pipelines/create_reference_predictions.py): `pipelines/create_reference_predictions.py`
- [Feature drift reports](https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/monitoring/evidently_report.py): `monitoring/evidently_report.py`
- [Prediction drift](https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/pipelines/prediction_drift.py): `pipelines/prediction_drift.py`
- [Performance evaluation](https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/pipelines/evaluate_performance.py): `pipelines/evaluate_performance.py`
- [Monitoring flow](https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/pipelines/monitoring_flow.py): `pipelines/monitoring_flow.py`

### Shared configuration

- [Feature schema](https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/src/open_food_mlops/config/features.py): `src/open_food_mlops/config/features.py`
- [Application settings](https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/src/open_food_mlops/config/settings.py): `src/open_food_mlops/config/settings.py`

### Other components

- **Open Food Facts**
- [Training CLI](https://github.com/ibraheem-al-hafith/open_food_mlops/blob/main/main.py): `main.py`
- **MLflow registry**
- **Pushgateway**
- **Prometheus and Grafana**

## Connections

- Open Food Facts → Data ingestion: supplies data (evidence: `README.md`)
- Ingestion flow → Data ingestion: runs ingestion (evidence: `pipelines/data_ingestion_flow.py`)
- Data ingestion → Feature schema: uses schema (evidence: `src/open_food_mlops/data/data_ingestor.py`)
- Training CLI → Experiment orchestration: starts training (evidence: `main.py`)
- Experiment orchestration → Dataset splitting: splits data (evidence: `src/open_food_mlops/experiments/orchestrator.py`)
- Experiment orchestration → Feature builder: builds features (evidence: `src/open_food_mlops/experiments/orchestrator.py`)
- Feature builder → Feature pipeline: uses transformers (evidence: `src/open_food_mlops/features/builder.py`)
- Experiment orchestration → Model registry: resolves models (evidence: `src/open_food_mlops/experiments/orchestrator.py`)
- Experiment orchestration → Optuna tuning: tunes candidates (evidence: `src/open_food_mlops/experiments/orchestrator.py`)
- Optuna tuning → Search-space specs: uses search spaces (evidence: `src/open_food_mlops/models/tuning/optuna_tuner.py`)
- Experiment orchestration → Model evaluation: evaluates candidates (evidence: `src/open_food_mlops/experiments/orchestrator.py`)
- Experiment orchestration → Champion selection: selects champion (evidence: `src/open_food_mlops/experiments/orchestrator.py`)
- Experiment orchestration → MLflow tracking: tracks runs (evidence: `src/open_food_mlops/experiments/orchestrator.py`)
- MLflow tracking → MLflow registry: logs and registers (evidence: `README.md`)
- Model registry → Model contract: registers implementations (evidence: `src/open_food_mlops/models/registry.py`)
- FastAPI serving → MLflow registry: loads production model (evidence: `README.md`)
- FastAPI serving → API schemas: validates requests (evidence: `serving/app.py`)
- FastAPI serving → Serving metrics: records metrics (evidence: `serving/app.py`)
- FastAPI serving → Prediction store: stores predictions (evidence: `serving/app.py`)
- Batch inference → MLflow registry: loads champion (evidence: `pipelines/batch_inference_flow.py`)
- Batch inference → Prediction store: writes predictions (evidence: `README.md`)
- Reference predictions → Batch inference: runs predictions (evidence: `pipelines/create_reference_predictions.py`)
- Reference data → Data ingestion: uses ingested data (evidence: `pipelines/reference_flow.py`)
- Feature drift reports → Prediction store: reads predictions (evidence: `monitoring/evidently_report.py`)
- Prediction drift → Prediction store: reads predictions (evidence: `pipelines/prediction_drift.py`)
- Performance evaluation → Prediction store: reads predictions (evidence: `pipelines/evaluate_performance.py`)
- Monitoring flow → Feature drift reports: runs report (evidence: `pipelines/monitoring_flow.py`)
- Monitoring flow → Prediction drift: runs drift check (evidence: `pipelines/monitoring_flow.py`)
- Monitoring flow → Performance evaluation: runs evaluation (evidence: `pipelines/monitoring_flow.py`)
- Feature drift reports → Pushgateway: pushes drift metrics (evidence: `monitoring/evidently_report.py`)
- Pushgateway → Prometheus and Grafana: scraped and visualized (evidence: `README.md`)
- Application settings → MLflow registry: configures tracking (evidence: `src/open_food_mlops/config/settings.py`)
- Application settings → Pushgateway: configures endpoint (evidence: `src/open_food_mlops/config/settings.py`)
