# arshnoor-singh-sohi/food-segmentation-pipeline architecture

> Architecture diagram of the GitHub repository arshnoor-singh-sohi/food-segmentation-pipeline, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/arshnoor-singh-sohi/food-segmentation-pipeline
- Repository: https://github.com/arshnoor-singh-sohi/food-segmentation-pipeline

## Overview

##### Project Type and Purpose
- This is a full-stack AI processing pipeline focused on food image analysis.
- It ingests raw food photos, applies object detection (YOLO), refines masks (SAM2), computes nutrition info via a local database, and outputs results (CSV, JSON, HTML, visualizations).
- An optional FastAPI server layer exposes endpoints for on-demand inference.

##### Main Components
a. Data Layer
- Input images (data/input/)
- Pretrained model weights (data/models/, weights/)
- Nutrition database JSON (data/nutrition_database.json)
b. Preprocessing
- src/preprocessing/food_preprocessor.py
- src/preprocessing/image_enhancer.py
c. Core Models
- YOLO detectors (src/models/yolo_detector.py)
- SAM2 predictor (src/models/sam2_predictor.py)
- Combined pipeline orchestrator (src/models/combined_pipeline.py)
- Fast-segmentation shortcuts (fast_yolo_segmentation.py, fast_segmentation.py)
d. Post-processing & Analytics
- Nutrition calculator (src/utils/nutrition_db.py)
- Visualization utilities (src/utils/visualization.py)
- File utilities for saving results (src/utils/file_utils.py)
e. Orchestration Scripts & CLI
- Single-image, batch, model-comparison scripts (scripts/*.py, enhanced_*_tester.py)
- Configuration loader (config/config.yaml, config/models.yaml)
f. API Layer
- FastAPI server (src/api/fastapi_server.py) exposing endpoints for single/batch processing
g. Output Store
- data/output/ and output_directory/ folders with timestamped subdirectories for each run
- Reports: HTML dashboards, CSV/JSON/Excel exports, saved mask overlays

##### Relationships & Data Flow
1. Input images → Preprocessor → Preprocessed images
3. YOLO outputs → SAM2 Predictor → Refined segmentation masks
4. Masks + detections → Nutrition DB → Calorie/nutrient estimates
5. Combined data → Visualization & Report generators → Persist to data/output or user-specified output_directory
6. Orchestration layer (scripts or FastAPI) sequences the above steps, reads config to select model variants, thresholds, output formats.
7. Model-comparison/test scripts iterate over multiple YOLO variants, compare metrics, generate HTML dashboards.

##### Architectural Patterns & Design Principles
- Pipeline pattern: each stage (ingest, preprocess, detect, segment, post-process) is a standalone module with clear input/output contracts.
- Modular design: models, preprocessing, utils, API, and scripts are separated into dedicated packages.
- Configuration-driven: YAML files in config/ define model paths and thresholds.
- Extensibility: new detectors/segmenters can be plugged into combined_pipeline by adhering to predictor interface.
- Testing harness: tests/ and standalone test_*.py ensure each module and end-to-end flow works.

##### Technologies & Frameworks
- Python 3.12
- PyTorch (YOLOv8/9/10, SAM2)
- FastAPI for serving
- Jinja2/Plotly/Matplotlib for HTML dashboards
- pandas/openpyxl for CSV/Excel exports
- PyYAML for configuration
- pytest for testing
- Jupyter notebooks for demos

##### Essential Details to Capture
- Model variants (YOLOv8/v9/v10) as subcomponents of the detector block.
- How config.yaml drives model selection thresholds.
- That the combined pipeline merges YOLO and SAM2 outputs.
- That batch vs single-image flows reuse the same modules with different drivers.
- Testing and model-comparison as parallel flows tapping into the detector and pipeline modules.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% External Interfaces
    CLI["CLI / Notebook"]:::external

    %% Orchestration Layer
    subgraph "Orchestration Layer"
        direction TB
        SingleImage["Process Single Image"]:::orchestration
        Batch["Process Batch"]:::orchestration
        TrainYOLO["Train YOLO"]:::orchestration
        EnhancedSingle["Enhanced Single Tester"]:::orchestration
        EnhancedBatch["Enhanced Batch Tester"]:::orchestration
        ModelComp["Model Comparison"]:::orchestration
        FastTest["Fast Test Entry"]:::orchestration
        ViewResults["View Results"]:::orchestration
        FastAPI["FastAPI Server"]:::orchestration
        ConfigMain["config/config.yaml"]:::orchestration
        ConfigModels["config/models.yaml"]:::orchestration
    end

    %% Core Pipeline
    subgraph "Core Pipeline"
        direction TB
        FoodPreproc["Food Preprocessor"]:::processing
        ImageEnhancer["Image Enhancer"]:::processing
        YOLODet["YOLO Detector\n(v8/v9/v10)"]:::processing
        SAM2Pred["SAM2 Predictor"]:::processing
        CombinedPipe["Combined Pipeline"]:::processing
        FastYolo["Fast YOLO Segmentation"]:::processing
        FastSeg["Fast Segmentation"]:::processing
        NutritionCalc["Nutrition Calculator"]:::processing
        Visualization["Visualization Utilities"]:::processing
        FileUtils["File Utilities"]:::processing
    end

    %% Data Stores
    subgraph "Data Stores"
        direction TB
        InputImages(("Input Images")):::datastore
        DataModels(("Model Weights")):::datastore
        NutritionDB(("Nutrition DB")):::datastore
        OutputDefault(("Default Output")):::datastore
        OutputCustom(("Custom Output Directory")):::datastore
    end

    %% External to Orchestration
    CLI --> SingleImage
    CLI --> Batch
    CLI --> TrainYOLO
    CLI --> EnhancedSingle
    CLI --> EnhancedBatch
    CLI --> ModelComp
    CLI --> FastTest
    CLI --> ViewResults
    CLI --> FastAPI

    %% Configuration connections
    ConfigMain --> SingleImage
    ConfigMain --> Batch
    ConfigMain --> FastAPI
    ConfigModels --> YOLODet

    %% Orchestration to Core Pipeline
    SingleImage --> FoodPreproc
    Batch --> FoodPreproc
    FastAPI --> FoodPreproc
    ModelComp --> YOLODet

    %% Data flow in Core Pipeline
    InputImages --> FoodPreproc --> ImageEnhancer --> YOLODet
    DataModels --> YOLODet
    YOLODet --> SAM2Pred
    DataModels --> SAM2Pred
    SAM2Pred --> CombinedPipe
    CombinedPipe --> NutritionCalc
    NutritionDB --> NutritionCalc
    NutritionCalc --> Visualization --> FileUtils --> OutputDefault
    FileUtils --> OutputCustom

    %% Fast shortcuts branch
    YOLODet --> FastYolo --> CombinedPipe
    YOLODet --> FastSeg --> CombinedPipe

    %% Click Events
    click InputImages "https://github.com/arshnoor-singh-sohi/food-segmentation-pipeline/tree/main/data/input/"
    click DataModels "https://github.com/arshnoor-singh-sohi/food-segmentation-pipeline/tree/main/data/models/"
    click NutritionDB "https://github.com/arshnoor-singh-sohi/food-segmentation-pipeline/blob/main/data/nutrition_database.json"
    click FoodPreproc "https://github.com/arshnoor-singh-sohi/food-segmentation-pipeline/blob/main/src/preprocessing/food_preprocessor.py"
    click ImageEnhancer "https://github.com/arshnoor-singh-sohi/food-segmentation-pipeline/blob/main/src/preprocessing/image_enhancer.py"
    click YOLODet "https://github.com/arshnoor-singh-sohi/food-segmentation-pipeline/blob/main/src/models/yolo_detector.py"
    click SAM2Pred "https://github.com/arshnoor-singh-sohi/food-segmentation-pipeline/blob/main/src/models/sam2_predictor.py"
    click CombinedPipe "https://github.com/arshnoor-singh-sohi/food-segmentation-pipeline/blob/main/src/models/combined_pipeline.py"
    click FastYolo "https://github.com/arshnoor-singh-sohi/food-segmentation-pipeline/blob/main/src/models/fast_yolo_segmentation.py"
    click FastSeg "https://github.com/arshnoor-singh-sohi/food-segmentation-pipeline/blob/main/src/models/fast_segmentation.py"
    click NutritionCalc "https://github.com/arshnoor-singh-sohi/food-segmentation-pipeline/blob/main/src/utils/nutrition_db.py"
    click Visualization "https://github.com/arshnoor-singh-sohi/food-segmentation-pipeline/blob/main/src/utils/visualization.py"
    click FileUtils "https://github.com/arshnoor-singh-sohi/food-segmentation-pipeline/blob/main/src/utils/file_utils.py"
    click SingleImage "https://github.com/arshnoor-singh-sohi/food-segmentation-pipeline/blob/main/scripts/process_single_image.py"
    click Batch "https://github.com/arshnoor-singh-sohi/food-segmentation-pipeline/blob/main/scripts/process_batch.py"
    click TrainYOLO "https://github.com/arshnoor-singh-sohi/food-segmentation-pipeline/blob/main/scripts/train_yolo_food.py"
    click EnhancedSingle "https://github.com/arshnoor-singh-sohi/food-segmentation-pipeline/blob/main/enhanced_single_image_tester.py"
    click EnhancedBatch "https://github.com/arshnoor-singh-sohi/food-segmentation-pipeline/blob/main/enhanced_batch_tester.py"
    click ModelComp "https://github.com/arshnoor-singh-sohi/food-segmentation-pipeline/blob/main/model_comparison_enhanced.py"
    click FastTest "https://github.com/arshnoor-singh-sohi/food-segmentation-pipeline/blob/main/fast_test.py"
    click ViewResults "https://github.com/arshnoor-singh-sohi/food-segmentation-pipeline/blob/main/view_results.py"
    click ConfigMain "https://github.com/arshnoor-singh-sohi/food-segmentation-pipeline/blob/main/config/config.yaml"
    click ConfigModels "https://github.com/arshnoor-singh-sohi/food-segmentation-pipeline/blob/main/config/models.yaml"
    click FastAPI "https://github.com/arshnoor-singh-sohi/food-segmentation-pipeline/blob/main/src/api/fastapi_server.py"

    %% Styles
    classDef datastore fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef processing fill:#90EE90,stroke:#333,stroke-width:1px
    classDef orchestration fill:#FFA500,stroke:#333,stroke-width:1px
    classDef external fill:#DDA0DD,stroke:#333,stroke-width:1px
```
