How arshnoor-singh-sohi/food-segmentation-pipeline works
Project Type and Purpose
• This is a full-stack AI processing pipeline focused on food image analysis.
• It ingests raw food photos, applies object detection (YOLO), refines masks (SAM2), computes nutrition info via a local database, and outputs results (CSV, JSON, HTML, visualizations).
• An optional FastAPI server layer exposes endpoints for on-demand inference.
Main Components
a. Data Layer
• Input images (data/input/)
• Pretrained model weights (data/models/, weights/)
• Nutrition database JSON (data/nutrition_database.json)
b. Preprocessing
• src/preprocessing/food_preprocessor.py
• src/preprocessing/image_enhancer.py
c. Core Models
• YOLO detectors (src/models/yolo_detector.py)
• SAM2 predictor (src/models/sam2_predictor.py)
• Combined pipeline orchestrator (src/models/combined_pipeline.py)
• Fast-segmentation shortcuts (fast_yolo_segmentation.py, fast_segmentation.py)
d. Post-processing & Analytics
• Nutrition calculator (src/utils/nutrition_db.py)
• Visualization utilities (src/utils/visualization.py)
• File utilities for saving results (src/utils/file_utils.py)
e. Orchestration Scripts & CLI
• Single-image, batch, model-comparison scripts (scripts/*.py, enhanced_*_tester.py)
• Configuration loader (config/config.yaml, config/models.yaml)
f. API Layer
• FastAPI server (src/api/fastapi_server.py) exposing endpoints for single/batch processing
g. Output Store
• data/output/ and output_directory/ folders with timestamped subdirectories for each run
• Reports: HTML dashboards, CSV/JSON/Excel exports, saved mask overlays
Relationships & Data Flow
1. Input images → Preprocessor → Preprocessed images
3. YOLO outputs → SAM2 Predictor → Refined segmentation masks
4. Masks + detections → Nutrition DB → Calorie/nutrient estimates
5. Combined data → Visualization & Report generators → Persist to data/output or user-specified output_directory
6. Orchestration layer (scripts or FastAPI) sequences the above steps, reads config to select model variants, thresholds, output formats.
7. Model-comparison/test scripts iterate over multiple YOLO variants, compare metrics, generate HTML dashboards.
Architectural Patterns & Design Principles
• Pipeline pattern: each stage (ingest, preprocess, detect, segment, post-process) is a standalone module with clear input/output contracts.
• Modular design: models, preprocessing, utils, API, and scripts are separated into dedicated packages.
• Configuration-driven: YAML files in config/ define model paths and thresholds.
• Extensibility: new detectors/segmenters can be plugged into combined_pipeline by adhering to predictor interface.
• Testing harness: tests/ and standalone test_*.py ensure each module and end-to-end flow works.
Technologies & Frameworks
• Python 3.12
• PyTorch (YOLOv8/9/10, SAM2)
• FastAPI for serving
• Jinja2/Plotly/Matplotlib for HTML dashboards
• pandas/openpyxl for CSV/Excel exports
• PyYAML for configuration
• pytest for testing
• Jupyter notebooks for demos
Essential Details to Capture
• Model variants (YOLOv8/v9/v10) as subcomponents of the detector block.
• How config.yaml drives model selection thresholds.
• That the combined pipeline merges YOLO and SAM2 outputs.
• That batch vs single-image flows reuse the same modules with different drivers.
• Testing and model-comparison as parallel flows tapping into the detector and pipeline modules.