How lannguyen0910/food-recognition works
Project type and purpose
• This is a full-stack web application for food image analysis: it lets users upload or supply a URL/webcam feed, performs object detection (food items), optionally classification or segmentation, then fetches nutrition data from Edamam and renders annotated images plus nutrition charts in the browser.
• It uses Flask as the web server, a custom “Theseus” Python template for ML pipelines (detection, classification, segmentation), static HTML/CSS/JS for the frontend, and the Edamam external API for nutrition info.
File structure
• Top-level:
• app.py – Flask application entrypoint
• backend/ – wrappers for Edamam API (api.py, secret.py), routes.py, utils, constants
• modules.py – orchestration of inference and post-processing
• configs/ – YAML configs for each ML task (detection, classification, segmentation)
• static/ – CSS, JS (Chart.js), assets (uploads, detections), CSV & metadata files
• templates/ – Jinja2 HTML templates for input pages and display pages
• theseus/ – core ML library: apis/inference pipelines, base datasets/models, classification/detection/segmentation subpackages, utilities
• No relational database; uses backend/edamam/db.json for simple caching and static CSV files under static/csv for nutrition data.
Key insights from README
• Inference flow in get_prediction (modules.py): runs detection (YOLOv5/YOLOv8), optional classification (EfficientNet-B4) for “Other Foods”, optional segmentation (UNet++), then posts nutrition queries to Edamam, caches results, writes CSV.
• Configuration-driven (device selection, thresholds) via YAML in configs/.
Main components
a. Frontend (Browser)
• HTML pages (templates/)
• JS modules: handle-upload-file.js, handle-input-url.js, display-nutrients-chart.js
• Chart.js for drawing graphs
b. Web Server (Flask)
• app.py: request routing
• backend/routes.py: API endpoints for each input method
• modules.py: orchestrator for inference + Edamam calls + CSV export
c. ML Inference Engine (Theseus)
• theseus/apis/inference: classify, detect, segmentize
• theseus/base: data loading, augmentations, models wrapper
• theseus/[classification,detection,segmentation]: task-specific datasets, augmentations, models
d. Data Store / Cache
• static/uploads & static/assets/uploads: stored input images
• static/csv & static/metadata: output CSV and metadata files
• backend/edamam/db.json: cached nutrition lookup
e. External Service
• Edamam Food Database API: receives item name → returns nutrition JSON
Component interactions
• User action on Frontend → POST to Flask endpoint
• Flask endpoint → modules.get_prediction → calls Theseus inference pipelines
• Inference output → modules applies fallback classification if needed → modules calls Edamam API
• Nutrition results → cached in db.json & saved to CSV → modules returns annotated image path & CSV URL
• Frontend receives response, displays image, asynchronously fetches CSV → Chart.js draws nutrition chart
Architectural patterns & principles
• Layered architecture: Presentation (HTML/CSS/JS) → Application (Flask routes & modules) → Domain (Theseus inference) → Data (file storage + simple JSON cache) → External (Edamam API)
• Configuration-driven pipelines (YAML configs) for easy switching of tasks or parameters
• Caching via perceptual hash (in modules) to avoid repeated inference
• Separation of concerns: small wrappers (backend/edamam), single-responsibility modules.py, self-contained ML lib (theseus/)
Essential elements
• Component names and responsibilities