# lannguyen0910/food-recognition architecture

> Architecture diagram of the GitHub repository lannguyen0910/food-recognition, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/lannguyen0910/food-recognition
- Repository: https://github.com/lannguyen0910/food-recognition

## Overview

##### Project type and purpose
- This is a full-stack web application for food image analysis: it lets users upload or supply a URL/webcam feed, performs object detection (food items), optionally classification or segmentation, then fetches nutrition data from Edamam and renders annotated images plus nutrition charts in the browser.
- It uses Flask as the web server, a custom “Theseus” Python template for ML pipelines (detection, classification, segmentation), static HTML/CSS/JS for the frontend, and the Edamam external API for nutrition info.

##### File structure
- Top-level:
- app.py – Flask application entrypoint
- backend/ – wrappers for Edamam API (api.py, secret.py), routes.py, utils, constants
- modules.py – orchestration of inference and post-processing
- configs/ – YAML configs for each ML task (detection, classification, segmentation)
- static/ – CSS, JS (Chart.js), assets (uploads, detections), CSV & metadata files
- templates/ – Jinja2 HTML templates for input pages and display pages
- theseus/ – core ML library: apis/inference pipelines, base datasets/models, classification/detection/segmentation subpackages, utilities
- No relational database; uses backend/edamam/db.json for simple caching and static CSV files under static/csv for nutrition data.

##### Key insights from README
- Inference flow in get_prediction (modules.py): runs detection (YOLOv5/YOLOv8), optional classification (EfficientNet-B4) for “Other Foods”, optional segmentation (UNet++), then posts nutrition queries to Edamam, caches results, writes CSV.
- Configuration-driven (device selection, thresholds) via YAML in configs/.

##### Main components
a. Frontend (Browser)
- HTML pages (templates/)
- JS modules: handle-upload-file.js, handle-input-url.js, display-nutrients-chart.js
- Chart.js for drawing graphs
b. Web Server (Flask)
- app.py: request routing
- backend/routes.py: API endpoints for each input method
- modules.py: orchestrator for inference + Edamam calls + CSV export
c. ML Inference Engine (Theseus)
- theseus/apis/inference: classify, detect, segmentize
- theseus/base: data loading, augmentations, models wrapper
- theseus/[classification,detection,segmentation]: task-specific datasets, augmentations, models
d. Data Store / Cache
- static/uploads & static/assets/uploads: stored input images
- static/csv & static/metadata: output CSV and metadata files
- backend/edamam/db.json: cached nutrition lookup
e. External Service
- Edamam Food Database API: receives item name → returns nutrition JSON

##### Component interactions
- User action on Frontend → POST to Flask endpoint
- Flask endpoint → modules.get_prediction → calls Theseus inference pipelines
- Inference output → modules applies fallback classification if needed → modules calls Edamam API
- Nutrition results → cached in db.json & saved to CSV → modules returns annotated image path & CSV URL
- Frontend receives response, displays image, asynchronously fetches CSV → Chart.js draws nutrition chart

##### Architectural patterns & principles
- Layered architecture: Presentation (HTML/CSS/JS) → Application (Flask routes & modules) → Domain (Theseus inference) → Data (file storage + simple JSON cache) → External (Edamam API)
- Configuration-driven pipelines (YAML configs) for easy switching of tasks or parameters
- Caching via perceptual hash (in modules) to avoid repeated inference
- Separation of concerns: small wrappers (backend/edamam), single-responsibility modules.py, self-contained ML lib (theseus/)

##### Essential elements
- Component names and responsibilities

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Frontend Layer
    subgraph "Frontend" 
        direction TB
        UploadHTML["upload-file.html"]:::frontend
        DetectHTML["detect-input-url.html"]:::frontend
        WebcamHTML["webcam-capture.html"]:::frontend
        HandleUpload["handle-upload-file.js"]:::frontend
        HandleURL["handle-input-url.js"]:::frontend
        HandleWebcam["handle-webcam-capture.js"]:::frontend
        DisplayChart["display-nutrients-chart.js"]:::frontend
        AllJS["all.js (bundled libs)"]:::frontend
        StyleCSS["style.css"]:::frontend
        BootstrapCSS["bootstrap.css"]:::frontend
        AboutCSS["about.css"]:::frontend
    end

    Browser["Browser (User)"]:::frontend

    %% Flask Server Layer
    subgraph "Flask Server" 
        direction TB
        App["Flask Entrypoint (app.py)"]:::backend
        Routes["Routing & API Endpoints (backend/routes.py)"]:::backend
        Modules["Inference Orchestrator (backend/modules.py)"]:::backend
        Utils["Constants & Utilities"]:::backend
        subgraph "Edamam Integration"
            direction TB
            API["API Wrapper (backend/edamam/api.py)"]:::backend
            Secret["API Credentials (backend/edamam/secret.py)"]:::backend
            DBCache["Nutrition Cache (backend/edamam/db.json)"]:::datastore
        end
    end

    %% Theseus ML Layer
    subgraph "Theseus Inference Engine"
        direction TB
        Detect["Detect Pipeline (theseus/apis/inference/detect.py)"]:::ml
        Classify["Classify Pipeline (theseus/apis/inference/classify.py)"]:::ml
        Segment["Segment Pipeline (theseus/apis/inference/segmentize.py)"]:::ml
        BaseUtils["Core Utilities & Models"]:::ml
        TaskSpecific["Task-specific Modules"]:::ml
    end

    %% Data Storage Layer
    subgraph "Data Storage" 
        direction TB
        Uploads["Uploaded Images (static/assets/uploads/)"]:::datastore
        Detections["Processed Images (static/assets/detections/)"]:::datastore
        CSV["Nutrition CSV Outputs (static/csv/)"]:::datastore
        Metadata["Metadata Files (static/metadata/)"]:::datastore
    end

    %% External Service
    External["Edamam Food API"]:::external

    %% Data Flows
    Browser -->|"HTTP POST"| App
    App -->|routes to| Routes
    Routes -->|calls| Modules
    Browser -->|"upload image"| Uploads
    Modules -->|runs| Detect
    Detect -->|fallback| Classify
    Classify -->|segmentation| Segment
    Segment -->|return results| Modules
    Modules -->|"Edamam Query"| API
    API -->|"HTTP Request"| External
    External -->|"Nutrition JSON"| API
    API -->|"cache"| DBCache
    Modules -->|"save image"| Detections
    Modules -->|"write CSV"| CSV
    Modules -->|"write metadata"| Metadata
    Browser -->|"fetch CSV"| CSV
    Browser -->|"display chart"| AllJS

    %% Click Events for Frontend
    click UploadHTML "https://github.com/lannguyen0910/food-recognition/blob/master/templates/upload-file.html"
    click DetectHTML "https://github.com/lannguyen0910/food-recognition/blob/master/templates/detect-input-url.html"
    click WebcamHTML "https://github.com/lannguyen0910/food-recognition/blob/master/templates/webcam-capture.html"
    click HandleUpload "https://github.com/lannguyen0910/food-recognition/blob/master/static/js/handle-upload-file.js"
    click HandleURL "https://github.com/lannguyen0910/food-recognition/blob/master/static/js/handle-input-url.js"
    click HandleWebcam "https://github.com/lannguyen0910/food-recognition/blob/master/static/js/handle-webcam-capture.js"
    click DisplayChart "https://github.com/lannguyen0910/food-recognition/blob/master/static/js/display-nutrients-chart.js"
    click AllJS "https://github.com/lannguyen0910/food-recognition/blob/master/static/js/all.js"
    click StyleCSS "https://github.com/lannguyen0910/food-recognition/blob/master/static/css/style.css"
    click BootstrapCSS "https://github.com/lannguyen0910/food-recognition/blob/master/static/css/bootstrap.css"
    click AboutCSS "https://github.com/lannguyen0910/food-recognition/blob/master/static/css/about.css"

    %% Click Events for Flask Server
    click App "https://github.com/lannguyen0910/food-recognition/blob/master/app.py"
    click Routes "https://github.com/lannguyen0910/food-recognition/blob/master/backend/routes.py"
    click Modules "https://github.com/lannguyen0910/food-recognition/blob/master/backend/modules.py"
    click Utils "https://github.com/lannguyen0910/food-recognition/blob/master/backend/constants.py"
    click Utils "https://github.com/lannguyen0910/food-recognition/blob/master/backend/utils.py"

    %% Click Events for Edamam Integration
    click API "https://github.com/lannguyen0910/food-recognition/blob/master/backend/edamam/api.py"
    click Secret "https://github.com/lannguyen0910/food-recognition/blob/master/backend/edamam/secret.py"
    click DBCache "https://github.com/lannguyen0910/food-recognition/blob/master/backend/edamam/db.json"

    %% Click Events for Theseus
    click Detect "https://github.com/lannguyen0910/food-recognition/blob/master/theseus/apis/inference/detect.py"
    click Classify "https://github.com/lannguyen0910/food-recognition/blob/master/theseus/apis/inference/classify.py"
    click Segment "https://github.com/lannguyen0910/food-recognition/blob/master/theseus/apis/inference/segmentize.py"
    click BaseUtils "https://github.com/lannguyen0910/food-recognition/tree/master/theseus/base/augmentations/"
    click BaseUtils "https://github.com/lannguyen0910/food-recognition/tree/master/theseus/base/datasets/"
    click BaseUtils "https://github.com/lannguyen0910/food-recognition/tree/master/theseus/base/models/"
    click TaskSpecific "https://github.com/lannguyen0910/food-recognition/tree/master/theseus/detection/"
    click TaskSpecific "https://github.com/lannguyen0910/food-recognition/tree/master/theseus/classification/"
    click TaskSpecific "https://github.com/lannguyen0910/food-recognition/tree/master/theseus/segmentation/"

    %% Click Events for Data Storage
    click Uploads "https://github.com/lannguyen0910/food-recognition/tree/master/static/assets/uploads/"
    click Detections "https://github.com/lannguyen0910/food-recognition/tree/master/static/assets/detections/"
    click CSV "https://github.com/lannguyen0910/food-recognition/tree/master/static/csv/"
    click Metadata "https://github.com/lannguyen0910/food-recognition/tree/master/static/metadata/"

    %% Styles
    classDef frontend fill:#e0f9e0,stroke:#2a7f2a,color:#2a7f2a
    classDef backend fill:#e0e8f9,stroke:#2a52be,color:#2a52be
    classDef ml fill:#fff4e5,stroke:#e38b00,color:#e38b00
    classDef datastore fill:#f2f2f2,stroke:#8a8a8a,color:#555555
    classDef external fill:#f3e5f5,stroke:#9c27b0,color:#9c27b0
```
