How aman3214/fruit_monitor works
1. System shape
This repository is a three-runtime IoT pipeline: an embedded/edge producer, a Python AI inference service, and a Node.js dashboard. A central MQTT broker is the main integration point between them.
2. Root orchestrator
main.py is the process launcher. It starts the dashboard, AI processor, and mock/test data generator as separate subprocesses, so the repo behaves like one system even though the pieces run in different runtimes and directories.
3. Edge data source
The hardware boundary is the ESP32-CAM firmware in hardware/esp32_cam_firmware.ino. It represents the real device that captures fruit images and sensor readings, then publishes them over MQTT. The root also includes mock_esp32.py and test/test_output.py for hardware-free simulation of that same data source.
4. Message backbone
MQTT on port 1883 is the primary transport for telemetry and image payloads. It decouples the hardware/simulator from both downstream consumers: the AI engine and the dashboard.
5. AI inference service
ai_engine/ai_processor.py is the core analytics service. It subscribes to MQTT, decodes incoming image data, and runs TensorFlow/Keras inference using fruit_model.h5. This service is the decision-making boundary where raw sensor/image streams become freshness/spoilage predictions.
6. Model runtime artifacts
ai_engine/fruit_model.h5 is the deployed inference artifact, while ai_engine/env_requirements.txt defines the Python runtime needed to execute it. This makes the inference service a separate deployable from the training code.
7. Dashboard backend
dashboard/server.js is a Node.js bridge between MQTT and the browser UI. It receives live system data from MQTT and forwards it to clients over Socket.io, so the frontend can update in real time without speaking MQTT directly.
8. Dashboard frontend
dashboard/public/ contains the browser-facing presentation layer: index.html, script.js, and style.css. This layer is purely visualization and interaction; it depends on the backend websocket stream rather than talking to the hardware or AI service itself.
9. Offline training pipeline
fruit_trainer/ is the model-development subsystem. It contains the dataset layout, dataset reorganization scripts, and train_model.py that produces the .h5 model consumed by the AI engine. The FRUIT-16K folder indicates the source dataset used to derive the training/validation splits.
10. Data lifecycle
The end-to-end flow is: ESP32-CAM or mock publisher → MQTT broker → AI processor for inference → dashboard for live display. The model lifecycle is separate: training dataset → trainer scripts → fruit_model.h5 → AI engine.
11. Persistence and reporting
logs/spoilage_report.json suggests the system also emits structured output for later review or audit, likely summarizing spoilage events or inference results. This is an important boundary between live streaming and stored reporting.
12. Testing and validation
test/test_images and the various test_output scripts provide a simulation and validation path for the pipeline without physical hardware. That makes the repository support both real-device operation and local demo/debug workflows.
13. Key architectural separation
The repo is intentionally split by responsibility: embedded acquisition, ML inference, web presentation, and model training. The important diagram edges are the MQTT broker, the Socket.io bridge to browsers, and the handoff from training artifacts to the deployed AI engine.
Main entry points: ai_processor.py AI processor · main.py Main · mock_esp32.py Mock ESP32 · server.js Server
Components 16
Orchestration
Top-level launcher and shared runtime coordination
- Main · Starts the dashboard, AI processor, and mock data generator as subprocesses
main.py - MQTT broker · Central transport for telemetry and image payloads
Edge capture
Physical and simulated fruit data sources
- ESP32-CAM · Embedded producer that captures fruit data and publishes telemetry over MQTT
hardware/esp32_cam_firmware.ino - Mock ESP32 · Hardware-free publisher that mimics the edge device
mock_esp32.py - Test output · Offline harness for generating or checking pipeline output
test/test_output.py
AI inference
Python analytics and model runtime
- AI processor · Subscribes to MQTT, decodes images, and runs freshness/spoilage prediction
ai_engine/ai_processor.py - Fruit model · Deployed Keras/TensorFlow inference model consumed by the AI processor
ai_engine/fruit_model.h5 - Python deps · Python dependency set for the inference service
ai_engine/env_requirements.txt
Dashboard
Node.js live view and browser UI
- Server · Bridges MQTT data to browser clients over Socket.io
dashboard/server.js - UI shell · Browser entry page for the dashboard
dashboard/public/index.html - Client script · Receives live updates and drives dashboard interaction
dashboard/public/script.js - Styles · Visual styling for the browser dashboard
dashboard/public/style.css
Training
Offline dataset preparation and model build
- FRUIT-16K · Raw labeled fruit image corpus used for model development
fruit_trainer/FRUIT-16K - Split data · Prepared train and validation folders for fresh and spoiled fruit
fruit_trainer/dataset - Train model · Builds the fruit classification model used by inference
fruit_trainer/train_model.py
Reporting
Persistent outputs and review artifacts
- Spoilage report · Structured output for spoilage or inference review
logs/spoilage_report.json
Connections 17
- Main → Mock ESP32: launches · Starts the simulator as part of the local system
- Main → AI processor: launches · Starts the inference service alongside the other components
- Main → Server: launches · Starts the dashboard backend process
- Mock ESP32 → MQTT broker: publishes · Emits simulated telemetry and image payloads
- ESP32-CAM → MQTT broker: publishes · Sends live device data to the message bus
- MQTT broker → AI processor: subscribes · Feeds telemetry and image streams into inference
- MQTT broker → Server: streams · Provides live system updates to the dashboard backend
- Server → Client script: socket.io · Pushes live updates to browser clients
- UI shell → Client script: loads · Hosts the dashboard client entry point
- Client script → Styles: uses · Applies dashboard presentation styles
- FRUIT-16K → Split data: organizes · Source images are curated into train and validation splits
- Split data → Train model: trains · Provides data for model fitting
- Train model → Fruit model: exports · Produces the deployed model artifact
- Fruit model → AI processor: loads · Inference service uses the trained model for predictions
- Python deps → AI processor: supports · Defines the Python runtime dependencies
- AI processor → Spoilage report: writes · Persists structured spoilage or inference results
- Test output → Mock ESP32: validates · Supports hardware-free pipeline checks