How jebinshaju/visionmaxing works
VisionMaxing is a mobile-first IoT + backend AI/TTS pipeline:
• Edge device (ESP32-CAM) captures images and exposes them via HTTP JPEG endpoints on a local network.
• FastAPI backend orchestrates:
• Pulling frames from the ESP32-CAM
• Converting image → text description (currently may be simple; code suggests optional AI via hf_client.py)
• Converting text → speech audio via tts.py (README mentions gTTS)
• Exposing API endpoints for the mobile app (JSON + audio URL)
• Flutter mobile app is the client UX: triggers snapshot/monitoring, fetches JSON + mp3, plays audio.
ESP32-CAM firmware (edge node)
From ESP32CAM/:
• main.ino: overall orchestration (Wi-Fi connect, camera init, server start, sensor loop)
• CameraModule.h: camera initialization + capture (uses yoursunny/esp32cam)
• WebServerModule.h: HTTP server that serves images (/cam-lo.jpg, /cam-mid.jpg, /cam-hi.jpg)
• UltrasonicModule.h: optional obstacle sensing + buzzer (an optional side-channel output)
FastAPI backend (orchestrator service)
From backend/app/:
• main.py: FastAPI entrypoint; includes route registration
• config.py: runtime config (ESP32 IP, temp paths, etc.)
• routes/
• ip_camera_routes.py: core pipeline: fetch camera image → describe → TTS → return response
• image_routes.py, audio_routes.py: likely separate endpoints for uploading images / serving audio
• tts.py: text-to-speech generation (gTTS or wrappers)
• hf_client.py: Hugging Face inference client (external AI dependency)
• utils.py: helpers (download image, temp file ops, naming, etc.)
Flutter mobile app (client)
From mobile_app/lib/main.dart:
• UI + settings (backend base URL)
• monitoring loop (periodic requests)
• audio playback (audioplayers)
• persistence (SharedPreferences)
Diagramming rules to keep it accurate (principal-engineer level)
• Protocol: HTTP/HTTPS
• Data type: JPEG, JSON, MP3
• Direction: who calls whom (important here)
• FastAPI -> ESP32-CAM: HTTP GET /cam-hi.jpg (image/jpeg)
• Flutter -> FastAPI: HTTP GET /describe-ip-camera (application/json)
• Flutter -> FastAPI: HTTP GET /audio/<id>.mp3 (audio/mpeg)
Boundaries and trust zones
At minimum:
• Local Wi-Fi / LAN: ESP32 + (often) backend
• Mobile network / Wi-Fi: phone connectivity to backend
• Internet: HF/gTTS
Configuration sources as first-class elements
These matter operationally:
• ESP32 Wi-Fi creds & pins (Config.h)
• Backend config (config.py, ESP32 IP)
• Mobile base URL in settings (SharedPreferences)
Them as “Config” notes attached to each container.
What to include for “as many components as possible” (without overthinking)
ESP32-CAM
• Wi-Fi Connect
• Camera Driver (CameraModule)
• HTTP Image Server (WebServerModule)
• Ultrasonic Sensor (UltrasonicModule)
• Buzzer/Alert Output
Backend (FastAPI)
• Uvicorn/ASGI Server
• API Routers (routes/*)
• IP Camera Client (fetch JPEG)
• Vision/Description module (local logic + optional HF client)
• TTS module (gTTS wrapper)
• Temp Audio Store (filesystem)
• Static/Audio Serving endpoint
Mobile App (Flutter)
• UI Controls (mic button, monitoring start/stop)
• Settings (backend URL)
• Network Client
• Monitoring Scheduler/Timer
• Audio Playback (audioplayers)
• Persistence (SharedPreferences)
External
• Hugging Face Inference API (optional)
• Google TTS (gTTS)
• Clear labels for each component (use filenames/modules where helpful: ip_camera_routes.py, tts.py)
• Network boundaries (LAN vs Internet)
• Explicit endpoints (camera JPG URLs; backend describe + audio URLs)
• A monitoring loop depiction (periodic polling)