# jebinshaju/visionmaxing architecture

> Architecture diagram of the GitHub repository jebinshaju/visionmaxing, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-03-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jebinshaju/visionmaxing
- Repository: https://github.com/jebinshaju/visionmaxing

## Overview

**VisionMaxing** is a **mobile-first IoT + backend AI/TTS pipeline**:
- **Edge device (ESP32-CAM)** captures images and exposes them via **HTTP JPEG endpoints** on a local network.
- **FastAPI backend** orchestrates:
- Pulling frames from the ESP32-CAM
- Converting image → **text description** (currently may be simple; code suggests optional AI via `hf_client.py`)
- Converting text → **speech audio** via `tts.py` (README mentions **gTTS**)
- Exposing API endpoints for the mobile app (JSON + audio URL)
- **Flutter mobile app** is the client UX: triggers snapshot/monitoring, fetches JSON + mp3, plays audio.

##### ESP32-CAM firmware (edge node)
From `ESP32CAM/`:
- `main.ino`: overall orchestration (Wi-Fi connect, camera init, server start, sensor loop)
- `CameraModule.h`: camera initialization + capture (uses `yoursunny/esp32cam`)
- `WebServerModule.h`: HTTP server that serves images (`/cam-lo.jpg`, `/cam-mid.jpg`, `/cam-hi.jpg`)
- `UltrasonicModule.h`: optional obstacle sensing + buzzer (an **optional side-channel output**)

##### FastAPI backend (orchestrator service)
From `backend/app/`:
- `main.py`: FastAPI entrypoint; includes route registration
- `config.py`: runtime config (ESP32 IP, temp paths, etc.)
- `routes/`
- `ip_camera_routes.py`: core pipeline: fetch camera image → describe → TTS → return response
- `image_routes.py`, `audio_routes.py`: likely separate endpoints for uploading images / serving audio
- `tts.py`: text-to-speech generation (gTTS or wrappers)
- `hf_client.py`: Hugging Face inference client (external AI dependency)
- `utils.py`: helpers (download image, temp file ops, naming, etc.)

##### Flutter mobile app (client)
From `mobile_app/lib/main.dart`:
- UI + settings (backend base URL)
- monitoring loop (periodic requests)
- audio playback (`audioplayers`)
- persistence (`SharedPreferences`)

##### Diagramming rules to keep it accurate (principal-engineer level)

- **Protocol**: HTTP/HTTPS
- **Data type**: JPEG, JSON, MP3
- **Direction**: who calls whom (important here)

- `FastAPI -> ESP32-CAM: HTTP GET /cam-hi.jpg (image/jpeg)`
- `Flutter -> FastAPI: HTTP GET /describe-ip-camera (application/json)`
- `Flutter -> FastAPI: HTTP GET /audio/<id>.mp3 (audio/mpeg)`

##### Boundaries and trust zones
At minimum:
- **Local Wi-Fi / LAN**: ESP32 + (often) backend
- **Mobile network / Wi-Fi**: phone connectivity to backend
- **Internet**: HF/gTTS

##### Configuration sources as first-class elements
These matter operationally:
- ESP32 Wi-Fi creds & pins (`Config.h`)
- Backend config (`config.py`, ESP32 IP)
- Mobile base URL in settings (SharedPreferences)

Them as “Config” notes attached to each container.

##### What to include for “as many components as possible” (without overthinking)

**ESP32-CAM**
- Wi-Fi Connect
- Camera Driver (`CameraModule`)
- HTTP Image Server (`WebServerModule`)
- Ultrasonic Sensor (`UltrasonicModule`)
- Buzzer/Alert Output

**Backend (FastAPI)**
- Uvicorn/ASGI Server
- API Routers (`routes/*`)
- IP Camera Client (fetch JPEG)
- Vision/Description module (local logic + optional HF client)
- TTS module (gTTS wrapper)
- Temp Audio Store (filesystem)
- Static/Audio Serving endpoint

**Mobile App (Flutter)**
- UI Controls (mic button, monitoring start/stop)
- Settings (backend URL)
- Network Client
- Monitoring Scheduler/Timer
- Audio Playback (audioplayers)
- Persistence (SharedPreferences)

**External**
- Hugging Face Inference API (optional)
- Google TTS (gTTS)

- Clear labels for each component (use filenames/modules where helpful: `ip_camera_routes.py`, `tts.py`)
- Network boundaries (LAN vs Internet)
- Explicit endpoints (camera JPG URLs; backend describe + audio URLs)
- A monitoring loop depiction (periodic polling)

## Diagram (Mermaid)

```mermaid
flowchart TD

%% =========================================================
%% Diagram 1 — System Context (C4-ish)
%% =========================================================
subgraph "Diagram 1 — System Context"
direction TB

subgraph "Actor"
direction TB
User["Visually impaired user"]:::actor
end

subgraph "VisionMaxing System"
direction TB
MobileApp["Flutter Mobile App"]:::mobile
BackendSvc["FastAPI Backend"]:::backend
EdgeESP["ESP32-CAM (Camera + Web Server)"]:::edge
end

subgraph "External Services (Internet)"
direction TB
HFCloud["Hugging Face Inference API (optional)"]:::external
GTTSCloud["Google TTS (gTTS)"]:::external
end

User <-->|"UX_interaction"| MobileApp
MobileApp <-->|"HTTP_JSON+MP3"| BackendSvc
BackendSvc -->|"HTTP_GET_JPEG"| EdgeESP
BackendSvc -->|"HTTPS_image->text(optional)"| HFCloud
BackendSvc -->|"HTTPS_text->speech"| GTTSCloud

LANNote["LAN/Wi-Fi: ESP32 + (often) Backend\nInternet: HF/gTTS\nMobile reaches Backend via Wi-Fi/Cell"]:::note
LANNote --- EdgeESP
LANNote --- BackendSvc
LANNote --- HFCloud
end

%% =========================================================
%% Diagram 2 — Container / Deployment
%% =========================================================
subgraph "Diagram 2 — Container / Deployment"
direction TB

subgraph "Local Wi-Fi / LAN"
direction TB
ESP32Node["ESP32-CAM Device"]:::edge
ESP32Http["HTTP JPEG Server\n:80\n/cam-lo.jpg\n/cam-mid.jpg\n/cam-hi.jpg"]:::edgeComp

BackendHost["Backend Host (Laptop/RPi/VM)"]:::backend
ASGI["Uvicorn + FastAPI\n:8000"]:::backendComp
TempFS[("Temp Audio Cache\n(mp3 on filesystem)")]:::storage
end

subgraph "Mobile Device"
direction TB
Phone["Phone (Flutter)"]:::mobile
UI["UI + Controls"]:::mobileComp
Timer["Monitoring Scheduler/Timer\n(every N seconds)"]:::mobileComp
NetClient["HTTP Client"]:::mobileComp
Player["Audio Playback (audioplayers)"]:::mobileComp
Prefs[("SharedPreferences\nBackend base URL")]:::storage
end

subgraph "Internet"
direction TB
HF2["Hugging Face Inference API (optional)"]:::external
GTTS2["Google TTS (gTTS)"]:::external
end

ESP32Node -->|"hosts"| ESP32Http
BackendHost -->|"runs"| ASGI
BackendHost -->|"writes/reads"| TempFS

Phone -->|"contains"| UI
Phone -->|"contains"| Timer
Phone -->|"contains"| NetClient
Phone -->|"contains"| Player
Phone -->|"stores"| Prefs

NetClient -->|"HTTP_GET_:8000_/describe-ip-camera_(application/json)"| ASGI
Timer -->|"triggers_periodic_requests"| NetClient
NetClient -->|"HTTP_GET_:8000_/audio/<id>.mp3_(audio/mpeg)"| ASGI

ASGI -->|"HTTP_GET_:80_/cam-hi.jpg_(image/jpeg)\n(pull-based)"| ESP32Http
ASGI -->|"HTTPS_image->text(optional)"| HF2
ASGI -->|"HTTPS_text->speech"| GTTS2
ASGI -->|"store_mp3"| TempFS

ConfESP["Config: Wi-Fi creds + pins + camera settings\n(Config.h)"]:::note
ConfBE["Config: ESP32 IP + temp paths\n(config.py)"]:::note
ConfMO["Config: Backend base URL\n(SharedPreferences)"]:::note

ConfESP --- ESP32Node
ConfBE --- BackendHost
ConfMO --- Phone
end

%% =========================================================
%% Diagram 3 — Backend Component Diagram (FastAPI internals)
%% =========================================================
subgraph "Diagram 3 — Backend Components (FastAPI 'Brain')"
direction TB

subgraph "FastAPI Backend (backend/app)"
direction TB

Entry["FastAPI Entrypoint + Route Registration\n(main.py)"]:::backendComp
Cfg["Runtime Configuration\n(config.py)"]:::config

subgraph "API Router Layer (routes/)"
direction TB
RouterAll["Routes Package (routes/)"]:::backendComp
IPCamRoute["IP Camera Pipeline Route\n(ip_camera_routes.py)"]:::api
AudioRoute["Audio Serving Endpoints\n(audio_routes.py)"]:::api
ImageRoute["Image Endpoints\n(image_routes.py)"]:::api
end

Utils["Helpers\n(utils.py)\n(download_image,temp_files,naming)"]:::backendComp

subgraph "Pipeline Services"
direction TB
CamClient["IP Camera Client\nHTTP GET http://ESP32_IP/cam-*.jpg"]:::service
Describe["Scene Description Engine\n(local + optional HF)"]:::service
HFClient["HF Inference Client\n(hf_client.py)"]:::service
TTS["Text-to-Speech Engine\n(tts.py, gTTS wrapper)"]:::service
Resp["Response Builder\nJSON {description,audio_url}"]:::service
end

TempAudio[("Temp Storage\nMP3 files")]:::storage
end

subgraph "Upstream/Downstream"
direction TB
Mobile3["Flutter App"]:::mobile
ESP323["ESP32 JPEG Endpoints\n/cam-lo.jpg,/cam-mid.jpg,/cam-hi.jpg"]:::edge
HF3["Hugging Face API (optional)"]:::external
GTTS3["Google TTS (gTTS)"]:::external
end

%% request/response + data flow
Mobile3 -->|"1_HTTP_request_description(JSON)"| IPCamRoute
IPCamRoute -->|"uses"| Utils
IPCamRoute -->|"2_call_fetch_latest_frame"| CamClient
CamClient -->|"2_HTTP_GET_/cam-hi.jpg_(image/jpeg)"| ESP323
IPCamRoute -->|"3_image->text"| Describe
Describe -->|"optional_inference_call"| HFClient
HFClient -->|"3_HTTPS_request"| HF3
Describe -->|"3_returns_description_text"| IPCamRoute
IPCamRoute -->|"4_text->mp3"| TTS
TTS -->|"4_HTTPS_text->speech"| GTTS3
TTS -->|"5_write_mp3"| TempAudio
IPCamRoute -->|"6_build_JSON_{description,audio_url}"| Resp
Resp -->|"6_HTTP_response_JSON"| Mobile3

Mobile3 -->|"7_HTTP_GET_/audio/<id>.mp3_(audio/mpeg)"| AudioRoute
AudioRoute -->|"read_mp3"| TempAudio
AudioRoute -->|"7_HTTP_response_MP3"| Mobile3

Entry -->|"registers"| RouterAll
RouterAll -->|"includes"| IPCamRoute
RouterAll -->|"includes"| AudioRoute
RouterAll -->|"includes"| ImageRoute
Entry -->|"reads"| Cfg

MonLoop["Monitoring loop: Flutter polls describe endpoint every N seconds"]:::note
MonLoop --- Mobile3
MonLoop --- IPCamRoute
end

%% =========================================================
%% Optional: ESP32 internal modules + side-channel obstacle alert
%% =========================================================
subgraph "ESP32-CAM Firmware (ESP32CAM/)"
direction TB
Ino["Firmware Orchestrator\n(main.ino)"]:::edgeComp
EspCfg["Device Configuration\n(Config.h)"]:::config
CamMod["Camera Init/Capture\n(CameraModule.h)"]:::edgeComp
WebMod["HTTP Image Server\n(WebServerModule.h)\n/cam-lo.jpg,/cam-mid.jpg,/cam-hi.jpg"]:::edgeComp
Ultra["Ultrasonic Obstacle Sensing (optional)\n(UltrasonicModule.h)"]:::edgeComp
Buzzer["Buzzer / Alert Output\n(local, works offline)"]:::edgeAct

Ino -->|"uses"| EspCfg
Ino -->|"uses"| CamMod
Ino -->|"starts"| WebMod
Ino -->|"optional_loop"| Ultra
Ultra -->|"trigger_alert"| Buzzer
CamMod -->|"provides_frames"| WebMod
end

%% Hidden nodes to attach extra click events without changing visuals
ReqTxt["Backend dependencies"]:::hidden
Readme["Documentation"]:::hidden

%% =========================================================
%% Click Events (from component_mapping)
%% =========================================================
click Ino "https://github.com/jebinshaju/visionmaxing/blob/main/ESP32CAM/main.ino"
click EspCfg "https://github.com/jebinshaju/visionmaxing/blob/main/ESP32CAM/Config.h"
click CamMod "https://github.com/jebinshaju/visionmaxing/blob/main/ESP32CAM/CameraModule.h"
click WebMod "https://github.com/jebinshaju/visionmaxing/blob/main/ESP32CAM/WebServerModule.h"
click Ultra "https://github.com/jebinshaju/visionmaxing/blob/main/ESP32CAM/UltrasonicModule.h"

click Entry "https://github.com/jebinshaju/visionmaxing/blob/main/backend/app/main.py"
click Cfg "https://github.com/jebinshaju/visionmaxing/blob/main/backend/app/config.py"
click RouterAll "https://github.com/jebinshaju/visionmaxing/tree/main/backend/app/routes/"
click IPCamRoute "https://github.com/jebinshaju/visionmaxing/blob/main/backend/app/routes/ip_camera_routes.py"
click AudioRoute "https://github.com/jebinshaju/visionmaxing/blob/main/backend/app/routes/audio_routes.py"
click ImageRoute "https://github.com/jebinshaju/visionmaxing/blob/main/backend/app/routes/image_routes.py"
click TTS "https://github.com/jebinshaju/visionmaxing/blob/main/backend/app/tts.py"
click HFClient "https://github.com/jebinshaju/visionmaxing/blob/main/backend/app/hf_client.py"
click Utils "https://github.com/jebinshaju/visionmaxing/blob/main/backend/app/utils.py"

click MobileApp "https://github.com/jebinshaju/visionmaxing/blob/main/mobile_app/lib/main.dart"
click Mobile3 "https://github.com/jebinshaju/visionmaxing/blob/main/mobile_app/lib/main.dart"
click Phone "https://github.com/jebinshaju/visionmaxing/blob/main/mobile_app/lib/main.dart"

click ReqTxt "https://github.com/jebinshaju/visionmaxing/blob/main/backend/requirements.txt"
click Readme "https://github.com/jebinshaju/visionmaxing/blob/main/readme.md"

%% =========================================================
%% Styles
%% =========================================================
classDef actor fill:#1f2937,stroke:#111827,color:#ffffff,stroke-width:1px
classDef edge fill:#065f46,stroke:#064e3b,color:#ffffff,stroke-width:1px
classDef edgeComp fill:#047857,stroke:#064e3b,color:#ffffff,stroke-width:1px
classDef edgeAct fill:#059669,stroke:#064e3b,color:#ffffff,stroke-width:1px

classDef backend fill:#1d4ed8,stroke:#1e3a8a,color:#ffffff,stroke-width:1px
classDef backendComp fill:#2563eb,stroke:#1e3a8a,color:#ffffff,stroke-width:1px
classDef api fill:#3b82f6,stroke:#1e3a8a,color:#ffffff,stroke-width:1px
classDef service fill:#60a5fa,stroke:#1e3a8a,color:#0b1220,stroke-width:1px

classDef mobile fill:#7c3aed,stroke:#4c1d95,color:#ffffff,stroke-width:1px
classDef mobileComp fill:#8b5cf6,stroke:#4c1d95,color:#ffffff,stroke-width:1px

classDef external fill:#ea580c,stroke:#9a3412,color:#ffffff,stroke-width:1px

classDef storage fill:#0f766e,stroke:#134e4a,color:#ffffff,stroke-width:1px
classDef config fill:#a16207,stroke:#78350f,color:#ffffff,stroke-width:1px
classDef note fill:#f3f4f6,stroke:#9ca3af,color:#111827,stroke-dasharray: 4 3
classDef hidden fill:#ffffff,stroke:#ffffff,color:#ffffff

%% ensure hidden nodes don't take attention
class ReqTxt,Readme hidden
```
