# soumyaritu/echorxaimediassistant architecture

> Architecture diagram of the GitHub repository soumyaritu/echorxaimediassistant, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-05-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/soumyaritu/echorxaimediassistant
- Repository: https://github.com/soumyaritu/echorxaimediassistant

## Overview

1. **Top-level shape**
   EchoRx is a two-part system: a **React Native mobile app** in `app/` and a **Python FastAPI backend** in `backend/`. The phone app captures the medicine photo and renders the result UI; the backend performs all AI reasoning and returns structured guidance.

2. **Client boundary**
   The app is an **Expo-based React Native client** (`app/App.js`, `app/index.js`, `app/app.json`). It is intentionally thin: it handles camera/photo capture, the 3-screen flow described in the README, accessibility, and text-to-speech playback, but does not do the AI inference itself.

3. **Backend boundary**
   The backend is a **local FastAPI service** (`backend/server.py`) that exposes the HTTP API used by the app. It acts as the orchestration layer for the AI pipeline and is configured through `backend/config.py` and environment variables such as `GEMINI_API_KEY`.

4. **Multi-agent AI pipeline**
   The core architecture is a **3-agent chain** under `backend/agents/`:
   - `labelsense.py` extracts what is visible in the image, such as medicine name, expiry, warnings, and confidence.
   - `safetyguard.py` validates and enriches the result with safety rules, dosage limits, expiry checks, and non-medicine hazard handling.
   - `voiceassistant.py` rewrites the output into short, spoken-friendly language.
   `root.py` coordinates the sequence and combines the final response.

5. **Data flow**
   The main flow is: **photo on phone → HTTP request to FastAPI → vision/analysis by LabelSense → safety validation by SafetyGuard → narration text by VoiceAssist → JSON response back to app → spoken output in the app**. This is the primary path an architecture diagram should show.

6. **External AI dependency**
   The system depends on **Google ADK** and **Gemini 2.5 Flash** for the image/agent reasoning. In architectural terms, Gemini is the external intelligence service behind label reading and safety interpretation, while ADK appears to provide the agent framework and orchestration model.

7. **Runtime split**
   The repo spans two runtimes:
   - **Node/Expo runtime** for the mobile client.
   - **Python/FastAPI runtime** for the backend agents.
   The README’s local-network setup implies the app talks to the backend over a configurable LAN API URL in `app/config.js` or `app/App.js`.

8. **Output contract**
   The backend likely returns a **structured medicine guidance payload** rather than free text only, because the app shows distinct UI sections such as medicine name, usage, max daily dose, safety check, and confidence. The voice text is derived from the same response so the visual and spoken outputs stay aligned.

9. **Safety and trust layer**
   A major architectural concern is **hallucination control**. The README describes explicit rules in `LabelSenseAgent` for expiry extraction and `SafetyGuardAgent` for real-time validation against the current date. This makes safety not just a prompt concern, but a dedicated subsystem in the backend.

10. **Accessibility layer in the client**
    The mobile app includes a substantial **accessibility presentation layer**: screen-reader headings, hidden decorative icons, descriptive labels/hints, and a text-to-speech playback control. This is part of the app architecture, not just styling, because it determines how users consume the backend result.

11. **No persistence / no database**
    There is no database, queue, or user account subsystem in this repo. The app appears to be **stateless per scan**: each medicine scan is analyzed on demand, returned, and spoken, with no long-term storage shown in the tree.

12. **Operational shape**
    The repository is designed for **local development and phone-on-LAN usage** rather than a hosted production stack. The backend runs on port 8000, and the app points to the developer machine’s IP address, so the architecture diagram should show a direct client-to-local-backend connection rather than cloud hosting.

13. **Key modules to represent**
    For a diagram, the important boxes are:
    - **Mobile UI / Expo app**
    - **FastAPI API server**
    - **Root agent orchestrator**
    - **LabelSense vision agent**
    - **SafetyGuard safety agent**
    - **VoiceAssist narration agent**
    - **Gemini/Google ADK external service**
    These are the real architectural boundaries in this codebase.

14. **End-user purpose**
    The system is intentionally optimized for **elderly and visually impaired users**, so the architecture is not just “scan and classify”; it is “scan, verify, simplify, and speak.” That purpose explains why the backend is multi-agent and why the frontend includes native accessibility and audio controls.

Main entry points: [`app/App.js`](https://github.com/soumyaritu/echorxaimediassistant/blob/main/app/App.js) (Expo app), [`backend/server.py`](https://github.com/soumyaritu/echorxaimediassistant/blob/main/backend/server.py) (FastAPI server), [`backend/agents/root.py`](https://github.com/soumyaritu/echorxaimediassistant/blob/main/backend/agents/root.py) (Root agent), [`backend/agents/labelsense.py`](https://github.com/soumyaritu/echorxaimediassistant/blob/main/backend/agents/labelsense.py) (LabelSense)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_client["Mobile client"]
  node_app_shell["Expo app<br/>React Native client<br/>[App.js]"]
  node_app_entry["App entry<br/>Startup<br/>[index.js]"]
  node_app_config["Client config<br/>Runtime config<br/>[config.js]"]
  node_app_manifest["App manifest<br/>Expo manifest<br/>[app.json]"]
  node_camera_flow(("Scan flow<br/>Capture UI<br/>[App.js]"))
  node_accessibility_tts(("Accessibility + speech<br/>A11y/UI<br/>[App.js]"))
end

subgraph group_backend["AI backend"]
  node_api_server["FastAPI server<br/>HTTP API<br/>[server.py]"]
  node_backend_config["Backend config<br/>Runtime config<br/>[config.py]"]
  node_root_orchestrator(("Root agent<br/>Orchestrator<br/>[root.py]"))
  node_labelsense{{"LabelSense<br/>Vision agent<br/>[labelsense.py]"}}
  node_safetyguard{{"SafetyGuard<br/>Safety agent<br/>[safetyguard.py]"}}
  node_voiceassistant{{"VoiceAssist<br/>Narration agent<br/>[voiceassistant.py]"}}
end

subgraph group_external["External AI"]
  node_adk_runtime[("Google ADK<br/>Agent framework")]
  node_gemini_model[("Gemini 2.5 Flash<br/>Vision model")]
end

node_app_entry -->|"boots"| node_app_shell
node_app_manifest -->|"defines"| node_app_shell
node_app_config -->|"backend URL"| node_api_server
node_app_shell -->|"captures photo"| node_camera_flow
node_camera_flow -->|"POST scan"| node_api_server
node_api_server -->|"dispatches"| node_root_orchestrator
node_backend_config -->|"configures"| node_api_server
node_root_orchestrator -->|"1. analyze"| node_labelsense
node_labelsense -->|"runs on"| node_adk_runtime
node_labelsense -->|"vision reasoning"| node_gemini_model
node_root_orchestrator -->|"2. validate"| node_safetyguard
node_safetyguard -->|"runs on"| node_adk_runtime
node_safetyguard -->|"policy reasoning"| node_gemini_model
node_root_orchestrator -->|"3. narrate"| node_voiceassistant
node_voiceassistant -->|"runs on"| node_adk_runtime
node_api_server -->|"JSON response"| node_app_shell
node_app_shell -->|"renders / speaks"| node_accessibility_tts

click node_app_shell "https://github.com/soumyaritu/echorxaimediassistant/blob/main/app/App.js"
click node_app_entry "https://github.com/soumyaritu/echorxaimediassistant/blob/main/app/index.js"
click node_app_config "https://github.com/soumyaritu/echorxaimediassistant/blob/main/app/config.js"
click node_app_manifest "https://github.com/soumyaritu/echorxaimediassistant/blob/main/app/app.json"
click node_camera_flow "https://github.com/soumyaritu/echorxaimediassistant/blob/main/app/App.js"
click node_accessibility_tts "https://github.com/soumyaritu/echorxaimediassistant/blob/main/app/App.js"
click node_api_server "https://github.com/soumyaritu/echorxaimediassistant/blob/main/backend/server.py"
click node_backend_config "https://github.com/soumyaritu/echorxaimediassistant/blob/main/backend/config.py"
click node_root_orchestrator "https://github.com/soumyaritu/echorxaimediassistant/blob/main/backend/agents/root.py"
click node_labelsense "https://github.com/soumyaritu/echorxaimediassistant/blob/main/backend/agents/labelsense.py"
click node_safetyguard "https://github.com/soumyaritu/echorxaimediassistant/blob/main/backend/agents/safetyguard.py"
click node_voiceassistant "https://github.com/soumyaritu/echorxaimediassistant/blob/main/backend/agents/voiceassistant.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_app_shell,node_app_entry,node_app_config,node_app_manifest,node_camera_flow,node_accessibility_tts toneBlue
class node_api_server,node_backend_config,node_root_orchestrator,node_labelsense,node_safetyguard,node_voiceassistant toneAmber
class node_adk_runtime,node_gemini_model toneMint
```

## Components

### Mobile client

Expo/React Native app that captures scans and presents accessible results.

- [Expo app](https://github.com/soumyaritu/echorxaimediassistant/blob/main/app/App.js): React Native client · Thin mobile shell for the scan flow and result presentation. · `app/App.js`
- [App entry](https://github.com/soumyaritu/echorxaimediassistant/blob/main/app/index.js): Startup · Launches the mobile application. · `app/index.js`
- [Client config](https://github.com/soumyaritu/echorxaimediassistant/blob/main/app/config.js): Runtime config · Holds the backend endpoint and app runtime settings. · `app/config.js`
- [App manifest](https://github.com/soumyaritu/echorxaimediassistant/blob/main/app/app.json): Expo manifest · Declares the Expo app metadata and build settings. · `app/app.json`
- [Scan flow](https://github.com/soumyaritu/echorxaimediassistant/blob/main/app/App.js): Capture UI · Handles photo capture and the three-screen scan-to-result experience. · `app/App.js`
- [Accessibility + speech](https://github.com/soumyaritu/echorxaimediassistant/blob/main/app/App.js): A11y/UI · Screen-reader friendly presentation and text-to-speech playback for results. · `app/App.js`

### AI backend

FastAPI service and agent pipeline that analyzes medicine photos.

- [FastAPI server](https://github.com/soumyaritu/echorxaimediassistant/blob/main/backend/server.py): HTTP API · Backend entrypoint that receives image requests from the mobile app. · `backend/server.py`
- [Backend config](https://github.com/soumyaritu/echorxaimediassistant/blob/main/backend/config.py): Runtime config · Loads API keys and service settings for the backend pipeline. · `backend/config.py`
- [Root agent](https://github.com/soumyaritu/echorxaimediassistant/blob/main/backend/agents/root.py): Orchestrator · Coordinates the backend analysis chain and assembles the final payload. · `backend/agents/root.py`
- [LabelSense](https://github.com/soumyaritu/echorxaimediassistant/blob/main/backend/agents/labelsense.py): Vision agent · Extracts medicine identity, visible warnings, expiry clues, and confidence. · `backend/agents/labelsense.py`
- [SafetyGuard](https://github.com/soumyaritu/echorxaimediassistant/blob/main/backend/agents/safetyguard.py): Safety agent · Applies dosage, expiry, and non-medicine hazard checks. · `backend/agents/safetyguard.py`
- [VoiceAssist](https://github.com/soumyaritu/echorxaimediassistant/blob/main/backend/agents/voiceassistant.py): Narration agent · Converts structured guidance into short spoken-friendly wording. · `backend/agents/voiceassistant.py`

### External AI

Google ADK and Gemini provide the model-backed reasoning layer.

- **Google ADK**: Agent framework · Agent orchestration framework used by the backend.
- **Gemini 2.5 Flash**: Vision model · External model used for image understanding and reasoning.

## Connections

- App entry → Expo app: boots · Starts the Expo client runtime.
- App manifest → Expo app: defines · Provides app metadata and launch configuration.
- Client config → FastAPI server: backend URL · Configures the LAN endpoint for scan requests.
- Expo app → Scan flow: captures photo · User takes a medicine photo and enters the scan flow.
- Scan flow → FastAPI server: POST scan · Sends the image to the backend for analysis.
- FastAPI server → Root agent: dispatches · Hands the request to the agent pipeline.
- Backend config → FastAPI server: configures · Supplies API keys and service settings.
- Root agent → LabelSense: 1. analyze · Extracts visible medicine details from the image.
- LabelSense → Google ADK: runs on · Uses the agent framework for orchestration.
- LabelSense → Gemini 2.5 Flash: vision reasoning · Delegates image interpretation to Gemini.
- Root agent → SafetyGuard: 2. validate · Checks safety rules and enriches the findings.
- SafetyGuard → Google ADK: runs on · Uses the agent framework for orchestration.
- SafetyGuard → Gemini 2.5 Flash: policy reasoning · Uses the model for safety interpretation when needed.
- Root agent → VoiceAssist: 3. narrate · Rewrites the result for spoken output.
- VoiceAssist → Google ADK: runs on · Uses the agent framework for orchestration.
- FastAPI server → Expo app: JSON response · Returns structured guidance to the mobile app.
- Expo app → Accessibility + speech: renders / speaks · Presents the result visually and through text-to-speech.
