# nlouis38/ada_app architecture

> Architecture diagram of the GitHub repository nlouis38/ada_app, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nlouis38/ada_app
- Repository: https://github.com/nlouis38/ada_app

## Overview

##### Project Type and Purpose
- This is a full-stack application with a Python backend built on Flask and SocketIO, and a React frontend. The system is designed as a conversational AI assistant, integrating multiple data sources and services (e.g., Google Gemini for AI, ElevenLabs for TTS, weather APIs, maps/directions, and web search).
- The primary communication mechanism is real-time bidirectional messaging using SocketIO, handling text, speech, and webcam video frames.

##### Core Components
a. Frontend (Client):
– React Application (found under client/ada-online)
– UI Components such as ChatBox, InputArea, AiVisualizer, WebcamFeed, WeatherWidget, MapWidget, CodeExecutionWidget, and SearchResultsWidget.
– Client-side features like the Web Speech API for speech-to-text, capturing video frames via a canvas, and audio playback via the Web Audio API.

b. Backend (Server):
– Flask Server with SocketIO for real-time communication.
– ADA class logic encapsulated in ADA_Online.py that implements the core AI assistant behavior.
– Asynchronous task management with asyncio to process inputs, interact with Google Gemini, and handle streaming tasks (text chunking, audio streaming via ElevenLabs).
– Specific Python functions that interact with external APIs (weather with python_weather, maps with googlemaps, search using googlesearch-python and BeautifulSoup).
– Multiple endpoints and SocketIO events for handling diverse datapaths.

c. External Services:
– Google Gemini API for conversational AI processing.
– ElevenLabs TTS API for generating audio outputs.
– Third-party APIs for weather, maps, and web search.
– Other asynchronous HTTP interactions using aiohttp for fetching and parsing external data.

##### Relationships and Flow
– The frontend sends user inputs (text, transcribed speech, or JPEG-encoded video frames) to the backend via SocketIO events.
– The Flask backend receives these inputs and dispatches them to the ADA core logic. This logic interacts with the Google Gemini API and may trigger function calls for weather, map, or search data.
– As each external API returns data, the backend streams response pieces (text chunks, audio chunks, API-specific data updates) back to the frontend via dedicated SocketIO events.
– The frontend components update their state in real-time to reflect the conversation, processed audio, and any updates to corresponding widgets.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Frontend Layer
    subgraph "Frontend Layer"
        A1["React Frontend"]
        A2["ChatBox Component"]
        A3["InputArea Component"]
        A4["AiVisualizer Component"]
        A5["WebcamFeed Component"]
        A6["WeatherWidget Component"]
        A7["MapWidget Component"]
        A8["CodeExecutionWidget Component"]
        A9["SearchResultsWidget Component"]
    end

    %% Backend Layer
    subgraph "Backend Layer"
        B1["Flask Server (SocketIO)"]
        B2["ADA Core Logic"]
        B3["ADA Live API Endpoints"]
    end

    %% External APIs
    subgraph "External APIs"
        C1["Google Gemini API"]
        C2["ElevenLabs TTS API"]
        C3["Weather API"]
        C4["Google Maps API"]
        C5["Web Search API"]
    end

    %% Frontend to Backend communication
    A3 -->|"send_text_message/send_transcribed_text/send_video_frame"| B1
    B1 -->|"process_input"| B2
    
    %% Backend internal communication
    B1 -->|"live_socket_events"| B3
    B3 -->|"socket_response"| B1

    %% Backend to External APIs (asynchronous calls shown with dashed arrows)
    B2 -->|"AI_processing"| C1
    B2 ---|"TTS_streaming"| C2
    B2 ---|"weather_lookup"| C3
    B2 ---|"maps_directions"| C4
    B2 ---|"search_query"| C5

    %% Backend responses to Frontend components
    B1 -->|"receive_text_chunk"| A2
    B1 -->|"receive_audio_chunk"| A4
    B1 -->|"weather_update"| A6
    B1 -->|"map_update"| A7
    B1 -->|"search_results_update"| A9

    %% Styles
    classDef frontend fill:#AED6F1,stroke:#1F618D,stroke-width:2px;
    classDef backend fill:#F9E79F,stroke:#B9770E,stroke-width:2px;
    classDef external fill:#A9DFBF,stroke:#196F3D,stroke-width:2px;
    class A1,A2,A3,A4,A5,A6,A7,A8,A9 frontend;
    class B1,B2,B3 backend;
    class C1,C2,C3,C4,C5 external;

    %% Click Events
    click A1 "https://github.com/nlouis38/ada_app/tree/main/client/ada-online"
    click A2 "https://github.com/nlouis38/ada_app/blob/main/client/ada-online/src/components/ChatBox.jsx"
    click A3 "https://github.com/nlouis38/ada_app/blob/main/client/ada-online/src/components/InputArea.jsx"
    click A4 "https://github.com/nlouis38/ada_app/blob/main/client/ada-online/src/components/AiVisualizer.jsx"
    click A5 "https://github.com/nlouis38/ada_app/blob/main/client/ada-online/src/components/WebcamFeed.jsx"
    click A6 "https://github.com/nlouis38/ada_app/blob/main/client/ada-online/src/components/WeatherWidget.jsx"
    click A7 "https://github.com/nlouis38/ada_app/blob/main/client/ada-online/src/components/MapWidget.jsx"
    click A8 "https://github.com/nlouis38/ada_app/blob/main/client/ada-online/src/components/CodeExecutionWidget.jsx"
    click A9 "https://github.com/nlouis38/ada_app/blob/main/client/ada-online/src/components/SearchResultsWidget.jsx"
    click B1 "https://github.com/nlouis38/ada_app/blob/main/server/app.py"
    click B2 "https://github.com/nlouis38/ada_app/blob/main/server/ADA_Online.py"
    click B3 "https://github.com/nlouis38/ada_app/blob/main/server/ADA_Live_API.py"
```
