# realjules/speakwise architecture

> Architecture diagram of the GitHub repository realjules/speakwise, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/realjules/speakwise
- Repository: https://github.com/realjules/speakwise

## Overview

##### Main Components Identification
- Frontend System (“Jules”): This includes the Telephony Interface that processes incoming calls, an Audio Router, Call Manager, and Admin Dashboard (evidenced by files under src/frontend/dashboard and docs).
- Core Engine (“Floris”): This component handles the natural language processing and speech services with modules for Speech-to-Text (CoreSTT), LLM processing, Text-to-Speech (CoreTTS), as well as the Agent Orchestrator, Session Manager, and an Agent API (see src/core/agent, src/core/llm, and src/core/utils).
- Browser-based AI Agent (“Leonard”): It automates web interactions and includes modules for web navigation, form filling, payment processing, and document capture. Look at the directories under src/browser_agent and some workflows in src/core/agent/workflows.
- External Systems: Include the Irembo Website, WhatsApp API for document delivery, and an underlying Database (from src/database).

##### Mapping the File Structure to Architectural Components
- The file tree clearly segregates the code into modules (src/browser_agent, src/core, src/frontend, src/integrations) corresponding to the above components.
- Supporting files such as Dockerfile, docker-compose.yml, and GitHub workflows that indicate containerization and CI/CD practices.
- Tests are organized separately, which shows a commitment to maintaining reliability through unit and integration tests.

##### Defining Relationships and Data Flow
- From the Telephony component, route the call audio to an Audio Router and then to a Call Manager that connects further with the Admin Dashboard (indicating management and real-time analytics).
- The Audio Router also sends audio streams to the Core Engine’s Speech-to-Text module.
- Within the Core Engine, illustrate the flow: text conversion from STT is sent to the LLM Service for intent recognition; responses are processed and sent to a Text-to-Speech module (CoreTTS) that outputs speech back via the Audio Router.
- The Agent Orchestrator and Session Manager (within the Core Engine) manage conversation context and workflow execution; they communicate with the Agent API.
- The Agent API then triggers commands to the Browser-based AI Agent, which interacts with the Irembo website via the Headless Browser component.
- Out internal flows: the Browser Agent consists of separate modules for Navigation, Form Filling, Payment Processing, and Document Capture. For instance, Document Capture sends data to an external WhatsApp API for delivery.
- A bidirectional communication between Session Manager and the Database, as well as the Dashboard’s analytics reading from the Database.

##### Technological and Architectural Details
- Use clear labels (e.g., “Telephony Interface”, “Audio Router”, “LLM Service”, “Agent Orchestrator”, “Web Navigator”, etc.).
– Data flows from user calls, through telephony, to speech processing in the core, then via commands to browser automation.
– A third distinct style for the browser automation modules.
– The Database should be distinct as a persistent storage component.
- Architectural patterns:
– Emphasize modular separation (three “subsystems” working in parallel).
– Indicate the use of API layers for communication between the core engine and browser agent.
– Capture the real-time analytics loop from the Dashboard to the Database.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Global User Node
    User["User with Phone"]:::external

    %% Frontend System – Jules
    subgraph "Frontend System – Jules"
        TI["Telephony Interface"]:::frontend
        AR["Audio Router"]:::frontend
        CM["Call Manager"]:::frontend
        AD["Admin Dashboard"]:::frontend
        DOC["Documentation"]:::frontend
    end

    %% Core Engine – Floris
    subgraph "Core Engine – Floris"
        STT["Speech-to-Text"]:::core
        LLM["LLM Service"]:::core
        TTS["Text-to-Speech"]:::core
        AO["Agent Orchestrator"]:::core
        SM["Session Manager"]:::core
        API["Agent API"]:::core
    end

    %% Browser-based AI Agent – Leonard
    subgraph "Browser-based AI Agent – Leonard"
        WN["Web Navigator"]:::browser
        FF["Form Filler"]:::browser
        PP["Payment Processing"]:::browser
        DC["Document Capture"]:::browser
    end

    %% External Systems
    TW["Telephony Provider"]:::external
    WA["WhatsApp API"]:::external
    IW["Irembo Website"]:::external
    DB["Persistent Database"]:::database

    %% Connections from User to Frontend
    User -->|"initiates call"| TI

    %% Frontend internal flow
    TI -->|"routes call"| AR
    AR -->|"manages call"| CM
    CM -->|"provides analytics"| AD
    AD -->|"references"| DOC
    TI -->|"forwards call info"| TW

    %% Audio Routing from Frontend to Core
    AR -->|"audio stream"| STT

    %% Core Engine Processing Flow
    STT -->|"transcribes audio"| LLM
    LLM -->|"analyzes intent"| AO
    LLM -->|"generates response"| TTS
    TTS -->|"outputs speech"| AR
    AO -->|"issues commands"| API

    %% Session Management and Database Interaction
    SM <-->|"synchronizes session"| DB
    AD <-->|"reads analytics"| DB

    %% Agent API triggers Browser-based AI Agent
    API -->|"executes workflow"| WN

    %% Browser Agent internal modules flow
    WN -->|"navigates to"| IW
    DC -->|"sends document"| WA

    %% Additional internal flows in Browser Agent (demonstrative)
    WN -->|"initiates form filling"| FF
    WN -->|"triggers payment"| PP

    %% Connect Session Manager with Agent API and Orchestrator
    AO -->|"updates context"| SM

    %% Click Events
    click AD "https://github.com/realjules/speakwise/tree/main/src/frontend/dashboard"
    click DOC "https://github.com/realjules/speakwise/blob/main/docs/architecture.md"
    click LLM "https://github.com/realjules/speakwise/tree/main/src/core/llm"
    click AO "https://github.com/realjules/speakwise/tree/main/src/core/agent"
    click SM "https://github.com/realjules/speakwise/tree/main/src/core/agent"
    click API "https://github.com/realjules/speakwise/tree/main/src/core/agent"
    click WN "https://github.com/realjules/speakwise/tree/main/src/browser_agent"
    click TW "https://github.com/realjules/speakwise/tree/main/src/integrations/telephony"
    click WA "https://github.com/realjules/speakwise/tree/main/src/integrations/messaging"
    click IW "https://github.com/realjules/speakwise/tree/main/src/browser_agent/services"
    click DB "https://github.com/realjules/speakwise/tree/main/src/database"

    %% Styles
    classDef frontend fill:#CFE2F3,stroke:#0B5394,stroke-width:2px;
    classDef core fill:#F9CB9C,stroke:#B45F06,stroke-width:2px;
    classDef browser fill:#D9EAD3,stroke:#38761D,stroke-width:2px;
    classDef external fill:#E6E6E6,stroke:#999,stroke-width:2px;
    classDef database fill:#FFD966,stroke:#BF9000,stroke-width:2px;
```
