How realjules/speakwise works
Main Components Identification
• Frontend System (“Jules”): This includes the Telephony Interface that processes incoming calls, an Audio Router, Call Manager, and Admin Dashboard (evidenced by files under src/frontend/dashboard and docs).
• Core Engine (“Floris”): This component handles the natural language processing and speech services with modules for Speech-to-Text (CoreSTT), LLM processing, Text-to-Speech (CoreTTS), as well as the Agent Orchestrator, Session Manager, and an Agent API (see src/core/agent, src/core/llm, and src/core/utils).
• Browser-based AI Agent (“Leonard”): It automates web interactions and includes modules for web navigation, form filling, payment processing, and document capture. Look at the directories under src/browser_agent and some workflows in src/core/agent/workflows.
• External Systems: Include the Irembo Website, WhatsApp API for document delivery, and an underlying Database (from src/database).
Mapping the File Structure to Architectural Components
• The file tree clearly segregates the code into modules (src/browser_agent, src/core, src/frontend, src/integrations) corresponding to the above components.
• Supporting files such as Dockerfile, docker-compose.yml, and GitHub workflows that indicate containerization and CI/CD practices.
• Tests are organized separately, which shows a commitment to maintaining reliability through unit and integration tests.
Defining Relationships and Data Flow
• From the Telephony component, route the call audio to an Audio Router and then to a Call Manager that connects further with the Admin Dashboard (indicating management and real-time analytics).
• The Audio Router also sends audio streams to the Core Engine’s Speech-to-Text module.
• Within the Core Engine, illustrate the flow: text conversion from STT is sent to the LLM Service for intent recognition; responses are processed and sent to a Text-to-Speech module (CoreTTS) that outputs speech back via the Audio Router.
• The Agent Orchestrator and Session Manager (within the Core Engine) manage conversation context and workflow execution; they communicate with the Agent API.
• The Agent API then triggers commands to the Browser-based AI Agent, which interacts with the Irembo website via the Headless Browser component.
• Out internal flows: the Browser Agent consists of separate modules for Navigation, Form Filling, Payment Processing, and Document Capture. For instance, Document Capture sends data to an external WhatsApp API for delivery.
• A bidirectional communication between Session Manager and the Database, as well as the Dashboard’s analytics reading from the Database.
Technological and Architectural Details
• Use clear labels (e.g., “Telephony Interface”, “Audio Router”, “LLM Service”, “Agent Orchestrator”, “Web Navigator”, etc.).
– Data flows from user calls, through telephony, to speech processing in the core, then via commands to browser automation.
– A third distinct style for the browser automation modules.
– The Database should be distinct as a persistent storage component.
• Architectural patterns:
– Emphasize modular separation (three “subsystems” working in parallel).
– Indicate the use of API layers for communication between the core engine and browser agent.
– Capture the real-time analytics loop from the Dashboard to the Database.