# ajayrajrthd/voice-assistant architecture

> Architecture diagram of the GitHub repository ajayrajrthd/voice-assistant, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ajayrajrthd/voice-assistant
- Repository: https://github.com/ajayrajrthd/voice-assistant

## Overview

##### Project Type and Purpose
- This is a Python-based, full-stack Voice Assistant application (per “Voice-Assistant” and the presence of app.py, templates/index.html, speech_processing and tools modules).
- It provides a web UI (Flask or similar) that captures user speech, converts it to text, feeds it to an AI agent, invokes domain-specific “tools” (calendar, contacts, email, web/knowledge search), then converts the AI’s response back to speech.

##### File Structure Analysis
- Top-level:
- app.py / main.py – entry points (web server and/or CLI launcher)
- requirements.txt, .env – dependencies and configuration
- templates/index.html – web-based UI
- scripts/ – build utilities for your knowledge-base index
- src/ – core application code, subdivided into:
– agents/agent.py – orchestrates prompts and decision-making
– prompts/prompts.py – static or dynamic prompt templates
– speech_processing/ – speech_to_text.py, text_to_speech.py, conversation_manager.py
– tools/ – base_tool.py plus subpackages for calendar, contacts, emails, search
– utils.py – shared helpers
- Patterns & Deployment:
- Modular “tool” plugin architecture for adding new capabilities
- Scripts to build and fetch vector indexes (knowledge base)
- Likely a Flask or FastAPI framework in app.py, using Jinja templates

##### README Insights
- Minimal, but the name and file tree imply a voice-first assistant

##### Main Components
a. Frontend (Web UI)
- Browser or embedded client rendering templates/index.html
- Captures audio input; plays back audio output
b. Backend Application
- app.py / main.py hosting HTTP endpoints or socket handlers
- ConversationManager coordinating STT → Agent → Tools → TTS
c. Speech Processing Layer
- speech_to_text.py (calls external STT API or local model)
- text_to_speech.py (TTS engine)
d. Agent Core
- agent.py uses prompts.py and LLM API (OpenAI, local model)
e. Tool Plugins
- calendar_tool.py, add/fetch_contact_tool.py, emailing_tool.py, search_web_tool.py, knowledge_base_tool.py
- Each implements a standardized interface (base_tool.py)
f. Data/Storage & External Services
- Knowledge-base index (built by scripts/create_index.py, scripts/fetch_index.py)
- External APIs: Calendar provider, Email SMTP/IMAP, Contacts datastore, Web search or vector store
g. Optional Components
- utils.py for logging, config
- requirements.txt / .env for deployment config
- venv for isolated environment

##### Component Interactions (Data Flow)
1. User speaks → Web UI captures audio → sends to Backend
2. Backend → speech_to_text → text transcript
3. Transcript → Agent → decides if a “tool” call is needed
4. If tool call: Agent → Tool plugin → External service or KB → returns structured data
5. Agent assembles final response → conversation_manager
6. conversation_manager → text_to_speech → audio output
7. Audio sent back to UI → played to user

##### Architectural Patterns & Principles
- Modular plugin architecture for tools (open/closed principle)
- Separation of concerns: UI vs. Speech Processing vs. Agent vs. Tools vs. Data
- Single-responsibility modules (each tool handles one domain)
- Scripts decouple index-creation from runtime

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Frontend Layer
    subgraph "Frontend (Web UI)"
        direction TB
        Browser["Browser"]:::ui
        UI["templates/index.html"]:::ui
    end

    %% Backend Core
    subgraph "Backend Application" 
        direction TB
        AP["app.py / main.py"]:::backend
        CM["conversation_manager"]:::backend
        Agent["agent.py"]:::backend
        Prompts["prompts.py"]:::backend
    end

    %% Speech Processing
    subgraph "Speech Processing" 
        direction TB
        STT["speech_to_text.py"]:::backend
        TTS["text_to_speech.py"]:::backend
    end

    %% Plugin Tools
    subgraph "Tool Plugins" 
        direction TB
        BaseTool["base_tool.py"]:::plugin
        CalTool["calendar_tool.py"]:::plugin
        AddContact["add_contact_tool.py"]:::plugin
        FetchContact["fetch_contact_tool.py"]:::plugin
        EmailTool["emailing_tool.py"]:::plugin
        WebSearch["search_web_tool.py"]:::plugin
        KBTool["knowledge_base_tool.py"]:::plugin
    end

    %% External Services & Data
    subgraph "External Services & Data"
        direction TB
        KBStore["Knowledge-Base Store"]:::storage
        CalendarAPI["Calendar API"]:::external
        ContactsDB["Contacts DB"]:::external
        EmailAPI["Email SMTP/IMAP"]:::external
        WebSearchAPI["Web Search API"]:::external
    end

    %% Offline Scripts
    subgraph "Offline Index Scripts"
        direction TB
        CreateIdx["create_index.py"]:::script
        FetchIdx["fetch_index.py"]:::script
    end

    %% Shared Utils and Config
    Utils["utils.py"]:::backend
    Env[".env / requirements.txt"]:::config

    %% Connections: UI to Backend
    Browser -->|"loads"| UI
    UI -->|"captures audio"| AP

    %% Backend flow
    AP -->|"orchestrates"| CM
    CM -->|"invoke"| STT
    STT -->|"text transcript"| CM
    CM -->|"send transcript"| Agent
    Agent -->|"use templates"| Prompts
    Agent -->|"decides tool call"| BaseTool

    %% Plugin calls
    BaseTool --> CalTool
    BaseTool --> AddContact
    BaseTool --> FetchContact
    BaseTool --> EmailTool
    BaseTool --> WebSearch
    BaseTool --> KBTool

    CalTool --> CalendarAPI
    AddContact --> ContactsDB
    FetchContact --> ContactsDB
    EmailTool --> EmailAPI
    WebSearch --> WebSearchAPI
    KBTool --> KBStore

    %% Back to agent
    CalTool -- data --> Agent
    AddContact -- data --> Agent
    FetchContact -- data --> Agent
    EmailTool -- data --> Agent
    WebSearch -- data --> Agent
    KBTool -- data --> Agent

    Agent -->|"response text"| CM
    CM -->|"invoke"| TTS
    TTS -->|"audio output"| Browser

    %% Offline scripts to store
    CreateIdx --> KBStore
    FetchIdx --> KBStore

    %% Utils and Config
    AP --> Utils
    Agent --> Utils
    CM --> Utils
    AP --> Env

    %% Click Events
    click UI "https://github.com/ajayrajrthd/voice-assistant/blob/main/templates/index.html"
    click AP "https://github.com/ajayrajrthd/voice-assistant/blob/main/app.py"
    click AP "https://github.com/ajayrajrthd/voice-assistant/blob/main/main.py"
    click Env "https://github.com/ajayrajrthd/voice-assistant/blob/main/.env"
    click Env "https://github.com/ajayrajrthd/voice-assistant/blob/main/requirements.txt"
    click CreateIdx "https://github.com/ajayrajrthd/voice-assistant/blob/main/scripts/create_index.py"
    click FetchIdx "https://github.com/ajayrajrthd/voice-assistant/blob/main/scripts/fetch_index.py"
    click CM "https://github.com/ajayrajrthd/voice-assistant/blob/main/src/speech_processing/conversation_manager.py"
    click STT "https://github.com/ajayrajrthd/voice-assistant/blob/main/src/speech_processing/speech_to_text.py"
    click TTS "https://github.com/ajayrajrthd/voice-assistant/blob/main/src/speech_processing/text_to_speech.py"
    click Agent "https://github.com/ajayrajrthd/voice-assistant/blob/main/src/agents/agent.py"
    click Prompts "https://github.com/ajayrajrthd/voice-assistant/blob/main/src/prompts/prompts.py"
    click BaseTool "https://github.com/ajayrajrthd/voice-assistant/blob/main/src/tools/base_tool.py"
    click CalTool "https://github.com/ajayrajrthd/voice-assistant/blob/main/src/tools/calendar/calendar_tool.py"
    click AddContact "https://github.com/ajayrajrthd/voice-assistant/blob/main/src/tools/contacts/add_contact_tool.py"
    click FetchContact "https://github.com/ajayrajrthd/voice-assistant/blob/main/src/tools/contacts/fetch_contact_tool.py"
    click EmailTool "https://github.com/ajayrajrthd/voice-assistant/blob/main/src/tools/emails/emailing_tool.py"
    click WebSearch "https://github.com/ajayrajrthd/voice-assistant/blob/main/src/tools/search/search_web_tool.py"
    click KBTool "https://github.com/ajayrajrthd/voice-assistant/blob/main/src/tools/search/knowledge_base_tool.py"
    click Utils "https://github.com/ajayrajrthd/voice-assistant/blob/main/src/utils.py"

    %% Styles
    classDef ui fill:#BBDEFB,stroke:#0D47A1,color:#0D47A1;
    classDef backend fill:#C8E6C9,stroke:#1B5E20,color:#1B5E20;
    classDef plugin fill:#FFE0B2,stroke:#E65100,color:#E65100;
    classDef external fill:#E0E0E0,stroke:#424242,color:#424242;
    classDef storage fill:#CFD8DC,stroke:#37474F,color:#37474F,stroke-dasharray: 5 5;
    classDef script fill:#F3E5F5,stroke:#6A1B9A,color:#6A1B9A;
    classDef config fill:#FFECB3,stroke:#FF6F00,color:#FF6F00;
```
