# bhancockio/adk-voice-agent architecture

> Architecture diagram of the GitHub repository bhancockio/adk-voice-agent, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bhancockio/adk-voice-agent
- Repository: https://github.com/bhancockio/adk-voice-agent

## Overview

##### Project Type and Purpose
- This is a full-stack voice-assistant integration project.
- Frontend: A static single-page interface (HTML/JavaScript) for recording audio, playing back responses, and sending voice data to the backend.
- Backend: A Python service (FastAPI + Uvicorn) that runs a “Jarvis” agent. The agent uses a large-language-model API (Gemini) to parse voice commands and dispatches Google Calendar operations via specialized tool modules.

##### File Structure Analysis
- Top level:
- setup_calendar_auth.py – OAuth 2.0 setup script.
- requirements.txt – Python dependencies.
- README.md – high-level setup and usage.
- app/
- main.py – FastAPI application entrypoint.
- static/ – Frontend assets: index.html and JS modules (audio recorder/player, PCM processors).
- jarvis/agent.py – Orchestrates model calls and tool selection.
- jarvis/tools/ – Calendar-specific operations: calendar_utils, create/edit/delete/list event.
- Credentials and token storage are external (`credentials.json` in root, `~/.credentials/calendar_token.json`).

##### Key Components
a. Frontend (Browser)
- index.html
- JS modules:
- audio-recorder.js + pcm-recorder-processor.js
- audio-player.js + pcm-player-processor.js
- app.js (handles UI and HTTP/WebSocket to backend)
b. Backend (FastAPI Service)
- API Gateway (uvicorn main:app)
- Jarvis Agent (agent.py)
- Tool modules (calendar_utils.py, create/list/edit/delete event)
- OAuth Setup Script (setup_calendar_auth.py) – one-time flow
c. External Services
- Google Calendar REST API
- Gemini (Google AI Studio) LLM API
d. Credentials & Storage
- credentials.json (OAuth client ID)
- ~/.credentials/calendar_token.json (OAuth tokens)

##### Component Relationships and Data Flow
1. User clicks “record” → audio-recorder.js captures PCM audio → sends binary blob to FastAPI endpoint.
2. FastAPI endpoint forwards audio to Gemini API for speech-to-text (or text-to-speech).
3. Agent receives transcript → selects appropriate tool module (e.g., create_event).
4. Tool uses calendar_utils to handle OAuth flow or call Google Calendar API.
5. Google Calendar responds with event data or confirmation.
6. Agent formulates a natural-language response → sends back to frontend.
7. Frontend’s audio-player.js plays the TTS audio or displays the text.

##### Architectural Patterns and Principles
- Separation of concerns: frontend UI vs. backend business logic.
- Modular tool pattern: each calendar operation is a standalone tool under jarvis/tools.
- Externalized authentication: OAuth handshake is managed by setup_calendar_auth.py and calendar_utils.
- Micro-layer for LLM integration: agent.py sits between raw input/output and tool logic.

##### Technologies and Frameworks
- Frontend: Vanilla JS, Web Audio API, PCM processor Web-Audio-Worklet.
- Backend: Python 3.8+, FastAPI, Uvicorn.
- APIs: Google Calendar API (OAuth 2.0), Gemini LLM API.
- Storage: Local filesystem for credentials and tokens.

- Use clear, consistent labels: e.g., “Audio Recorder”, “FastAPI /main.py”, “agent.py”, “Google Calendar API”.
- From Browser → FastAPI (audio/text requests)
- From FastAPI → Gemini API (LLM calls)
- From FastAPI → Google Calendar API (REST calls)
- From APIs back to FastAPI → Browser
- Blue for frontend elements
- Green for backend/application logic
- Orange for external services
- Gray for storage/credentials

##### Final Checklist
- All major components (Frontend, Backend, External APIs, Storage)
- Technology callouts next to each component
- Annotation of OAuth setup process as an initialization flow

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Client (Browser)"
        direction TB
        Index["index.html"]:::frontend
        AppJS["app.js"]:::frontend
        AudioRec["audio-recorder.js"]:::frontend
        PCMRec["pcm-recorder-processor.js"]:::frontend
        AudioPlay["audio-player.js"]:::frontend
        PCMPlay["pcm-player-processor.js"]:::frontend
    end

    subgraph "Server (Python FastAPI)"
        direction TB
        Gateway["FastAPI / main.py"]:::backend
        Agent["agent.py"]:::backend
        subgraph "Tools (jarvis/tools)"
            direction TB
            CalUtils["calendar_utils.py"]:::backend
            CreateEvt["create_event.py"]:::backend
            DelEvt["delete_event.py"]:::backend
            EditEvt["edit_event.py"]:::backend
            ListEvt["list_events.py"]:::backend
        end
        Setup["setup_calendar_auth.py"]:::storage
        Req["requirements.txt"]:::storage
        Readme["README.md"]:::storage
    end

    subgraph "External Services"
        direction TB
        Gemini["Gemini LLM API"]:::external
        GCal["Google Calendar API"]:::external
    end

    subgraph "Credentials & Storage"
        direction TB
        Creds["credentials.json"]:::storage
        Token["~/.credentials/calendar_token.json"]:::storage
    end

    %% Data Flow
    Index -->|"load UI"| AppJS
    AppJS -->|"invoke record"| AudioRec
    AudioRec -->|"process PCM"| PCMRec
    PCMRec -->|"Audio Blob"| Gateway
    Gateway -->|"forward audio"| Agent
    Agent -->|"LLM call"| Gemini
    Gemini -->|"Text Transcript"| Agent
    Agent -->|"select tool"| CreateEvt
    Agent -->|"select tool"| DelEvt
    Agent -->|"select tool"| EditEvt
    Agent -->|"select tool"| ListEvt
    CreateEvt -->|"utils call"| CalUtils
    DelEvt -->|"utils call"| CalUtils
    EditEvt -->|"utils call"| CalUtils
    ListEvt -->|"utils call"| CalUtils
    CalUtils -->|"OAuth/REST"| GCal
    GCal -->|"API Response"| CalUtils
    CalUtils -->|"tool response"| Agent
    Agent -->|"Response Text/TTS"| Gateway
    Gateway -->|"playback data"| PCMPlay
    PCMPlay -->|"PCM Audio"| AudioPlay
    AudioPlay -->|"play audio"| AppJS

    %% OAuth Setup Flow
    Setup --> Creds
    CalUtils --> Token

    %% Styling and Click Events
    click Index "https://github.com/bhancockio/adk-voice-agent/blob/main/app/static/index.html"
    click AppJS "https://github.com/bhancockio/adk-voice-agent/blob/main/app/static/js/app.js"
    click AudioRec "https://github.com/bhancockio/adk-voice-agent/blob/main/app/static/js/audio-recorder.js"
    click PCMRec "https://github.com/bhancockio/adk-voice-agent/blob/main/app/static/js/pcm-recorder-processor.js"
    click AudioPlay "https://github.com/bhancockio/adk-voice-agent/blob/main/app/static/js/audio-player.js"
    click PCMPlay "https://github.com/bhancockio/adk-voice-agent/blob/main/app/static/js/pcm-player-processor.js"
    click Gateway "https://github.com/bhancockio/adk-voice-agent/blob/main/app/main.py"
    click Agent "https://github.com/bhancockio/adk-voice-agent/blob/main/app/jarvis/agent.py"
    click CalUtils "https://github.com/bhancockio/adk-voice-agent/blob/main/app/jarvis/tools/calendar_utils.py"
    click CreateEvt "https://github.com/bhancockio/adk-voice-agent/blob/main/app/jarvis/tools/create_event.py"
    click DelEvt "https://github.com/bhancockio/adk-voice-agent/blob/main/app/jarvis/tools/delete_event.py"
    click EditEvt "https://github.com/bhancockio/adk-voice-agent/blob/main/app/jarvis/tools/edit_event.py"
    click ListEvt "https://github.com/bhancockio/adk-voice-agent/blob/main/app/jarvis/tools/list_events.py"
    click Setup "https://github.com/bhancockio/adk-voice-agent/blob/main/setup_calendar_auth.py"
    click Req "https://github.com/bhancockio/adk-voice-agent/blob/main/requirements.txt"
    click Readme "https://github.com/bhancockio/adk-voice-agent/blob/main/README.md"

    classDef frontend fill:#cce5ff,stroke:#3399ff,color:#000033;
    classDef backend fill:#d4edda,stroke:#28a745,color:#155724;
    classDef external fill:#ffe5b4,stroke:#ff9933,color:#663300;
    classDef storage fill:#e2e3e5,stroke:#6c757d,color:#343a40;
```
