# asharma2040/chatgpt-home-assistant-bot architecture

> Architecture diagram of the GitHub repository asharma2040/chatgpt-home-assistant-bot, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-02-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/asharma2040/chatgpt-home-assistant-bot
- Repository: https://github.com/asharma2040/chatgpt-home-assistant-bot

## Overview

##### Project type + purpose (from README + tree)
This “project” is primarily an **integration/assembly architecture** rather than a conventional codebase: it documents how to build a **DIY AI robot** running on a **Raspberry Pi** using:
- **Home Assistant Supervised** (Docker-based)
- **OpenAI / ChatGPT API** (LLM reasoning + responses)
- **ElevenLabs API** (text-to-speech voice output)
- A **3.5" SPI TFT display** for a face/visuals
- **USB microphone + USB speaker**
- A **Flask custom expression API** (likely to control robot “expressions” such as blinking/mouth/face assets)

##### What the file structure implies
Since there is no source code in the repo, infer components from the README:
- **Raspberry Pi OS Lite** host
- **Docker** + **Home Assistant Supervised** installed on the Pi
- **NetworkManager**, **os-agent**, etc. (supporting services for supervised HA)
- **SPI display overlay config** (`/boot/firmware/config.txt`) indicating a direct hardware path: Pi → SPI TFT
- **Flask service** mentioned as “custom expression API” (treat as a separate service/container unless you know it’s a HA add-on)

##### Extract key architecture hints from the README
- HA Supervised runs on the Pi and typically orchestrates **HA Core + Supervisor + Add-ons** via Docker.
- The robot uses external cloud services:
- OpenAI API (LLM)
- ElevenLabs API (TTS)
- Hardware I/O is local:
- Mic input → speech capture
- Speaker output → audio playback
- Output → face/expressions
- Flask acts as an **API layer** for expressions (likely called by HA automations/scripts)

##### Physical / Hardware layer
- Raspberry Pi 4
- 3.5" SPI TFT Display (over SPI)
- USB Microphone
- USB Speaker
- (Optional listed) 5V Fan (GPIO or always-on; if controlled, show GPIO)

##### Host OS layer
- Raspberry Pi OS Lite
- Kernel/device overlays (specifically the `dtoverlay=piscreen...` enabling SPI framebuffer)

##### Container / platform layer
- Docker Engine
- Home Assistant Supervised stack:
- HA Supervisor
- HA Core

##### Custom services
- Flask “Expression API” service (container or systemd service)
- Responsible for “blink”, “mouth”, “face state”, “animation commands”
- Outputs to display (framebuffer) or triggers a UI process

##### External cloud APIs
- OpenAI API (ChatGPT)
- ElevenLabs API

##### Client / control surfaces (optional but often real)
- Home Assistant Web UI (from phone/laptop on LAN)
- Any wake-word trigger device (if the mic is direct on Pi, keep it local)

**Voice input path**
- USB Microphone → (local capture component: HA pipeline or custom script) → HA Core automation / conversation agent

**LLM request path**
- HA Core / automation → HTTPS → OpenAI API → response text → HA Core

**TTS path**
- HA Core → HTTPS → ElevenLabs API → audio stream/file → local playback → USB Speaker

**Expression/face path**
- HA Core automation → HTTP (LAN) → Flask Expression API
- Flask Expression API → SPI TFT Display (framebuffer/SPI driver) to render blinking/talking/face changes

**User/admin path**
- Phone/Laptop Browser → HTTP(S) on LAN → Home Assistant Web UI (`:8123`)

##### Architectural patterns/principles
- **Orchestrator pattern**: Home Assistant acts as the central event bus/automation engine.
- **Edge device + cloud AI**: Pi performs local I/O + orchestration; reasoning/voice synthesis are cloud services.
- **Service boundary**: Flask expression control is an internal microservice behind HA automations.
- **Hardware abstraction**: Display enabled via OS overlay; mic/speaker via USB audio devices.

##### Relevant technologies explicitly
- “Home Assistant Supervised (Docker)”
- “HA Core / Supervisor”
- “Flask Expression API”
- “OpenAI API (HTTPS)”
- “ElevenLabs API (HTTPS)”
- “SPI TFT (dtoverlay=piscreen…)”
- “USB Audio (Mic/Speaker)”

- Pi hardware → OS → Docker → HA containers + Flask container

Example sequence:
- Mic captures speech
- HA pipeline triggers
- HA calls OpenAI → gets text
- HA calls ElevenLabs → gets audio
- Audio played on speaker
- HA calls Flask to set “talking/blinking” state during playback

- SPI display wiring/logical connection
- USB devices
- GPIO (fan) if controlled

## Diagram (Mermaid)

```mermaid
flowchart TD

%% =========================
%% Zones / Actors (Vertical)
%% =========================
U["User Device<br/>(Phone/Laptop Browser)"]:::client

subgraph "LAN"
direction TB
LAN_NOTE["Local network access"]:::note
end

subgraph "Raspberry Pi 4 (Robot Edge Device)"
direction TB

    HW_PI["Raspberry Pi 4"]:::hardware

    subgraph "Physical I/O (Peripherals)"
    direction TB
        HW_MIC["USB Microphone<br/>(USB Audio In)"]:::hardware
        HW_SPK["USB Speaker<br/>(USB Audio Out)"]:::hardware
        HW_TFT["3.5&quot; SPI TFT Display<br/>(SPI Framebuffer)"]:::hardware
        HW_FAN["5V Fan (Optional)<br/>(GPIO/Power)"]:::hardware
    end

    subgraph "Host OS Layer"
    direction TB
        OS["Raspberry Pi OS Lite"]:::os
        OVERLAY["Kernel/Device Overlay<br/>(dtoverlay=piscreen...)<br/>/boot/firmware/config.txt"]:::os
        SUPPORT["Support Services<br/>(NetworkManager, os-agent, etc.)"]:::os
    end

    subgraph "Container / Platform Layer"
    direction TB
        DOCKER["Docker Engine"]:::platform

        subgraph "Home Assistant Supervised (Docker)"
        direction TB
            SUP["HA Supervisor"]:::ha
            CORE["HA Core<br/>(Automations + Conversation)"]:::ha
            ADDONS["HA Add-ons<br/>(Optional: STT/TTS/Pipeline)"]:::ha
            DB[("HA Storage/DB<br/>(config + state)")]:::data
        end

        subgraph "Custom Service"
        direction TB
            FLASK["Flask<br/>Custom Expression API<br/>(blink/mouth/face state)"]:::service
        end
    end
end

subgraph "Cloud Services"
direction TB
    OPENAI["OpenAI API<br/>(ChatGPT / LLM)"]:::cloud
    ELEVEN["ElevenLabs API<br/>(Text-to-Speech)"]:::cloud
end

%% =========================
%% Primary Flows
%% =========================
U -->|"HTTP(S) LAN (:8123)"| CORE
LAN_NOTE --- CORE

%% Voice input path
HW_MIC -->|"USB audio frames"| ADDONS
ADDONS -->|"transcript/event"| CORE

%% LLM request path
CORE -->|"HTTPS JSON (prompt)"| OPENAI
OPENAI -->|"HTTPS JSON (response text)"| CORE

%% TTS path
CORE -->|"HTTPS (TTS request)"| ELEVEN
ELEVEN -->|"HTTPS (audio stream/file)"| CORE
CORE -->|"local playback (audio out)"| HW_SPK

%% Expression / face path
CORE -->|"HTTP (expression commands)"| FLASK
FLASK -->|"render/update (framebuffer via SPI driver)"| HW_TFT
OVERLAY -->|"enables SPI display"| HW_TFT

%% Platform dependencies (vertical)
OS -->|"hosts"| DOCKER
SUPPORT -->|"network + supervised requirements"| SUP
DOCKER -->|"runs"| SUP
DOCKER -->|"runs"| CORE
DOCKER -->|"runs"| ADDONS
DOCKER -->|"runs"| FLASK
CORE -->|"read/write state"| DB

%% Hardware anchoring
HW_PI --- OS
HW_PI --- HW_MIC
HW_PI --- HW_SPK
HW_PI --- HW_TFT
HW_PI -.->|"optional control/power"| HW_FAN

%% =========================
%% Click Events (mapped)
%% =========================
click HW_PI "https://github.com/asharma2040/chatgpt-home-assistant-bot/blob/main/README.md"
click OS "https://github.com/asharma2040/chatgpt-home-assistant-bot/blob/main/README.md"
click OVERLAY "https://github.com/asharma2040/chatgpt-home-assistant-bot/blob/main/README.md"
click DOCKER "https://github.com/asharma2040/chatgpt-home-assistant-bot/blob/main/README.md"
click SUP "https://github.com/asharma2040/chatgpt-home-assistant-bot/blob/main/README.md"
click CORE "https://github.com/asharma2040/chatgpt-home-assistant-bot/blob/main/README.md"
click ADDONS "https://github.com/asharma2040/chatgpt-home-assistant-bot/blob/main/README.md"
click FLASK "https://github.com/asharma2040/chatgpt-home-assistant-bot/blob/main/README.md"
click OPENAI "https://github.com/asharma2040/chatgpt-home-assistant-bot/blob/main/README.md"
click ELEVEN "https://github.com/asharma2040/chatgpt-home-assistant-bot/blob/main/README.md"
click HW_TFT "https://github.com/asharma2040/chatgpt-home-assistant-bot/blob/main/README.md"
click HW_MIC "https://github.com/asharma2040/chatgpt-home-assistant-bot/blob/main/README.md"
click HW_SPK "https://github.com/asharma2040/chatgpt-home-assistant-bot/blob/main/README.md"
click HW_FAN "https://github.com/asharma2040/chatgpt-home-assistant-bot/blob/main/README.md"
click U "https://github.com/asharma2040/chatgpt-home-assistant-bot/blob/main/README.md"
click LAN_NOTE "https://github.com/asharma2040/chatgpt-home-assistant-bot/blob/main/README.md"

%% =========================
%% Styling (Color + Shapes)
%% =========================
classDef client fill:#E8F1FF,stroke:#2B5FB8,stroke-width:2px,color:#0B1F44
classDef hardware fill:#FFF2CC,stroke:#B8860B,stroke-width:2px,color:#3A2A00
classDef os fill:#E7FFE7,stroke:#2E7D32,stroke-width:2px,color:#0F2A12
classDef platform fill:#EDE7FF,stroke:#5E35B1,stroke-width:2px,color:#24124D
classDef ha fill:#E1F5FE,stroke:#0277BD,stroke-width:2px,color:#06324B
classDef service fill:#FCE4EC,stroke:#AD1457,stroke-width:2px,color:#4A0B23
classDef cloud fill:#F3E5F5,stroke:#6A1B9A,stroke-width:2px,color:#2E0A42
classDef data fill:#FFF8E1,stroke:#FF8F00,stroke-width:2px,color:#3A2200
classDef note fill:#F5F5F5,stroke:#616161,stroke-width:1.5px,color:#1F1F1F
```
