How asharma2040/chatgpt-home-assistant-bot works
Project type + purpose (from README + tree)
This “project” is primarily an integration/assembly architecture rather than a conventional codebase: it documents how to build a DIY AI robot running on a Raspberry Pi using:
• Home Assistant Supervised (Docker-based)
• OpenAI / ChatGPT API (LLM reasoning + responses)
• ElevenLabs API (text-to-speech voice output)
• A 3.5" SPI TFT display for a face/visuals
• USB microphone + USB speaker
• A Flask custom expression API (likely to control robot “expressions” such as blinking/mouth/face assets)
What the file structure implies
Since there is no source code in the repo, infer components from the README:
• Raspberry Pi OS Lite host
• Docker + Home Assistant Supervised installed on the Pi
• NetworkManager, os-agent, etc. (supporting services for supervised HA)
• SPI display overlay config (/boot/firmware/config.txt) indicating a direct hardware path: Pi → SPI TFT
• Flask service mentioned as “custom expression API” (treat as a separate service/container unless you know it’s a HA add-on)
Extract key architecture hints from the README
• HA Supervised runs on the Pi and typically orchestrates HA Core + Supervisor + Add-ons via Docker.
• The robot uses external cloud services:
• OpenAI API (LLM)
• ElevenLabs API (TTS)
• Hardware I/O is local:
• Mic input → speech capture
• Speaker output → audio playback
• Output → face/expressions
• Flask acts as an API layer for expressions (likely called by HA automations/scripts)
Physical / Hardware layer
• Raspberry Pi 4
• 3.5" SPI TFT Display (over SPI)
• USB Microphone
• USB Speaker
• (Optional listed) 5V Fan (GPIO or always-on; if controlled, show GPIO)
Host OS layer
• Raspberry Pi OS Lite
• Kernel/device overlays (specifically the dtoverlay=piscreen... enabling SPI framebuffer)
Container / platform layer
• Docker Engine
• Home Assistant Supervised stack:
• HA Supervisor
• HA Core
Custom services
• Flask “Expression API” service (container or systemd service)
• Responsible for “blink”, “mouth”, “face state”, “animation commands”
• Outputs to display (framebuffer) or triggers a UI process
External cloud APIs
• OpenAI API (ChatGPT)
• ElevenLabs API
Client / control surfaces (optional but often real)
• Home Assistant Web UI (from phone/laptop on LAN)
• Any wake-word trigger device (if the mic is direct on Pi, keep it local)
Voice input path
• USB Microphone → (local capture component: HA pipeline or custom script) → HA Core automation / conversation agent
LLM request path
• HA Core / automation → HTTPS → OpenAI API → response text → HA Core
TTS path
• HA Core → HTTPS → ElevenLabs API → audio stream/file → local playback → USB Speaker
Expression/face path
• HA Core automation → HTTP (LAN) → Flask Expression API
• Flask Expression API → SPI TFT Display (framebuffer/SPI driver) to render blinking/talking/face changes
User/admin path
• Phone/Laptop Browser → HTTP(S) on LAN → Home Assistant Web UI (:8123)
Architectural patterns/principles
• Orchestrator pattern: Home Assistant acts as the central event bus/automation engine.
• Edge device + cloud AI: Pi performs local I/O + orchestration; reasoning/voice synthesis are cloud services.
• Service boundary: Flask expression control is an internal microservice behind HA automations.
• Hardware abstraction: Display enabled via OS overlay; mic/speaker via USB audio devices.
Relevant technologies explicitly
• “Home Assistant Supervised (Docker)”
• “HA Core / Supervisor”
• “Flask Expression API”
• “OpenAI API (HTTPS)”
• “ElevenLabs API (HTTPS)”
• “SPI TFT (dtoverlay=piscreen…)”
• “USB Audio (Mic/Speaker)”
• Pi hardware → OS → Docker → HA containers + Flask container
Example sequence:
• Mic captures speech
• HA pipeline triggers
• HA calls OpenAI → gets text
• HA calls ElevenLabs → gets audio
• Audio played on speaker
• HA calls Flask to set “talking/blinking” state during playback
• SPI display wiring/logical connection
• USB devices
• GPIO (fan) if controlled