How ayhansh/mac-ai-buddy works
Mac AI Buddy is a laptop-run conversational controller for an ESP32 desk robot. The main flow captures speech, sends it to an AI provider for a response and possible tool requests, executes approved local capabilities, then synthesizes and plays speech. Distinct branches cover local semantic memory and guarded X posting. The README and sampled source support the main wiring; some subsystem details are known only from documented capabilities and the source index.
Main entry points: robot_agent.py Conversation agent · capture.py Turn capture · manager.py Memory retrieval · hardware.py Robot capabilities
Components 22
Runtime orchestration
- Composition root
minibot/cli.py - Configuration
minibot/config.py - Conversation agent
minibot/agent/robot_agent.py - Event bus
minibot/events/bus.py
Conversation and audio
- Microphone input
minibot/audio/mac_mic.py - Turn capture
minibot/audio/capture.py - Voice activity
minibot/audio/vad.py - Audio DSP
minibot/audio/dsp.py - AI interface
minibot/ai/provider.py - Gemini provider
minibot/ai/gemini_provider.py - OpenAI provider
minibot/ai/openai_provider.py - Speech synthesis
minibot/speech/elevenlabs_provider.py
Robot hardware
- Robot capabilities
minibot/robot/hardware.py - ESP32 transport
minibot/robot/esp32_client.py - Robot firmware
ai_mini_bot.ino
Local memory
- Memory retrieval
minibot/memory/manager.py - Local embeddings
minibot/memory/embeddings.py - SQLite memory
minibot/memory/store.py
X account
- Posting policy
minibot/social/account.py - X transport
minibot/social/x_client.py
Other components
- User
- X service
Connections 22
- User → Microphone input: speaks to
README.md - Microphone input → Turn capture: provides audio
minibot/agent/robot_agent.py - Turn capture → Voice activity: uses voice gating
minibot/audio/capture.py - Turn capture → Audio DSP: uses processing
minibot/audio/capture.py - Conversation agent → Turn capture: captures turns
minibot/agent/robot_agent.py - Conversation agent → AI interface: sends audio
README.md - Gemini provider → AI interface: implements
minibot/ai/gemini_provider.py - OpenAI provider → AI interface: implements
minibot/ai/openai_provider.py - Conversation agent → Speech synthesis: synthesizes replies
minibot/agent/robot_agent.py - Conversation agent → Robot capabilities: dispatches tools
minibot/agent/robot_agent.py - Robot capabilities → ESP32 transport: delegates commands
minibot/robot/hardware.py - ESP32 transport → Robot firmware: calls HTTP API
README.md - Speech synthesis → Robot capabilities: plays audio
minibot/speech/elevenlabs_provider.py - Conversation agent → Memory retrieval: reads and writes
minibot/agent/robot_agent.py - Memory retrieval → Local embeddings: embeds queries
minibot/memory/manager.py - Memory retrieval → SQLite memory: retrieves and stores
minibot/memory/manager.py - Conversation agent → Posting policy: requests social actions
minibot/agent/robot_agent.py - Posting policy → X transport: publishes through
minibot/social/account.py - X transport → X service: sends API requests
minibot/social/x_client.py - Conversation agent → Event bus: publishes events
minibot/agent/robot_agent.py - Composition root → Conversation agent: constructs and starts
minibot/cli.py - Composition root → Configuration: loads settings
minibot/cli.py