# firexrwt/george-droid architecture

> Architecture diagram of the GitHub repository firexrwt/george-droid, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/firexrwt/george-droid
- Repository: https://github.com/firexrwt/george-droid

## Overview

##### Project Type and Purpose
- George-Droid is an open-source, Python-based streaming assistant bot.
- It is not a traditional web “full-stack” app but rather a single coordinated application that:
- Listens to microphone audio (STT)
- Reads/writes Twitch chat events
- Sends prompts to a cloud LLM via Together AI
- Uses a local RAG memory store (FAISS + Sentence-Transformers)
- Generates speech via an external Piper TTS binary
- Provides visual context via screenshots
- Outputs audio, chat messages, and text overlays (OBS)

##### File-Structure Analysis
- Top-level files:
- main.py – orchestration of all modules and event loops
- requirements.txt – libraries (twitchio, faster-whisper, sentence-transformers, faiss, etc.)
- .env – Twitch & Together AI credentials
- Directories & binaries:
- backend/memory_store.py – RAG memory implementation (FAISS index + embeddings)
- piper_tts_bin/ – Piper TTS executable + Espeak-NG data files
- voices/ – ONNX voice models for Piper TTS
- Implicit components: Faster-Whisper model (STT), VAD logic in main.py, TwitchIO event handlers, OBS overlay text file

##### Key README Insights
- STT: Faster-Whisper with VAD
- TTS: Piper TTS external binary, swapping .onnx model+config
- LLM: Together AI API (e.g., llama-4-scout-17b)
- Memory: RAG with FAISS + Sentence-Transformers
- Chat: twitchio
- Visual Context: screenshots fed into LLM

##### Main Architectural Components (to include
a. Input Components
- Microphone → VAD → Faster-Whisper STT
- Twitch Chat Listener (twitchio)
- Screenshot Capture (periodic or on-demand)
b. Processing Components
- Prompt Assembler (main.py logic to merge STT, chat, screenshots, memory)
- Memory Module (backend/memory_store.py):
- FAISS Index (local data_george_memory folder)
- Embedding engine (Sentence-Transformers)
- LLM API Client (Together AI)
c. Output Components
- TTS Module: Piper TTS binary (piper_tts_bin/piper.exe + voices/*.onnx)
- Twitch Chat Bot (sending chat messages)
- OBS Overlay Text Writer (obs_ai_response.txt)
d. External Services & Data Stores
- Together AI API (LLM)
- Local FAISS index (data folder)
- Piper TTS binary + Espeak-NG assets
- ONNX voice models

##### Component Relationships & Data Flow
1. Audio Input → STT → Text → Prompt Assembler
2. Chat Listener → Chat Events → Prompt Assembler
3. Screenshot → Image Data → Prompt Assembler
4. Prompt Assembler queries Memory Module → retrieves context → builds final prompt
5. Final Prompt → Together AI API → LLM Response text
6. LLM Response →
- Memory Module (optionally store new context)
- TTS Module → Audio Output (speaker)
- Chat Bot → Twitch chat
- OBS Overlay → write obs_ai_response.txt
7. Loop back to listening

##### Architectural Patterns & Principles
- Event-driven pipeline: audio, chat, visuals trigger processing events
- Modular separation: Input / Memory / LLM / Output
- RAG pattern: retrieval + generation for contextual responses
- Single process monolith, but logically separated into modules in code

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph "Local System"
        subgraph "Input Components"
            STT["Microphone → VAD → Faster-Whisper STT"]:::input
            ChatListener["Twitch Chat Listener (twitchio)"]:::input
            Screenshot["Screenshot Capture"]:::input
        end
        subgraph "Processing Components"
            Assembler["Prompt Assembler"]:::process
            Memory["RAG Memory Module"]:::process
            LLMClient["LLM API Client"]:::process
        end
        subgraph "Storage Components"
            FAISS["FAISS Index"]:::storage
            Embedding["Embedding Engine (SBERT)"]:::storage
            PiperBin["Piper TTS Binary"]:::storage
            TTSAssets["Espeak-NG Data"]:::storage
            VoiceModels["ONNX Voice Models"]:::storage
        end
        subgraph "Orchestration"
            Main["main.py"]:::orchestration
        end
        subgraph "Configuration & Dependencies"
            Env[".env"]:::config
            Req["requirements.txt"]:::config
        end
    end

    subgraph "Output Components"
        TTSModule["TTS Module"]:::output
        ChatBot["Twitch Chat Bot"]:::output
        OBSWriter["OBS Overlay Writer"]:::output
    end

    subgraph "External Services"
        TogetherAI["Together AI API"]:::external
        TwitchAPI["Twitch API"]:::external
    end

    %% Data Flows
    STT -->|"audio streams"| Assembler
    ChatListener -->|"chat events"| Assembler
    Screenshot -->|"images"| Assembler

    Assembler -->|"retrieval queries"| Memory
    Memory -->|"context"| Assembler

    Assembler -->|"final prompt"| LLMClient
    LLMClient -->|"LLM response"| Assembler

    Assembler -->|"store context"| Memory

    LLMClient -->|"API request"| TogetherAI
    TogetherAI -->|"response text"| LLMClient

    Assembler -->|"text"| ChatBot
    Assembler -->|"text overlay"| OBSWriter
    Assembler -->|"prompt text"| TTSModule

    Memory --> FAISS
    Memory --> Embedding

    TTSModule --> PiperBin
    PiperBin --> TTSAssets
    PiperBin --> VoiceModels

    Main --> STT
    Main --> ChatListener
    Main --> Screenshot
    Main --> Assembler
    Main --> Memory
    Main --> LLMClient
    Main --> TTSModule
    Main --> ChatBot
    Main --> OBSWriter
    Main -.-> Env
    Main -.-> Req

    %% External integrations
    ChatListener -->|connects| TwitchAPI
    ChatBot -->|posts| TwitchAPI

    %% Click Events
    click STT "https://github.com/firexrwt/george-droid/blob/main/main.py"
    click ChatListener "https://github.com/firexrwt/george-droid/blob/main/main.py"
    click Screenshot "https://github.com/firexrwt/george-droid/blob/main/main.py"
    click Assembler "https://github.com/firexrwt/george-droid/blob/main/main.py"
    click Memory "https://github.com/firexrwt/george-droid/blob/main/backend/memory_store.py"
    click LLMClient "https://github.com/firexrwt/george-droid/blob/main/main.py"
    click PiperBin "https://github.com/firexrwt/george-droid/blob/main/piper_tts_bin/piper.exe"
    click TTSAssets "https://github.com/firexrwt/george-droid/tree/main/piper_tts_bin/espeak-ng-data/"
    click VoiceModels "https://github.com/firexrwt/george-droid/blob/main/voices/ru_RU-ruslan-medium.onnx"
    click VoiceModels "https://github.com/firexrwt/george-droid/blob/main/voices/ru_RU-ruslan-medium.onnx.json"
    click Env "https://github.com/firexrwt/george-droid/blob/main/.env"
    click Req "https://github.com/firexrwt/george-droid/blob/main/requirements.txt"
    click Main "https://github.com/firexrwt/george-droid/blob/main/main.py"

    %% Styles
    classDef input fill:#cce5ff,stroke:#66b2ff,stroke-width:1px
    classDef process fill:#d4edda,stroke:#28a745,stroke-width:1px
    classDef storage fill:#fff3cd,stroke:#ffcd39,stroke-width:1px,stroke-dasharray: 5 5
    classDef output fill:#ffeeba,stroke:#ffdd57,stroke-width:1px
    classDef external fill:#e2e3e5,stroke:#6c757d,stroke-width:1px,stroke-dasharray: 2 2
    classDef orchestration fill:#f8d7da,stroke:#dc3545,stroke-width:1px
    classDef config fill:#d1ecf1,stroke:#17a2b8,stroke-width:1px
```
