# ahmadhassan-bted/b architecture

> Architecture diagram of the GitHub repository ahmadhassan-bted/b, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-08-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ahmadhassan-bted/b
- Repository: https://github.com/ahmadhassan-bted/b

## Overview

#### Purpose and Entry Point
B is a Python/PyQt6 desktop companion that observes desktop context, accepts typed or spoken input, runs LLM reasoning, and renders speech, text, emotion, and movement in a transparent overlay. Startup is `main.py`, which composes modules, registers global hotkeys, and drives the 60 FPS application tick.

#### Event-Driven Core
`core/bus.py` is the architectural boundary between subsystems: modules publish and subscribe through a thread-safe Qt signal-based EventBus rather than calling one another directly. `main.py` owns process lifecycle and bus wiring. Core control flow is perception/input → bus → cognition/state → bus → visual/audio/motion outputs.

#### Perception and Context Acquisition
Active application changes originate from `sensors/window_tracker.py`. `vision/semantic.py` extracts structured UI context via UI Automation and quality/cooldown logic. When semantic extraction is unavailable, `vision/mss_capture.py` captures the screen and uses OCR as a fallback. These stages emit context events rather than exposing platform-specific APIs to the brain.

#### User Input and Work Modes
`ui/input_box.py` provides typed interaction; `audio/ears.py` provides microphone transcription using Faster-Whisper with voice activity detection. `brain/work_mode.py` defines the focused/proactive work-mode behavior activated by global hotkey. Input is normalized into bus events consumed by cognition, keeping UI and audio capture independent from reasoning.

#### Cognition, State, and Autonomy
`brain/llm.py` is the CognitiveEngine: it combines context/history, invokes either Groq cloud inference or local llama-cpp, and parses streamed output. `brain/context.py` manages contextual state, while `brain/soul.py` owns emotional/conversation state and sentence-ready stream buffering. `brain/autonomy_loop.py` schedules unsolicited thoughts only when timing and context quality permit. Cognition publishes speech, expression, and movement requests; it does not directly render or play them.

#### Expression and Overlay Runtime
`ui/overlay.py` hosts the transparent, click-through desktop presentation. `ui/face.py` renders the animated face, `ui/chat.py` displays spoken text, and `physics/kinematics.py` translates movement requests into eased/Bezier motion. These are Qt-driven, 60 FPS output consumers coordinated through events and state transitions.

#### Audio and External Dependencies
`audio/speaker.py` turns generated sentences into Piper ONNX speech and applies Pedalboard DSP effects; packaged voice assets are `voices/b_voice.onnx` and `voices/b_voice.onnx.json`. External services/runtime boundaries include Groq API credentials from `.env`, optional local GGUF/llama-cpp inference, Windows UI Automation/Win32 hooks, Tesseract OCR, MSS screen capture, Faster-Whisper, and PyQt6. Primary target is Windows 10/11, with experimental Linux/macOS support.

Main entry points: [`core/bus.py`](https://github.com/ahmadhassan-bted/b/blob/main/core/bus.py) (Event bus), [`brain/llm.py`](https://github.com/ahmadhassan-bted/b/blob/main/brain/llm.py) (Cognitive engine), [`ui/overlay.py`](https://github.com/ahmadhassan-bted/b/blob/main/ui/overlay.py) (Transparent overlay), [`vision/semantic.py`](https://github.com/ahmadhassan-bted/b/blob/main/vision/semantic.py) (Semantic UI extraction)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Application Runtime"]
  node_main["Application entry<br/>Python bootstrap<br/>[main.py]"]
  node_bus["Event bus<br/>Qt signal bus<br/>[bus.py]"]
end

subgraph group_perception["Perception &amp; Input"]
  node_window_tracker["Window tracker<br/>desktop sensor<br/>[window_tracker.py]"]
  node_semantic["Semantic UI extraction<br/>UI Automation context<br/>[semantic.py]"]
  node_screen_capture["OCR fallback capture<br/>screen context<br/>[mss_capture.py]"]
  node_input_box["Typed input<br/>Qt input UI<br/>[input_box.py]"]
  node_ears["Voice transcription<br/>microphone input<br/>[ears.py]"]
end

subgraph group_cognition["Cognition &amp; State"]
  node_work_mode["Work mode<br/>interaction policy<br/>[work_mode.py]"]
  node_context[("Context state<br/>state manager<br/>[context.py]")]
  node_soul[("Emotion &amp; conversation state<br/>state manager<br/>[soul.py]")]
  node_llm{{"Cognitive engine<br/>LLM orchestration<br/>[llm.py]"}}
  node_autonomy["Autonomy loop<br/>thought scheduler<br/>[autonomy_loop.py]"]
  node_inference{{"LLM providers<br/>Groq / local llama-cpp"}}
end

subgraph group_output["Overlay Outputs"]
  node_overlay["Transparent overlay<br/>Qt presentation host<br/>[overlay.py]"]
  node_face["Animated face<br/>visual renderer<br/>[face.py]"]
  node_chat["Speech text display<br/>text renderer<br/>[chat.py]"]
  node_kinematics["Motion engine<br/>movement renderer<br/>[kinematics.py]"]
  node_speaker["Speech synthesis<br/>Piper audio output<br/>[speaker.py]"]
  node_voice_assets["Packaged voice assets<br/>ONNX voice model<br/>[b_voice.onnx]"]
end

node_main -->|"wires lifecycle"| node_bus
node_main -->|"drives 60 FPS tick"| node_overlay
node_window_tracker -->|"active app changes"| node_semantic
node_semantic -.->|"fallback when unavailable"| node_screen_capture
node_semantic -->|"structured context"| node_bus
node_screen_capture -->|"OCR context"| node_bus
node_input_box -->|"typed input"| node_bus
node_ears -->|"transcribed input"| node_bus
node_bus -->|"hotkey and input events"| node_work_mode
node_bus -->|"context events"| node_context
node_bus -->|"normalized input"| node_llm
node_context -->|"current context"| node_llm
node_soul -->|"conversation and emotion"| node_llm
node_autonomy -->|"proactive prompts"| node_llm
node_llm -->|"streaming inference"| node_inference
node_llm -->|"speech, expression, movement"| node_bus
node_bus -->|"stream and state events"| node_soul
node_bus -->|"presentation events"| node_overlay
node_overlay -->|"renders expression"| node_face
node_overlay -->|"renders spoken text"| node_chat
node_bus -->|"movement requests"| node_kinematics
node_bus -->|"sentence-ready speech"| node_speaker
node_speaker -->|"loads voice model"| node_voice_assets

click node_main "https://github.com/ahmadhassan-bted/b/blob/main/main.py"
click node_bus "https://github.com/ahmadhassan-bted/b/blob/main/core/bus.py"
click node_window_tracker "https://github.com/ahmadhassan-bted/b/blob/main/sensors/window_tracker.py"
click node_semantic "https://github.com/ahmadhassan-bted/b/blob/main/vision/semantic.py"
click node_screen_capture "https://github.com/ahmadhassan-bted/b/blob/main/vision/mss_capture.py"
click node_input_box "https://github.com/ahmadhassan-bted/b/blob/main/ui/input_box.py"
click node_ears "https://github.com/ahmadhassan-bted/b/blob/main/audio/ears.py"
click node_work_mode "https://github.com/ahmadhassan-bted/b/blob/main/brain/work_mode.py"
click node_context "https://github.com/ahmadhassan-bted/b/blob/main/brain/context.py"
click node_soul "https://github.com/ahmadhassan-bted/b/blob/main/brain/soul.py"
click node_llm "https://github.com/ahmadhassan-bted/b/blob/main/brain/llm.py"
click node_autonomy "https://github.com/ahmadhassan-bted/b/blob/main/brain/autonomy_loop.py"
click node_overlay "https://github.com/ahmadhassan-bted/b/blob/main/ui/overlay.py"
click node_face "https://github.com/ahmadhassan-bted/b/blob/main/ui/face.py"
click node_chat "https://github.com/ahmadhassan-bted/b/blob/main/ui/chat.py"
click node_kinematics "https://github.com/ahmadhassan-bted/b/blob/main/physics/kinematics.py"
click node_speaker "https://github.com/ahmadhassan-bted/b/blob/main/audio/speaker.py"
click node_voice_assets "https://github.com/ahmadhassan-bted/b/blob/main/voices/b_voice.onnx"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_main,node_bus toneBlue
class node_window_tracker,node_semantic,node_screen_capture,node_input_box,node_ears toneAmber
class node_work_mode,node_context,node_soul,node_llm,node_autonomy,node_inference toneMint
class node_overlay,node_face,node_chat,node_kinematics,node_speaker,node_voice_assets toneRose
```

## Components

### Application Runtime

- [Application entry](https://github.com/ahmadhassan-bted/b/blob/main/main.py): Python bootstrap · Composes modules, registers hotkeys, and drives the 60 FPS tick. · `main.py`
- [Event bus](https://github.com/ahmadhassan-bted/b/blob/main/core/bus.py): Qt signal bus · Thread-safe boundary between subsystems. · `core/bus.py`

### Perception & Input

- [Window tracker](https://github.com/ahmadhassan-bted/b/blob/main/sensors/window_tracker.py): desktop sensor · `sensors/window_tracker.py`
- [Semantic UI extraction](https://github.com/ahmadhassan-bted/b/blob/main/vision/semantic.py): UI Automation context · `vision/semantic.py`
- [OCR fallback capture](https://github.com/ahmadhassan-bted/b/blob/main/vision/mss_capture.py): screen context · `vision/mss_capture.py`
- [Typed input](https://github.com/ahmadhassan-bted/b/blob/main/ui/input_box.py): Qt input UI · `ui/input_box.py`
- [Voice transcription](https://github.com/ahmadhassan-bted/b/blob/main/audio/ears.py): microphone input · Faster-Whisper transcription with voice activity detection. · `audio/ears.py`

### Cognition & State

- [Work mode](https://github.com/ahmadhassan-bted/b/blob/main/brain/work_mode.py): interaction policy · Focused and proactive behavior activated by global hotkey. · `brain/work_mode.py`
- [Context state](https://github.com/ahmadhassan-bted/b/blob/main/brain/context.py): state manager · `brain/context.py`
- [Emotion & conversation state](https://github.com/ahmadhassan-bted/b/blob/main/brain/soul.py): state manager · Buffers sentence-ready streamed output. · `brain/soul.py`
- [Cognitive engine](https://github.com/ahmadhassan-bted/b/blob/main/brain/llm.py): LLM orchestration · Combines context and history, streams inference, and emits output requests. · `brain/llm.py`
- [Autonomy loop](https://github.com/ahmadhassan-bted/b/blob/main/brain/autonomy_loop.py): thought scheduler · Schedules unsolicited thoughts when timing and context quality allow. · `brain/autonomy_loop.py`
- **LLM providers**: Groq / local llama-cpp · Cloud Groq inference or optional local GGUF runtime.

### Overlay Outputs

- [Transparent overlay](https://github.com/ahmadhassan-bted/b/blob/main/ui/overlay.py): Qt presentation host · Click-through desktop overlay runtime. · `ui/overlay.py`
- [Animated face](https://github.com/ahmadhassan-bted/b/blob/main/ui/face.py): visual renderer · `ui/face.py`
- [Speech text display](https://github.com/ahmadhassan-bted/b/blob/main/ui/chat.py): text renderer · `ui/chat.py`
- [Motion engine](https://github.com/ahmadhassan-bted/b/blob/main/physics/kinematics.py): movement renderer · Applies eased and Bezier motion. · `physics/kinematics.py`
- [Speech synthesis](https://github.com/ahmadhassan-bted/b/blob/main/audio/speaker.py): Piper audio output · Piper ONNX voice synthesis with Pedalboard DSP. · `audio/speaker.py`
- [Packaged voice assets](https://github.com/ahmadhassan-bted/b/blob/main/voices/b_voice.onnx): ONNX voice model · `voices/b_voice.onnx`

## Connections

- Application entry → Event bus: wires lifecycle
- Application entry → Transparent overlay: drives 60 FPS tick
- Window tracker → Semantic UI extraction: active app changes
- Semantic UI extraction → OCR fallback capture: fallback when unavailable
- Semantic UI extraction → Event bus: structured context
- OCR fallback capture → Event bus: OCR context
- Typed input → Event bus: typed input
- Voice transcription → Event bus: transcribed input
- Event bus → Work mode: hotkey and input events
- Event bus → Context state: context events
- Event bus → Cognitive engine: normalized input
- Context state → Cognitive engine: current context
- Emotion & conversation state → Cognitive engine: conversation and emotion
- Autonomy loop → Cognitive engine: proactive prompts
- Cognitive engine → LLM providers: streaming inference
- Cognitive engine → Event bus: speech, expression, movement
- Event bus → Emotion & conversation state: stream and state events
- Event bus → Transparent overlay: presentation events
- Transparent overlay → Animated face: renders expression
- Transparent overlay → Speech text display: renders spoken text
- Event bus → Motion engine: movement requests
- Event bus → Speech synthesis: sentence-ready speech
- Speech synthesis → Packaged voice assets: loads voice model