# tomertir/ai-control-agent architecture

> Architecture diagram of the GitHub repository tomertir/ai-control-agent, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tomertir/ai-control-agent
- Repository: https://github.com/tomertir/ai-control-agent

## Overview

AI Control Agent turns desktop commands into local OS actions. The principal workflow is text or recognized speech → Claude tool use → local tool dispatch → action result; voice mode repeats this loop. The README also advertises gesture input, but the available source excerpts do not show its implementation, so that branch is marked unverified. Components share agent.py because the repository has a single runtime module; edges are limited to relationships shown in the excerpts or README.

Main entry points: [`agent.py`](https://github.com/tomertir/ai-control-agent/blob/main/agent.py) (Tool dispatch)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_inputs["Command input"]
  node_entry["Mode menu<br/>[agent.py]"]
  node_voice["Voice capture<br/>[agent.py]"]
  node_speech["Speech recognition<br/>[agent.py]"]
  node_text["Text command<br/>[agent.py]"]
  node_gesture["Gesture input<br/>[agent.py]"]
end

subgraph group_orchestration["Agent orchestration"]
  node_agent["Agent loop<br/>[agent.py]"]
  node_dispatch["Tool dispatch<br/>[agent.py]"]
end

subgraph group_actions["Desktop actions"]
  node_app["Open application<br/>[agent.py]"]
  node_media["Media control<br/>[agent.py]"]
  node_scroll["Page scrolling<br/>[agent.py]"]
  node_search["Web search<br/>[agent.py]"]
  node_typing["Text entry<br/>[agent.py]"]
  node_screenshot["Screenshot capture<br/>[agent.py]"]
end

subgraph group_integrations["External systems"]
  node_claude["Claude tool use"]
  node_os["Desktop OS"]
  node_browser["Web browser"]
end

node_user(("User"))

node_user -->|"selects mode"| node_entry
node_entry -->|"starts voice mode"| node_voice
node_entry -->|"accepts command"| node_text
node_voice -->|"captures speech"| node_speech
node_speech -->|"supplies text"| node_agent
node_text -->|"supplies command"| node_agent
node_user -.->|"provides gestures"| node_gesture
node_gesture -.->|"supplies intent"| node_agent
node_agent -->|"sends command"| node_claude
node_claude -->|"returns tool choice"| node_agent
node_agent -->|"dispatches tool calls"| node_dispatch
node_dispatch -->|"invokes"| node_app
node_dispatch -->|"invokes"| node_media
node_dispatch -->|"invokes"| node_scroll
node_dispatch -->|"invokes"| node_search
node_dispatch -->|"invokes"| node_typing
node_dispatch -->|"invokes"| node_screenshot
node_app -->|"launches application"| node_os
node_media -->|"sends media keys"| node_os
node_scroll -->|"scrolls window"| node_os
node_search -.->|"opens search"| node_browser
node_typing -->|"types text"| node_os
node_screenshot -->|"captures screen"| node_os

click node_entry "https://github.com/tomertir/ai-control-agent/blob/main/agent.py"
click node_voice "https://github.com/tomertir/ai-control-agent/blob/main/agent.py"
click node_speech "https://github.com/tomertir/ai-control-agent/blob/main/agent.py"
click node_text "https://github.com/tomertir/ai-control-agent/blob/main/agent.py"
click node_gesture "https://github.com/tomertir/ai-control-agent/blob/main/agent.py"
click node_agent "https://github.com/tomertir/ai-control-agent/blob/main/agent.py"
click node_dispatch "https://github.com/tomertir/ai-control-agent/blob/main/agent.py"
click node_app "https://github.com/tomertir/ai-control-agent/blob/main/agent.py"
click node_media "https://github.com/tomertir/ai-control-agent/blob/main/agent.py"
click node_scroll "https://github.com/tomertir/ai-control-agent/blob/main/agent.py"
click node_search "https://github.com/tomertir/ai-control-agent/blob/main/agent.py"
click node_typing "https://github.com/tomertir/ai-control-agent/blob/main/agent.py"
click node_screenshot "https://github.com/tomertir/ai-control-agent/blob/main/agent.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_entry,node_voice,node_speech,node_text,node_gesture,node_user toneBlue
class node_agent,node_dispatch toneAmber
class node_app,node_media,node_scroll,node_search,node_typing,node_screenshot toneMint
class node_claude,node_os,node_browser toneRose
```

## Components

### Command input

- [Mode menu](https://github.com/tomertir/ai-control-agent/blob/main/agent.py): `agent.py`
- [Voice capture](https://github.com/tomertir/ai-control-agent/blob/main/agent.py): `agent.py`
- [Speech recognition](https://github.com/tomertir/ai-control-agent/blob/main/agent.py): `agent.py`
- [Text command](https://github.com/tomertir/ai-control-agent/blob/main/agent.py): `agent.py`
- [Gesture input](https://github.com/tomertir/ai-control-agent/blob/main/agent.py): `agent.py`

### Agent orchestration

- [Agent loop](https://github.com/tomertir/ai-control-agent/blob/main/agent.py): `agent.py`
- [Tool dispatch](https://github.com/tomertir/ai-control-agent/blob/main/agent.py): `agent.py`

### Desktop actions

- [Open application](https://github.com/tomertir/ai-control-agent/blob/main/agent.py): `agent.py`
- [Media control](https://github.com/tomertir/ai-control-agent/blob/main/agent.py): `agent.py`
- [Page scrolling](https://github.com/tomertir/ai-control-agent/blob/main/agent.py): `agent.py`
- [Web search](https://github.com/tomertir/ai-control-agent/blob/main/agent.py): `agent.py`
- [Text entry](https://github.com/tomertir/ai-control-agent/blob/main/agent.py): `agent.py`
- [Screenshot capture](https://github.com/tomertir/ai-control-agent/blob/main/agent.py): `agent.py`

### External systems

- **Claude tool use**
- **Desktop OS**
- **Web browser**

### Other components

- **User**

## Connections

- User → Mode menu: selects mode (evidence: `README.md`)
- Mode menu → Voice capture: starts voice mode (evidence: `agent.py`)
- Mode menu → Text command: accepts command (evidence: `README.md`)
- Voice capture → Speech recognition: captures speech (evidence: `README.md`)
- Speech recognition → Agent loop: supplies text (evidence: `README.md`)
- Text command → Agent loop: supplies command (evidence: `README.md`)
- User → Gesture input: provides gestures
- Gesture input → Agent loop: supplies intent (evidence: `agent.py`)
- Agent loop → Claude tool use: sends command (evidence: `agent.py`)
- Claude tool use → Agent loop: returns tool choice (evidence: `agent.py`)
- Agent loop → Tool dispatch: dispatches tool calls (evidence: `agent.py`)
- Tool dispatch → Open application: invokes (evidence: `agent.py`)
- Tool dispatch → Media control: invokes (evidence: `agent.py`)
- Tool dispatch → Page scrolling: invokes (evidence: `agent.py`)
- Tool dispatch → Web search: invokes (evidence: `agent.py`)
- Tool dispatch → Text entry: invokes (evidence: `agent.py`)
- Tool dispatch → Screenshot capture: invokes (evidence: `agent.py`)
- Open application → Desktop OS: launches application (evidence: `agent.py`)
- Media control → Desktop OS: sends media keys (evidence: `agent.py`)
- Page scrolling → Desktop OS: scrolls window (evidence: `agent.py`)
- Web search → Web browser: opens search
- Text entry → Desktop OS: types text (evidence: `agent.py`)
- Screenshot capture → Desktop OS: captures screen (evidence: `agent.py`)
