# greywarden07/touchbot architecture

> Architecture diagram of the GitHub repository greywarden07/touchbot, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-23). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/greywarden07/touchbot
- Repository: https://github.com/greywarden07/touchbot

## Overview

##### Project type and purpose
- TouchBot is an agent-based automation framework (an open-source tool) for Android UI interaction and documentation.
- It supports two modes: human demonstration (records user taps/etc.) and autonomous exploration (agent explores automatically).
- It uses visual (screenshots) and XML UI dumps, feeds them to an LLM, then generates step-by-step documentation.

##### File structure
- Top-level scripts:
- learn.py & run.py: CLI entry points
- config.yaml: holds API keys & behavior settings
- requirements.txt: Python dependencies
- scripts/ directory: modular components
- and_controller.py: ADB/UI controller interface
- task_executor.py: executes UI actions (tap, swipe, etc.)
- step_recorder.py: captures human actions
- self_explorer.py: orchestrates autonomous exploration
- prompts.py: prompt templates
- model.py: LLM invocation layer
- document_generation.py: post-processing & doc creation
- utils.py & config.py: shared utilities & config loader

##### README insights
- Describes features, usage, modes, and where configuration lives.
- Mentions config.yaml for API keys (we will use Gemini API key).

##### Main components & interactions
a. Components
- CLI Interface (learn.py / run.py)
- Config Loader (scripts/config.py reading config.yaml)
- Mode Orchestrator
- HumanDemo Module (scripts/step_recorder.py)
- AutoExplore Module (scripts/self_explorer.py)
- UI Controller (scripts/and_controller.py)
- Task Executor (scripts/task_executor.py)
- Screenshot+XML Grabber (via and_controller.py)
- Prompt Templates (scripts/prompts.py)
- LLM Interface (scripts/model.py → Gemini API)
- Doc Generator (scripts/document_generation.py)
- Artifacts Store (tasks/ directory)
- External Service: Gemini API endpoint
b. Relationships
- CLI → reads config → invokes chosen mode
- Mode orchestrator → calls UI Controller & Task Executor → captures UI state
- Captured data → passed to Prompt Templates → packaged into LLM request
- Model layer → sends to Gemini API → receives JSON/text response
- Response → fed into Doc Generator → writes markdown/html into tasks/
c. Architectural patterns
- Modular pipeline: clear separation of data capture, reasoning, and output
- Adapter for external service (model.py abstracts Gemini calls)
- Strategy pattern for modes (HumanDemo vs AutoExplore)
d. Key technologies
- Python 3.8+
- ADB/UI Automator (via and_controller.py)
- Gemini LLM (instead of OpenAI)
- YAML config

##### Special note: Gemini API key
- Model.py reading “gemini_api_key” from config.yaml before making requests

##### Final advice
- Be as detailed as possible in showing each script’s responsibility.
- Essential flows without overcomplicating—group low-level utils under “Utilities.”
Layer 1 (top): CLI & Config
Layer 2: Mode Orchestrator
Layer 3: Capture & Execution
Layer 4: LLM Interface & External Service
Layer 5: Documentation Output

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Layer 1: CLI & Configuration
    subgraph "Layer 1: CLI & Configuration"
        direction TB
        Learn["learn.py"]:::orchestration
        Run["run.py"]:::orchestration
        ConfigLoader["scripts/config.py"]:::config
        ConfigFile["config.yaml"]:::config
    end

    Learn -->|"invokes"| Orchestrator
    Run -->|"invokes"| Orchestrator
    Learn -->|reads| ConfigLoader
    Run -->|reads| ConfigLoader
    ConfigLoader -->|loads| ConfigFile

    %% Layer 2: Mode Orchestrator
    subgraph "Layer 2: Mode Orchestrator"
        direction TB
        Orchestrator["Mode Orchestrator"]:::orchestration
        HumanDemo["HumanDemo Module\n(step_recorder.py)"]:::orchestration
        AutoExplore["AutoExplore Module\n(self_explorer.py)"]:::orchestration
    end

    Orchestrator --> HumanDemo
    Orchestrator --> AutoExplore

    %% Layer 3: Capture & Execution
    subgraph "Layer 3: Capture & Execution"
        direction TB
        UIController["UI Controller\n(and_controller.py)"]:::capture
        TaskExecutor["Task Executor\n(task_executor.py)"]:::capture
        Utilities["Utilities\n(utils.py)"]:::utilities
    end

    HumanDemo --> UIController
    HumanDemo --> TaskExecutor
    AutoExplore --> UIController
    AutoExplore --> TaskExecutor
    UIController --> Utilities
    TaskExecutor --> Utilities

    %% Layer 4: LLM Interface & External Service
    subgraph "Layer 4: LLM Interface & External Service"
        direction TB
        Prompts["Prompt Templates\n(prompts.py)"]:::capture
        Model["LLM Interface\n(model.py)"]:::externalService
        Gemini["Gemini API"]:::externalService
    end

    UIController -->|"screenshots+XML"| Prompts
    TaskExecutor -->|"actions"| Prompts
    Prompts -->|"formatted prompt"| Model
    ConfigLoader -->|"provides gemini_api_key"| Model
    Model -->|"API request"| Gemini
    Gemini -->|"response"| Model

    %% Layer 5: Documentation Output
    subgraph "Layer 5: Documentation Output"
        direction TB
        DocGen["Doc Generator\n(document_generation.py)"]:::docs
        Artifacts["Artifacts Store\ntasks/"]:::docs
    end

    Model -->|"LLM output"| DocGen
    DocGen -->|"writes docs"| Artifacts

    %% Project Metadata & Dependencies
    subgraph "Project Metadata & Dependencies"
        direction TB
        Readme["README.md"]:::metadata
        Requirements["requirements.txt"]:::metadata
    end

    %% Click Events
    click Learn "https://github.com/greywarden07/touchbot/blob/main/learn.py"
    click Run "https://github.com/greywarden07/touchbot/blob/main/run.py"
    click ConfigLoader "https://github.com/greywarden07/touchbot/blob/main/scripts/config.py"
    click ConfigFile "https://github.com/greywarden07/touchbot/blob/main/config.yaml"
    click HumanDemo "https://github.com/greywarden07/touchbot/blob/main/scripts/step_recorder.py"
    click AutoExplore "https://github.com/greywarden07/touchbot/blob/main/scripts/self_explorer.py"
    click UIController "https://github.com/greywarden07/touchbot/blob/main/scripts/and_controller.py"
    click TaskExecutor "https://github.com/greywarden07/touchbot/blob/main/scripts/task_executor.py"
    click Prompts "https://github.com/greywarden07/touchbot/blob/main/scripts/prompts.py"
    click Model "https://github.com/greywarden07/touchbot/blob/main/scripts/model.py"
    click DocGen "https://github.com/greywarden07/touchbot/blob/main/scripts/document_generation.py"
    click Utilities "https://github.com/greywarden07/touchbot/blob/main/scripts/utils.py"
    click Readme "https://github.com/greywarden07/touchbot/blob/main/README.md"
    click Requirements "https://github.com/greywarden07/touchbot/blob/main/requirements.txt"

    %% Styles
    classDef orchestration fill:#D0E6FF,stroke:#3B78FF,color:#000
    classDef capture fill:#D4F5D4,stroke:#28A828,color:#000
    classDef config fill:#FFE8A1,stroke:#FFBD2E,color:#000
    classDef externalService fill:#E9D4FF,stroke:#A54EFF,color:#000
    classDef utilities fill:#F0F0F0,stroke:#999999,color:#000
    classDef docs fill:#FFF2CC,stroke:#E6B800,color:#000
    classDef metadata fill:#EBEBEB,stroke:#CCCCCC,color:#000
```
