# alen-saji-05/androidagent architecture

> Architecture diagram of the GitHub repository alen-saji-05/androidagent, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/alen-saji-05/androidagent
- Repository: https://github.com/alen-saji-05/androidagent

## Overview

This repository is an early Android UI agent: a user goal is decomposed into subgoals, grounded against the visible screen, translated into ADB actions, and reassessed as the screen changes. The implemented screen path combines icon detection and OCR into classified UI elements; planning and grounding support replaceable model-backed and offline implementations. The README describes the end-to-end loop, while sampled code confirms its injected components and internal calls. The verifier, accessibility-tree path, shared memory, and Android app are documented as not yet implemented; concrete CLI wiring is not inferred here.

Main entry points: [`executor/loop.py`](https://github.com/alen-saji-05/androidagent/blob/main/executor/loop.py) (Agent loop), [`perception/pipeline.py`](https://github.com/alen-saji-05/androidagent/blob/main/perception/pipeline.py) (Screen parser), [`planner/planner.py`](https://github.com/alen-saji-05/androidagent/blob/main/planner/planner.py) (Plan state machine), [`executor/grounder.py`](https://github.com/alen-saji-05/androidagent/blob/main/executor/grounder.py) (Grounder backends)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_control["Agent control"]
  node_loop["Agent loop<br/>[loop.py]"]
  node_confirm["Confirmation<br/>[loop.py]"]
end

subgraph group_planning["Planning"]
  node_planner["Plan state machine<br/>[planner.py]"]
  node_backend["Planner backend<br/>[backend.py]"]
  node_digest["Screen digest<br/>[digest.py]"]
end

subgraph group_perception["Screen perception"]
  node_parser["Screen parser<br/>[pipeline.py]"]
  node_detector["Icon detector<br/>[detector.py]"]
  node_ocr["OCR backends<br/>[ocr.py]"]
  node_fusion["Element fusion<br/>[fusion.py]"]
  node_classifier["Element classifier<br/>[classify.py]"]
  node_schema["Screen state<br/>[schema.py]"]
end

subgraph group_grounding["Action grounding"]
  node_grounder["Grounder backends<br/>[grounder.py]"]
  node_heuristics["Grounding heuristics<br/>[grounding.py]"]
end

subgraph group_device["Device control"]
  node_executor["Action executor<br/>[executor.py]"]
  node_adb["ADB interface<br/>[adb.py]"]
end

node_user(("User"))
node_gemini{{"Gemini API"}}
node_groq{{"Groq API"}}
node_phone(("Android device"))

node_user -->|"submits goal"| node_loop
node_loop -->|"decomposes and reassesses"| node_planner
node_planner -->|"requests plan updates"| node_backend
node_backend -.->|"generates JSON"| node_gemini
node_planner -->|"digests screen"| node_digest
node_loop -.->|"requests screen state"| node_parser
node_parser -->|"detects regions"| node_detector
node_parser -->|"reads text"| node_ocr
node_parser -->|"merges detections and text"| node_fusion
node_parser -->|"classifies elements"| node_classifier
node_parser -->|"builds screen state"| node_schema
node_loop -->|"grounds subgoal"| node_grounder
node_grounder -->|"uses keyword matching"| node_heuristics
node_grounder -.->|"requests action decision"| node_groq
node_loop -->|"requests approval"| node_confirm
node_loop -->|"dispatches actions"| node_executor
node_executor -->|"issues device commands"| node_adb
node_adb -->|"taps, types, captures"| node_phone
node_phone -->|"returns screenshot"| node_adb

click node_loop "https://github.com/alen-saji-05/androidagent/blob/main/executor/loop.py"
click node_confirm "https://github.com/alen-saji-05/androidagent/blob/main/executor/loop.py"
click node_planner "https://github.com/alen-saji-05/androidagent/blob/main/planner/planner.py"
click node_backend "https://github.com/alen-saji-05/androidagent/blob/main/planner/backend.py"
click node_digest "https://github.com/alen-saji-05/androidagent/blob/main/planner/digest.py"
click node_parser "https://github.com/alen-saji-05/androidagent/blob/main/perception/pipeline.py"
click node_detector "https://github.com/alen-saji-05/androidagent/blob/main/perception/detector.py"
click node_ocr "https://github.com/alen-saji-05/androidagent/blob/main/perception/ocr.py"
click node_fusion "https://github.com/alen-saji-05/androidagent/blob/main/perception/fusion.py"
click node_classifier "https://github.com/alen-saji-05/androidagent/blob/main/perception/classify.py"
click node_schema "https://github.com/alen-saji-05/androidagent/blob/main/perception/schema.py"
click node_grounder "https://github.com/alen-saji-05/androidagent/blob/main/executor/grounder.py"
click node_heuristics "https://github.com/alen-saji-05/androidagent/blob/main/executor/grounding.py"
click node_executor "https://github.com/alen-saji-05/androidagent/blob/main/executor/executor.py"
click node_adb "https://github.com/alen-saji-05/androidagent/blob/main/executor/adb.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_loop,node_confirm,node_user toneBlue
class node_planner,node_backend,node_digest toneAmber
class node_parser,node_detector,node_ocr,node_fusion,node_classifier,node_schema,node_gemini,node_groq toneMint
class node_grounder,node_heuristics toneRose
class node_executor,node_adb,node_phone toneIndigo
```

## Components

### Agent control

- [Agent loop](https://github.com/alen-saji-05/androidagent/blob/main/executor/loop.py): `executor/loop.py`
- [Confirmation](https://github.com/alen-saji-05/androidagent/blob/main/executor/loop.py): `executor/loop.py`

### Planning

- [Plan state machine](https://github.com/alen-saji-05/androidagent/blob/main/planner/planner.py): `planner/planner.py`
- [Planner backend](https://github.com/alen-saji-05/androidagent/blob/main/planner/backend.py): `planner/backend.py`
- [Screen digest](https://github.com/alen-saji-05/androidagent/blob/main/planner/digest.py): `planner/digest.py`

### Screen perception

- [Screen parser](https://github.com/alen-saji-05/androidagent/blob/main/perception/pipeline.py): `perception/pipeline.py`
- [Icon detector](https://github.com/alen-saji-05/androidagent/blob/main/perception/detector.py): `perception/detector.py`
- [OCR backends](https://github.com/alen-saji-05/androidagent/blob/main/perception/ocr.py): `perception/ocr.py`
- [Element fusion](https://github.com/alen-saji-05/androidagent/blob/main/perception/fusion.py): `perception/fusion.py`
- [Element classifier](https://github.com/alen-saji-05/androidagent/blob/main/perception/classify.py): `perception/classify.py`
- [Screen state](https://github.com/alen-saji-05/androidagent/blob/main/perception/schema.py): `perception/schema.py`

### Action grounding

- [Grounder backends](https://github.com/alen-saji-05/androidagent/blob/main/executor/grounder.py): `executor/grounder.py`
- [Grounding heuristics](https://github.com/alen-saji-05/androidagent/blob/main/executor/grounding.py): `executor/grounding.py`

### Device control

- [Action executor](https://github.com/alen-saji-05/androidagent/blob/main/executor/executor.py): `executor/executor.py`
- [ADB interface](https://github.com/alen-saji-05/androidagent/blob/main/executor/adb.py): `executor/adb.py`

### Other components

- **User**
- **Gemini API**
- **Groq API**
- **Android device**

## Connections

- User → Agent loop: submits goal
- Agent loop → Plan state machine: decomposes and reassesses
- Plan state machine → Planner backend: requests plan updates
- Planner backend → Gemini API: generates JSON
- Plan state machine → Screen digest: digests screen
- Agent loop → Screen parser: requests screen state
- Screen parser → Icon detector: detects regions
- Screen parser → OCR backends: reads text
- Screen parser → Element fusion: merges detections and text
- Screen parser → Element classifier: classifies elements
- Screen parser → Screen state: builds screen state
- Agent loop → Grounder backends: grounds subgoal
- Grounder backends → Grounding heuristics: uses keyword matching
- Grounder backends → Groq API: requests action decision
- Agent loop → Confirmation: requests approval
- Agent loop → Action executor: dispatches actions
- Action executor → ADB interface: issues device commands
- ADB interface → Android device: taps, types, captures
- Android device → ADB interface: returns screenshot