How hkuds/phonecli works
PhoneCLI compiles mobile-app navigation into callable commands, then routes requests through deterministic command replay before model-backed GUI handling for unfamiliar screens. Its broader product includes device control, bounded conversational memory, replaceable model backends, and Android task evaluation; the repository also contains model/data-training code. The README documents the main routing behavior, while sampled source confirms model calls, Android control, and evaluation mechanics. Several runtime modules were not sampled, so their connections are shown only where the README explicitly describes them.
Main entry points: agent.py Agent orchestration · and_controller.py Android controller · task.py Task execution · model.py Model adapters
Components 21
Mobile Harness
- Request runner
phonecli/run.py - Agent orchestration
phonecli/agent.py - App-map crawler
phonecli/build_map.py - Compiled app map
phonecli/app_map.py - Command interface
phonecli/cli.py - Android controller
utils_mobile/and_controller.py - GUI executor
page_executor/simple_vision_executor.py - Text executor
page_executor/text_executor.py - UI parsing
utils_mobile/xml_tool.py
Models and Memory
- Model adapters
agent/model.py - LLM client
phonecli/llm_client.py - Conversation memory
phonecli/memory.py - Token accounting
phonecli/token_usage.py
Evaluation
- Evaluation runner
evaluation/evaluation.py - Task execution
evaluation/task.py - Task definitions
evaluation/definition.py - LLM judge
evaluation/tasks/llm_evaluator.py
Model Training
- Training data
prepare_data/visual_model_data/data_maker.py - SFT and RL
model_training/R1-V/src/r1-v/src/open_r1
Other components
- Mobile request
- Mobile device
Connections 19
- Mobile request → Request runner: submits request
README.md - Request runner → Agent orchestration: routes request
README.md - App-map crawler → Compiled app map: compiles navigation
README.md - Agent orchestration → Compiled app map: replays commands
README.md - Agent orchestration → GUI executor: falls back
README.md - Command interface → Android controller: issues device commands
- GUI executor → Android controller: acts on device
- Text executor → Android controller: acts on device
page_executor/text_executor.py - Android controller → Mobile device: controls
utils_mobile/and_controller.py - Android controller → UI parsing: captures UI state
- Agent orchestration → Model adapters: requests model action
- Model adapters → LLM client: calls completion
- Agent orchestration → Conversation memory: uses context
README.md - Model adapters → Token accounting: records usage
agent/model.py - Evaluation runner → Task execution: runs tasks
- Task execution → Task definitions: loads task metrics
evaluation/task.py - Task execution → LLM judge: judges results
evaluation/tasks/clock/clock.py - Task execution → Token accounting: reports tokens
evaluation/task.py - Training data → SFT and RL: feeds training