How mobilellm/agentprog works
AgentProg is a long-horizon Android GUI agent: a task description is turned into a Semantic Task Program, then interpreted step-by-step into device actions using live screenshots and UI observations. Its workflow context and belief state support control-flow pruning, persistent task data, repeated-step recall and recovery. AndroidWorld is the companion evaluation environment, with registered task suites, device interaction and task-specific success checks. Edges below are limited to relationships explicit in the README or visible in the supplied excerpts/SOURCE INDEX; paths for unsampled subsystems identify only existing directories.
Main entry points: agentprog.py Execution controller · workflow_query_llm.py Program runner · run.py Evaluation runner · pipeline.py Workflow pipeline
Components 25
AgentProg runtime
- CLI entry
agentprog/cli.py - Workflow pipeline
agentprog/plan/code_exec/workflow/pipeline.py - Foundation model
agentprog/all_utils/fm.py - Model API utilities
agentprog/all_utils/general_utils.py
Program and context
- Program generation
agentprog/plan/code_exec/workflow/generate_workflow.py - Program runner
agentprog/plan/code_exec/workflow/workflow_query_llm.py - Execution controller
agentprog/plan/code_exec/workflow/core/agentprog.py - Workflow context
agentprog/plan/workflow_utils.py - Agent context
agentprog/plan/agentprog_utils.py - Belief state
agentprog/plan/belief/belief_state.py - Program prompts
agentprog/plan/code_exec/workflow/workflow_prompts
Android interaction
- Mobile API
agentprog/all_utils/mobile_utils.py - UI-TARS locator
agentprog/all_utils/ui_tars_utils.py
AndroidWorld evaluation
- Evaluation runner
eval/android_world/run.py - Task registry
eval/android_world/android_world/registry.py - Task evaluation
eval/android_world/android_world/task_evals/task_eval.py - Information retrieval tasks
eval/android_world/android_world/task_evals/information_retrieval/information_retrieval.py - Async environment
eval/android_world/android_world/env/interface.py - World controller
eval/android_world/android_world/env/android_world_controller.py - ADB utilities
eval/android_world/android_world/env/adb_utils.py - UI representation
eval/android_world/android_world/env/representation_utils.py - Retrieval task helpers
eval/android_world/android_world/task_evals/information_retrieval - AgentProg adapter
eval/android_world/android_world/agents/agentprog.py
Other components
- Task user
- Android device
Connections 24
- Task user → CLI entry: submits task
README.md - CLI entry → Workflow pipeline: starts pipeline
agentprog/cli.py - Workflow pipeline → Program generation: generates program
agentprog/plan/code_exec/workflow/pipeline.py - Workflow pipeline → Program runner: runs program
agentprog/plan/code_exec/workflow/pipeline.py - Program runner → Execution controller: dispatches execution
agentprog/plan/code_exec/workflow/workflow_query_llm.py - Execution controller → Workflow context: updates context
agentprog/plan/code_exec/workflow/core/agentprog.py - Execution controller → Belief state: uses belief state
agentprog/plan/code_exec/workflow/core/agentprog.py - Execution controller → Agent context: builds agent context
agentprog/plan/code_exec/workflow/core/agentprog.py - Execution controller → Program prompts: uses prompt set
agentprog/plan/code_exec/workflow/core/agentprog.py - Program runner → Mobile API: captures screenshots
agentprog/plan/code_exec/workflow/workflow_query_llm.py - Program runner → Foundation model: queries model
agentprog/plan/code_exec/workflow/workflow_query_llm.py - Foundation model → Model API utilities: formats model requests
agentprog/all_utils/fm.py - Mobile API → UI-TARS locator: selects locator
agentprog/all_utils/mobile_utils.py - Mobile API → Android device: acts on device
- Evaluation runner → Task registry: loads tasks
eval/android_world/run.py - Task registry → Task evaluation: registers evaluations
eval/android_world/android_world/registry.py - Task registry → Information retrieval tasks: registers tasks
eval/android_world/android_world/registry.py - Information retrieval tasks → Task evaluation: extends evaluation
eval/android_world/android_world/task_evals/information_retrieval/information_retrieval.py - Information retrieval tasks → Retrieval task helpers: sets task state
eval/android_world/android_world/task_evals/information_retrieval/information_retrieval.py - Async environment → World controller: uses controller
eval/android_world/android_world/env/interface.py - World controller → UI representation: converts UI tree
eval/android_world/android_world/env/android_world_controller.py - Async environment → ADB utilities: dispatches actions
eval/android_world/android_world/env/interface.py - Evaluation runner → Async environment: runs against environment
eval/android_world/run.py - Evaluation runner → AgentProg adapter: selects AgentProg agent
eval/android_world/run.py