# zoelee23277789/robot-agent architecture

> Architecture diagram of the GitHub repository zoelee23277789/robot-agent, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/zoelee23277789/robot-agent
- Repository: https://github.com/zoelee23277789/robot-agent
- Video tour (about a minute): https://gitdiagram.com/zoelee23277789/robot-agent/video

## Overview

RoboMaster LLM Agent turns a user task into a robot result through one agent loop: it observes telemetry and camera images, asks an LLM to choose registered actions, invokes skills or perception, and sends robot commands over HTTP to a separate server. The server supplies observations and executes commands against the robot or simulator; navigation uses a server-side occupancy map and planner. Separate branches cover scene support and milestone-based evaluation. The README documents several subsystem relationships; source excerpts are partial, so unsampled internals are represented at directory level where appropriate.

Main entry points: [`robot_agent/service.py`](https://github.com/zoelee23277789/robot-agent/blob/main/robot_agent/service.py) (Observe-act loop), [`Test_Robot/robot_server.py`](https://github.com/zoelee23277789/robot-agent/blob/main/Test_Robot/robot_server.py) (Robot HTTP server), [`RobotAgent.py`](https://github.com/zoelee23277789/robot-agent/blob/main/RobotAgent.py) (Agent entry), [`robot_agent/llm_factory.py`](https://github.com/zoelee23277789/robot-agent/blob/main/robot_agent/llm_factory.py) (LLM factory)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_agent["Agent control"]
  node_entry["Agent entry<br/>[RobotAgent.py]"]
  node_loop["Observe-act loop<br/>[service.py]"]
  node_actions["Action registry<br/>[actions.py]"]
  node_skills["Robot skills<br/>[skills.py]"]
  node_perception["Visual perception<br/>[perception.py]"]
  node_client["HTTP client<br/>[client.py]"]
end

subgraph group_robot["Robot runtime"]
  node_server["Robot HTTP server<br/>[robot_server.py]"]
  node_navigation["Occupancy map<br/>[nav_map.py]"]
  node_connection["SDK connection<br/>[rm_connection.py]"]
end

subgraph group_llm["LLM interface"]
  node_factory["LLM factory<br/>[llm_factory.py]"]
  node_messages["Message types<br/>[messages.py]"]
  node_openai["OpenAI adapter<br/>[chat.py]"]
  node_google["Google adapter<br/>[chat.py]"]
end

subgraph group_evaluation["Evaluation"]
  node_evaluator["Evaluation runner<br/>[run_evaluation.py]"]
  node_metrics["Milestone metrics<br/>[metrics.py]"]
  node_judge["Milestone judge<br/>[judge.py]"]
end

subgraph group_environment["Scene support"]
  node_scenes["Scene builders"]
  node_sensor_setup["Camera and sensors"]
end

node_operator(("Task operator"))
node_device["Robot or simulator"]

node_operator -->|"sets task"| node_entry
node_entry -->|"starts loop"| node_loop
node_entry -->|"builds model"| node_factory
node_entry -->|"creates client"| node_client
node_loop -->|"dispatches actions"| node_actions
node_loop -->|"calls skills"| node_skills
node_loop -->|"requests perception"| node_perception
node_loop -->|"reads and acts"| node_client
node_loop -->|"uses LLM"| node_factory
node_factory -->|"selects provider"| node_openai
node_factory -->|"selects provider"| node_google
node_openai -->|"serializes messages"| node_messages
node_google -->|"serializes messages"| node_messages
node_client -->|"sends HTTP requests"| node_server
node_server -->|"plans routes"| node_navigation
node_server -->|"connects through"| node_connection
node_connection -->|"controls device"| node_device
node_server -->|"reads and commands"| node_device
node_evaluator -->|"runs agent"| node_loop
node_evaluator -->|"evaluates runs"| node_metrics
node_evaluator -->|"judges milestones"| node_judge
node_metrics -->|"uses judgments"| node_judge
node_scenes -->|"builds simulator scenes"| node_device
node_sensor_setup -->|"adds sensing"| node_device
node_server -->|"initializes sensors"| node_sensor_setup

click node_entry "https://github.com/zoelee23277789/robot-agent/blob/main/RobotAgent.py"
click node_loop "https://github.com/zoelee23277789/robot-agent/blob/main/robot_agent/service.py"
click node_actions "https://github.com/zoelee23277789/robot-agent/blob/main/robot_agent/actions.py"
click node_skills "https://github.com/zoelee23277789/robot-agent/blob/main/robot_agent/skills.py"
click node_perception "https://github.com/zoelee23277789/robot-agent/blob/main/robot_agent/perception.py"
click node_client "https://github.com/zoelee23277789/robot-agent/blob/main/robot_agent/client.py"
click node_factory "https://github.com/zoelee23277789/robot-agent/blob/main/robot_agent/llm_factory.py"
click node_messages "https://github.com/zoelee23277789/robot-agent/blob/main/robot_agent/llm/messages.py"
click node_openai "https://github.com/zoelee23277789/robot-agent/blob/main/robot_agent/llm/openai/chat.py"
click node_google "https://github.com/zoelee23277789/robot-agent/blob/main/robot_agent/llm/google/chat.py"
click node_server "https://github.com/zoelee23277789/robot-agent/blob/main/Test_Robot/robot_server.py"
click node_navigation "https://github.com/zoelee23277789/robot-agent/blob/main/Test_Robot/nav_map.py"
click node_connection "https://github.com/zoelee23277789/robot-agent/blob/main/Test_Robot/rm_connection.py"
click node_evaluator "https://github.com/zoelee23277789/robot-agent/blob/main/robot_eval/run_evaluation.py"
click node_metrics "https://github.com/zoelee23277789/robot-agent/blob/main/robot_eval/metrics.py"
click node_judge "https://github.com/zoelee23277789/robot-agent/blob/main/robot_eval/judge.py"
click node_scenes "https://github.com/zoelee23277789/robot-agent/tree/main/Env"
click node_sensor_setup "https://github.com/zoelee23277789/robot-agent/tree/main/Env"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_entry,node_loop,node_actions,node_skills,node_perception,node_client toneBlue
class node_server,node_navigation,node_connection toneAmber
class node_factory,node_messages,node_openai,node_google toneMint
class node_evaluator,node_metrics,node_judge,node_operator toneRose
class node_scenes,node_sensor_setup,node_device toneIndigo
```

## Components

### Agent control

- [Agent entry](https://github.com/zoelee23277789/robot-agent/blob/main/RobotAgent.py): `RobotAgent.py`
- [Observe-act loop](https://github.com/zoelee23277789/robot-agent/blob/main/robot_agent/service.py): `robot_agent/service.py`
- [Action registry](https://github.com/zoelee23277789/robot-agent/blob/main/robot_agent/actions.py): `robot_agent/actions.py`
- [Robot skills](https://github.com/zoelee23277789/robot-agent/blob/main/robot_agent/skills.py): `robot_agent/skills.py`
- [Visual perception](https://github.com/zoelee23277789/robot-agent/blob/main/robot_agent/perception.py): `robot_agent/perception.py`
- [HTTP client](https://github.com/zoelee23277789/robot-agent/blob/main/robot_agent/client.py): `robot_agent/client.py`

### Robot runtime

- [Robot HTTP server](https://github.com/zoelee23277789/robot-agent/blob/main/Test_Robot/robot_server.py): `Test_Robot/robot_server.py`
- [Occupancy map](https://github.com/zoelee23277789/robot-agent/blob/main/Test_Robot/nav_map.py): `Test_Robot/nav_map.py`
- [SDK connection](https://github.com/zoelee23277789/robot-agent/blob/main/Test_Robot/rm_connection.py): `Test_Robot/rm_connection.py`

### LLM interface

- [LLM factory](https://github.com/zoelee23277789/robot-agent/blob/main/robot_agent/llm_factory.py): `robot_agent/llm_factory.py`
- [Message types](https://github.com/zoelee23277789/robot-agent/blob/main/robot_agent/llm/messages.py): `robot_agent/llm/messages.py`
- [OpenAI adapter](https://github.com/zoelee23277789/robot-agent/blob/main/robot_agent/llm/openai/chat.py): `robot_agent/llm/openai/chat.py`
- [Google adapter](https://github.com/zoelee23277789/robot-agent/blob/main/robot_agent/llm/google/chat.py): `robot_agent/llm/google/chat.py`

### Evaluation

- [Evaluation runner](https://github.com/zoelee23277789/robot-agent/blob/main/robot_eval/run_evaluation.py): `robot_eval/run_evaluation.py`
- [Milestone metrics](https://github.com/zoelee23277789/robot-agent/blob/main/robot_eval/metrics.py): `robot_eval/metrics.py`
- [Milestone judge](https://github.com/zoelee23277789/robot-agent/blob/main/robot_eval/judge.py): `robot_eval/judge.py`

### Scene support

- [Scene builders](https://github.com/zoelee23277789/robot-agent/tree/main/Env): `Env`
- [Camera and sensors](https://github.com/zoelee23277789/robot-agent/tree/main/Env): `Env`

### Other components

- **Task operator**
- **Robot or simulator**

## Connections

- Task operator → Agent entry: sets task
- Agent entry → Observe-act loop: starts loop
- Agent entry → LLM factory: builds model (evidence: `RobotAgent.py`)
- Agent entry → HTTP client: creates client (evidence: `RobotAgent.py`)
- Observe-act loop → Action registry: dispatches actions (evidence: `robot_agent/service.py`)
- Observe-act loop → Robot skills: calls skills (evidence: `robot_agent/service.py`)
- Observe-act loop → Visual perception: requests perception (evidence: `robot_agent/service.py`)
- Observe-act loop → HTTP client: reads and acts (evidence: `robot_agent/service.py`)
- Observe-act loop → LLM factory: uses LLM
- LLM factory → OpenAI adapter: selects provider (evidence: `robot_agent/llm_factory.py`)
- LLM factory → Google adapter: selects provider (evidence: `robot_agent/llm_factory.py`)
- OpenAI adapter → Message types: serializes messages (evidence: `robot_agent/llm/openai/chat.py`)
- Google adapter → Message types: serializes messages (evidence: `robot_agent/llm/google/chat.py`)
- HTTP client → Robot HTTP server: sends HTTP requests
- Robot HTTP server → Occupancy map: plans routes (evidence: `Test_Robot/robot_server.py`)
- Robot HTTP server → SDK connection: connects through (evidence: `Test_Robot/robot_server.py`)
- SDK connection → Robot or simulator: controls device
- Robot HTTP server → Robot or simulator: reads and commands
- Evaluation runner → Observe-act loop: runs agent
- Evaluation runner → Milestone metrics: evaluates runs (evidence: `robot_eval/run_evaluation.py`)
- Evaluation runner → Milestone judge: judges milestones (evidence: `robot_eval/run_evaluation.py`)
- Milestone metrics → Milestone judge: uses judgments (evidence: `robot_eval/metrics.py`)
- Scene builders → Robot or simulator: builds simulator scenes
- Camera and sensors → Robot or simulator: adds sensing
- Robot HTTP server → Camera and sensors: initializes sensors