How zoelee23277789/robot-agent works
RoboMaster LLM Agent turns a user task into a robot result through one agent loop: it observes telemetry and camera images, asks an LLM to choose registered actions, invokes skills or perception, and sends robot commands over HTTP to a separate server. The server supplies observations and executes commands against the robot or simulator; navigation uses a server-side occupancy map and planner. Separate branches cover scene support and milestone-based evaluation. The README documents several subsystem relationships; source excerpts are partial, so unsampled internals are represented at directory level where appropriate.
Main entry points: service.py Observe-act loop · robot_server.py Robot HTTP server · RobotAgent.py Agent entry · llm_factory.py LLM factory
Components 20
Agent control
- Agent entry
RobotAgent.py - Observe-act loop
robot_agent/service.py - Action registry
robot_agent/actions.py - Robot skills
robot_agent/skills.py - Visual perception
robot_agent/perception.py - HTTP client
robot_agent/client.py
Robot runtime
- Robot HTTP server
Test_Robot/robot_server.py - Occupancy map
Test_Robot/nav_map.py - SDK connection
Test_Robot/rm_connection.py
LLM interface
- LLM factory
robot_agent/llm_factory.py - Message types
robot_agent/llm/messages.py - OpenAI adapter
robot_agent/llm/openai/chat.py - Google adapter
robot_agent/llm/google/chat.py
Evaluation
- Evaluation runner
robot_eval/run_evaluation.py - Milestone metrics
robot_eval/metrics.py - Milestone judge
robot_eval/judge.py
Scene support
- Scene builders
Env - Camera and sensors
Env
Other components
- Task operator
- Robot or simulator
Connections 25
- Task operator → Agent entry: sets task
- Agent entry → Observe-act loop: starts loop
- Agent entry → LLM factory: builds model
RobotAgent.py - Agent entry → HTTP client: creates client
RobotAgent.py - Observe-act loop → Action registry: dispatches actions
robot_agent/service.py - Observe-act loop → Robot skills: calls skills
robot_agent/service.py - Observe-act loop → Visual perception: requests perception
robot_agent/service.py - Observe-act loop → HTTP client: reads and acts
robot_agent/service.py - Observe-act loop → LLM factory: uses LLM
- LLM factory → OpenAI adapter: selects provider
robot_agent/llm_factory.py - LLM factory → Google adapter: selects provider
robot_agent/llm_factory.py - OpenAI adapter → Message types: serializes messages
robot_agent/llm/openai/chat.py - Google adapter → Message types: serializes messages
robot_agent/llm/google/chat.py - HTTP client → Robot HTTP server: sends HTTP requests
- Robot HTTP server → Occupancy map: plans routes
Test_Robot/robot_server.py - Robot HTTP server → SDK connection: connects through
Test_Robot/robot_server.py - SDK connection → Robot or simulator: controls device
- Robot HTTP server → Robot or simulator: reads and commands
- Evaluation runner → Observe-act loop: runs agent
- Evaluation runner → Milestone metrics: evaluates runs
robot_eval/run_evaluation.py - Evaluation runner → Milestone judge: judges milestones
robot_eval/run_evaluation.py - Milestone metrics → Milestone judge: uses judgments
robot_eval/metrics.py - Scene builders → Robot or simulator: builds simulator scenes
- Camera and sensors → Robot or simulator: adds sensing
- Robot HTTP server → Camera and sensors: initializes sensors