# vector-wangel/xlerobot architecture

> Architecture diagram of the GitHub repository vector-wangel/xlerobot, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/vector-wangel/xlerobot
- Repository: https://github.com/vector-wangel/xlerobot

## Overview

XLeRobot is an embodied-AI mobile manipulation robot project, with software for real-robot operation, teleoperation, simulation, and browser-based remote control. The main user journey is to provide commands through a controller or browser, route them through a robot-specific or remote-control interface, and operate the robot or its simulation; VR provides a separate teleoperation path. The graph includes documented capabilities whose implementation was not sampled, but omits speculative connections between them. Some subsystem paths identify exact directories because their individual runtime files were not inspected.

Main entry points: [`web_control/client/src/App.tsx`](https://github.com/vector-wangel/xlerobot/blob/main/web_control/client/src/App.tsx) (Control Dashboard), [`web_control/server/main.py`](https://github.com/vector-wangel/xlerobot/blob/main/web_control/server/main.py) (Web Server), [`XLeVR/xlevr/inputs/vr_ws_server.py`](https://github.com/vector-wangel/xlerobot/blob/main/XLeVR/xlevr/inputs/vr_ws_server.py) (VR WebSocket), [`software/src/teleporators/xlerobot_vr/vr_monitor.py`](https://github.com/vector-wangel/xlerobot/blob/main/software/src/teleporators/xlerobot_vr/vr_monitor.py) (VR Monitor)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_real["Real Robot"]
  node_robot["XLeRobot<br/>[xlerobot.py]"]
  node_wheels["Two-wheel Robot"]
  node_mecanum["Mecanum Robot<br/>[xlerobot.py]"]
  node_robotclient["Robot Client<br/>[xlerobot_client.py]"]
  node_wheelclient["Two-wheel Client"]
  node_mecanumclient["Mecanum Client<br/>[xlerobot_client.py]"]
end

subgraph group_teleop["Teleoperation"]
  node_vr["VR Teleoperation<br/>[xlerobot_vr.py]"]
  node_vrmonitor["VR Monitor<br/>[vr_monitor.py]"]
  node_vrserver["VR WebSocket<br/>[vr_ws_server.py]"]
  node_vrconfig["VR Configuration<br/>[config.py]"]
  node_vrbrowser["Browser VR UI<br/>[vr_app.js]"]
  node_joycon["Joy-Con Input<br/>[joyconrobotics.py]"]
end

subgraph group_sim["Simulation"]
  node_maniskill["ManiSkill Simulation<br/>[xlerobot.py]"]
  node_maniskillhost["ManiSkill Host"]
  node_mujoco["MuJoCo Simulation<br/>[xlerobot_mujoco.py]"]
end

subgraph group_remote["Remote Control"]
  node_webapp["Control Dashboard<br/>[App.tsx]"]
  node_socket["Socket Controls<br/>[useSocket.ts]"]
  node_console["Control Console"]
  node_tabs["Multi-tab Panel<br/>[MultiTabPanel.tsx]"]
  node_video["Robot Video"]
  node_server["Web Server<br/>[main.py]"]
  node_core["Remote Core<br/>[remote_core.py]"]
end

subgraph group_internal["Control Internals"]
  node_protocol["Robot Protocol<br/>[protocol.py]"]
  node_streaming["Video Streaming<br/>[streaming.py]"]
  node_recorder["Data Recording<br/>[record.py]"]
  node_joyconlib["Joy-Con Drivers<br/>[joycon.py]"]
end

node_operator(("Operator"))
node_webuser(("Remote Operator"))

node_operator -.->|"controls via VR"| node_vrbrowser
node_vrbrowser -.->|"sends controller data"| node_vrserver
node_vrserver -->|"provides goals"| node_vrmonitor
node_vrmonitor -->|"supports teleoperation"| node_vr
node_vrconfig -->|"configures"| node_vrserver
node_operator -->|"controls with Joy-Con"| node_joycon
node_joycon -->|"uses drivers"| node_joyconlib
node_operator -.->|"operates"| node_robot
node_operator -.->|"operates"| node_wheels
node_operator -.->|"operates"| node_mecanum
node_robotclient -.->|"commands remotely"| node_robot
node_wheelclient -.->|"commands remotely"| node_wheels
node_mecanumclient -.->|"commands remotely"| node_mecanum
node_operator -.->|"runs simulation"| node_maniskillhost
node_maniskillhost -->|"loads agent"| node_maniskill
node_operator -.->|"runs simulation"| node_mujoco
node_webuser -->|"uses dashboard"| node_webapp
node_webapp -->|"uses socket controls"| node_socket
node_webapp -->|"renders console"| node_console
node_webapp -->|"renders panels"| node_tabs
node_webapp -->|"renders video"| node_video
node_socket -.->|"connects by Socket.IO"| node_server
node_server -->|"dispatches commands"| node_core
node_server -->|"uses video manager"| node_streaming
node_core -->|"encodes commands"| node_protocol
node_webuser -.->|"records robot data"| node_recorder

click node_robot "https://github.com/vector-wangel/xlerobot/blob/main/software/src/robots/xlerobot/xlerobot.py"
click node_wheels "https://github.com/vector-wangel/xlerobot/blob/main/software/src/robots/xlerobot_2wheels/xlerobot_2wheels.py"
click node_mecanum "https://github.com/vector-wangel/xlerobot/blob/main/software/src/robots/xlerobot_mecanum/xlerobot.py"
click node_robotclient "https://github.com/vector-wangel/xlerobot/blob/main/software/src/robots/xlerobot/xlerobot_client.py"
click node_wheelclient "https://github.com/vector-wangel/xlerobot/blob/main/software/src/robots/xlerobot_2wheels/xlerobot_2wheels_client.py"
click node_mecanumclient "https://github.com/vector-wangel/xlerobot/blob/main/software/src/robots/xlerobot_mecanum/xlerobot_client.py"
click node_vr "https://github.com/vector-wangel/xlerobot/blob/main/software/src/teleporators/xlerobot_vr/xlerobot_vr.py"
click node_vrmonitor "https://github.com/vector-wangel/xlerobot/blob/main/software/src/teleporators/xlerobot_vr/vr_monitor.py"
click node_vrserver "https://github.com/vector-wangel/xlerobot/blob/main/XLeVR/xlevr/inputs/vr_ws_server.py"
click node_vrconfig "https://github.com/vector-wangel/xlerobot/blob/main/XLeVR/xlevr/config.py"
click node_vrbrowser "https://github.com/vector-wangel/xlerobot/blob/main/XLeVR/web-ui/vr_app.js"
click node_joycon "https://github.com/vector-wangel/xlerobot/blob/main/software/joyconrobotics/joyconrobotics.py"
click node_maniskill "https://github.com/vector-wangel/xlerobot/blob/main/simulation/Maniskill/agents/xlerobot/xlerobot.py"
click node_maniskillhost "https://github.com/vector-wangel/xlerobot/blob/main/simulation/Maniskill/run_xlerobot_sim_host.py"
click node_mujoco "https://github.com/vector-wangel/xlerobot/blob/main/simulation/mujoco/xlerobot_mujoco.py"
click node_webapp "https://github.com/vector-wangel/xlerobot/blob/main/web_control/client/src/App.tsx"
click node_socket "https://github.com/vector-wangel/xlerobot/blob/main/web_control/client/src/hooks/useSocket.ts"
click node_console "https://github.com/vector-wangel/xlerobot/blob/main/web_control/client/src/components/features/BottomControlConsole.tsx"
click node_tabs "https://github.com/vector-wangel/xlerobot/blob/main/web_control/client/src/components/features/MultiTabPanel.tsx"
click node_video "https://github.com/vector-wangel/xlerobot/blob/main/web_control/client/src/components/features/RobotVideoCanvas.tsx"
click node_server "https://github.com/vector-wangel/xlerobot/blob/main/web_control/server/main.py"
click node_core "https://github.com/vector-wangel/xlerobot/blob/main/web_control/server/core/remote_core.py"
click node_protocol "https://github.com/vector-wangel/xlerobot/blob/main/web_control/server/core/protocol.py"
click node_streaming "https://github.com/vector-wangel/xlerobot/blob/main/web_control/server/api/streaming.py"
click node_recorder "https://github.com/vector-wangel/xlerobot/blob/main/software/src/record.py"
click node_joyconlib "https://github.com/vector-wangel/xlerobot/blob/main/software/joyconrobotics/joycon.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_robot,node_wheels,node_mecanum,node_robotclient,node_wheelclient,node_mecanumclient toneBlue
class node_vr,node_vrmonitor,node_vrserver,node_vrconfig,node_vrbrowser,node_joycon toneAmber
class node_maniskill,node_maniskillhost,node_mujoco toneMint
class node_webapp,node_socket,node_console,node_tabs,node_video,node_server,node_core toneRose
class node_protocol,node_streaming,node_recorder,node_joyconlib,node_operator,node_webuser toneIndigo
```

## Components

### Real Robot

- [XLeRobot](https://github.com/vector-wangel/xlerobot/blob/main/software/src/robots/xlerobot/xlerobot.py): `software/src/robots/xlerobot/xlerobot.py`
- [Two-wheel Robot](https://github.com/vector-wangel/xlerobot/blob/main/software/src/robots/xlerobot_2wheels/xlerobot_2wheels.py): `software/src/robots/xlerobot_2wheels/xlerobot_2wheels.py`
- [Mecanum Robot](https://github.com/vector-wangel/xlerobot/blob/main/software/src/robots/xlerobot_mecanum/xlerobot.py): `software/src/robots/xlerobot_mecanum/xlerobot.py`
- [Robot Client](https://github.com/vector-wangel/xlerobot/blob/main/software/src/robots/xlerobot/xlerobot_client.py): `software/src/robots/xlerobot/xlerobot_client.py`
- [Two-wheel Client](https://github.com/vector-wangel/xlerobot/blob/main/software/src/robots/xlerobot_2wheels/xlerobot_2wheels_client.py): `software/src/robots/xlerobot_2wheels/xlerobot_2wheels_client.py`
- [Mecanum Client](https://github.com/vector-wangel/xlerobot/blob/main/software/src/robots/xlerobot_mecanum/xlerobot_client.py): `software/src/robots/xlerobot_mecanum/xlerobot_client.py`

### Teleoperation

- [VR Teleoperation](https://github.com/vector-wangel/xlerobot/blob/main/software/src/teleporators/xlerobot_vr/xlerobot_vr.py): `software/src/teleporators/xlerobot_vr/xlerobot_vr.py`
- [VR Monitor](https://github.com/vector-wangel/xlerobot/blob/main/software/src/teleporators/xlerobot_vr/vr_monitor.py): `software/src/teleporators/xlerobot_vr/vr_monitor.py`
- [VR WebSocket](https://github.com/vector-wangel/xlerobot/blob/main/XLeVR/xlevr/inputs/vr_ws_server.py): `XLeVR/xlevr/inputs/vr_ws_server.py`
- [VR Configuration](https://github.com/vector-wangel/xlerobot/blob/main/XLeVR/xlevr/config.py): `XLeVR/xlevr/config.py`
- [Browser VR UI](https://github.com/vector-wangel/xlerobot/blob/main/XLeVR/web-ui/vr_app.js): `XLeVR/web-ui/vr_app.js`
- [Joy-Con Input](https://github.com/vector-wangel/xlerobot/blob/main/software/joyconrobotics/joyconrobotics.py): `software/joyconrobotics/joyconrobotics.py`

### Simulation

- [ManiSkill Simulation](https://github.com/vector-wangel/xlerobot/blob/main/simulation/Maniskill/agents/xlerobot/xlerobot.py): `simulation/Maniskill/agents/xlerobot/xlerobot.py`
- [ManiSkill Host](https://github.com/vector-wangel/xlerobot/blob/main/simulation/Maniskill/run_xlerobot_sim_host.py): `simulation/Maniskill/run_xlerobot_sim_host.py`
- [MuJoCo Simulation](https://github.com/vector-wangel/xlerobot/blob/main/simulation/mujoco/xlerobot_mujoco.py): `simulation/mujoco/xlerobot_mujoco.py`

### Remote Control

- [Control Dashboard](https://github.com/vector-wangel/xlerobot/blob/main/web_control/client/src/App.tsx): `web_control/client/src/App.tsx`
- [Socket Controls](https://github.com/vector-wangel/xlerobot/blob/main/web_control/client/src/hooks/useSocket.ts): `web_control/client/src/hooks/useSocket.ts`
- [Control Console](https://github.com/vector-wangel/xlerobot/blob/main/web_control/client/src/components/features/BottomControlConsole.tsx): `web_control/client/src/components/features/BottomControlConsole.tsx`
- [Multi-tab Panel](https://github.com/vector-wangel/xlerobot/blob/main/web_control/client/src/components/features/MultiTabPanel.tsx): `web_control/client/src/components/features/MultiTabPanel.tsx`
- [Robot Video](https://github.com/vector-wangel/xlerobot/blob/main/web_control/client/src/components/features/RobotVideoCanvas.tsx): `web_control/client/src/components/features/RobotVideoCanvas.tsx`
- [Web Server](https://github.com/vector-wangel/xlerobot/blob/main/web_control/server/main.py): `web_control/server/main.py`
- [Remote Core](https://github.com/vector-wangel/xlerobot/blob/main/web_control/server/core/remote_core.py): `web_control/server/core/remote_core.py`

### Control Internals

- [Robot Protocol](https://github.com/vector-wangel/xlerobot/blob/main/web_control/server/core/protocol.py): `web_control/server/core/protocol.py`
- [Video Streaming](https://github.com/vector-wangel/xlerobot/blob/main/web_control/server/api/streaming.py): `web_control/server/api/streaming.py`
- [Data Recording](https://github.com/vector-wangel/xlerobot/blob/main/software/src/record.py): `software/src/record.py`
- [Joy-Con Drivers](https://github.com/vector-wangel/xlerobot/blob/main/software/joyconrobotics/joycon.py): `software/joyconrobotics/joycon.py`

### Other components

- **Operator**
- **Remote Operator**

## Connections

- Operator → Browser VR UI: controls via VR (evidence: `README.md`)
- Browser VR UI → VR WebSocket: sends controller data
- VR WebSocket → VR Monitor: provides goals (evidence: `XLeVR/xlevr/inputs/vr_ws_server.py`)
- VR Monitor → VR Teleoperation: supports teleoperation (evidence: `software/src/teleporators/xlerobot_vr/xlerobot_vr.py`)
- VR Configuration → VR WebSocket: configures (evidence: `XLeVR/xlevr/inputs/vr_ws_server.py`)
- Operator → Joy-Con Input: controls with Joy-Con (evidence: `README.md`)
- Joy-Con Input → Joy-Con Drivers: uses drivers (evidence: `software/joyconrobotics/joyconrobotics.py`)
- Operator → XLeRobot: operates (evidence: `README.md`)
- Operator → Two-wheel Robot: operates (evidence: `README.md`)
- Operator → Mecanum Robot: operates (evidence: `README.md`)
- Robot Client → XLeRobot: commands remotely
- Two-wheel Client → Two-wheel Robot: commands remotely
- Mecanum Client → Mecanum Robot: commands remotely
- Operator → ManiSkill Host: runs simulation (evidence: `README.md`)
- ManiSkill Host → ManiSkill Simulation: loads agent (evidence: `simulation/Maniskill/run_xlerobot_sim_host.py`)
- Operator → MuJoCo Simulation: runs simulation (evidence: `README.md`)
- Remote Operator → Control Dashboard: uses dashboard (evidence: `web_control/client/src/App.tsx`)
- Control Dashboard → Socket Controls: uses socket controls (evidence: `web_control/client/src/App.tsx`)
- Control Dashboard → Control Console: renders console (evidence: `web_control/client/src/App.tsx`)
- Control Dashboard → Multi-tab Panel: renders panels (evidence: `web_control/client/src/App.tsx`)
- Control Dashboard → Robot Video: renders video (evidence: `web_control/client/src/App.tsx`)
- Socket Controls → Web Server: connects by Socket.IO
- Web Server → Remote Core: dispatches commands (evidence: `web_control/server/main.py`)
- Web Server → Video Streaming: uses video manager (evidence: `web_control/server/main.py`)
- Remote Core → Robot Protocol: encodes commands (evidence: `web_control/server/core/remote_core.py`)
- Remote Operator → Data Recording: records robot data (evidence: `README.md`)