# hexafuture/hexashell architecture

> Architecture diagram of the GitHub repository hexafuture/hexashell, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hexafuture/hexashell
- Repository: https://github.com/hexafuture/hexashell

## Overview

HexaShell is a task-agnostic harness for coding agents controlling simulated and real robots. The principal workflow is an episode: a model receives a task card and observations, runs Python tool calls in a sandbox, and gets motion/perception results while a benchmark or robot session supplies observations and reports the verdict. The graph preserves the documented simulation, real-robot, perception, and motion branches. Some adapter internals and prompt/skill assets are unsampled or absent from the supplied tree; those links are not inferred beyond explicit documentation or indexed imports.

Main entry points: [`hexashell/runtime/episode.py`](https://github.com/hexafuture/hexashell/blob/main/hexashell/runtime/episode.py) (Episode loop), [`hexashell/robot/host.py`](https://github.com/hexafuture/hexashell/blob/main/hexashell/robot/host.py) (Robot API host), [`hexashell/vision/scene.py`](https://github.com/hexafuture/hexashell/blob/main/hexashell/vision/scene.py) (Scene perception), [`hexashell/benches/armserver.py`](https://github.com/hexafuture/hexashell/blob/main/hexashell/benches/armserver.py) (Benchmark server)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_agent["Agent episode"]
  node_episode["Episode loop<br/>[episode.py]"]
  node_conversation["Conversation<br/>[conversation.py]"]
  node_llm["Model client<br/>[llm.py]"]
  node_taskcard["Task card<br/>[card.py]"]
  node_records["Episode records<br/>[records.py]"]
  node_session["Session contract<br/>[session.py]"]
  node_kernelhost["Cell host<br/>[host.py]"]
  node_worker["Sandbox worker<br/>[worker.py]"]
  node_sandbox["Python sandbox<br/>[sandbox.py]"]
end

subgraph group_tools["Robot tools"]
  node_robotapi["Robot API host<br/>[host.py]"]
  node_motion["Motion planner<br/>[driver.py]"]
  node_ik["Inverse kinematics<br/>[ik.py]"]
  node_kinematics["Arm kinematics<br/>[chain.py]"]
  node_state["Observation state<br/>[state.py]"]
end

subgraph group_perception["Perception"]
  node_scene["Scene perception<br/>[scene.py]"]
  node_rgbd["RGB-D geometry<br/>[rgbd.py]"]
  node_cameras["Camera models<br/>[cameras.py]"]
end

subgraph group_adapters["Execution adapters"]
  node_benchsession["Simulator session<br/>[session.py]"]
  node_benchserver["Benchmark server<br/>[armserver.py]"]
  node_libero["LIBERO adapter<br/>[server.py]"]
  node_robocasa["RoboCasa adapter<br/>[server.py]"]
  node_robotwin["RoboTwin adapter<br/>[server.py]"]
  node_xpl["RoboDojo bridge<br/>[session.py]"]
  node_real["Real robot session<br/>[session.py]"]
end

subgraph group_integrations["External systems"]
  node_sam["SAM 3 service<br/>[sam3_server.py]"]
end

node_user(("Coding agent"))
node_simulator["Benchmark simulator"]
node_robot["PiPER-X robots"]

node_user -->|"starts episode"| node_episode
node_episode -->|"runs conversation"| node_conversation
node_episode -->|"queries model"| node_llm
node_episode -->|"builds task card"| node_taskcard
node_episode -->|"writes records"| node_records
node_episode -->|"uses session"| node_session
node_episode -->|"runs cell"| node_kernelhost
node_kernelhost -->|"dispatches cell"| node_worker
node_worker -->|"executes code"| node_sandbox
node_robotapi -->|"delegates motion"| node_motion
node_motion -->|"plans paths"| node_ik
node_motion -->|"uses arm chain"| node_kinematics
node_robotapi -->|"serves perception"| node_scene
node_scene -->|"uses RGB-D geometry"| node_rgbd
node_scene -->|"uses camera models"| node_cameras
node_scene -->|"requests segmentation"| node_sam
node_xpl -->|"creates robot host"| node_robotapi
node_xpl -->|"configures perception"| node_scene
node_benchsession -->|"starts server"| node_benchserver
node_libero -->|"uses server base"| node_benchserver
node_robocasa -->|"uses server base"| node_benchserver
node_robotwin -->|"uses server base"| node_benchserver
node_simulator -.->|"runs benchmark"| node_benchserver
node_real -.->|"provides robot host"| node_robotapi
node_robot -.->|"receives control"| node_real
node_robotapi -->|"reads observations"| node_state

click node_episode "https://github.com/hexafuture/hexashell/blob/main/hexashell/runtime/episode.py"
click node_conversation "https://github.com/hexafuture/hexashell/blob/main/hexashell/runtime/conversation.py"
click node_llm "https://github.com/hexafuture/hexashell/blob/main/hexashell/runtime/llm.py"
click node_taskcard "https://github.com/hexafuture/hexashell/blob/main/hexashell/runtime/card.py"
click node_records "https://github.com/hexafuture/hexashell/blob/main/hexashell/runtime/records.py"
click node_session "https://github.com/hexafuture/hexashell/blob/main/hexashell/runtime/session.py"
click node_kernelhost "https://github.com/hexafuture/hexashell/blob/main/hexashell/kernel/host.py"
click node_worker "https://github.com/hexafuture/hexashell/blob/main/hexashell/kernel/worker.py"
click node_sandbox "https://github.com/hexafuture/hexashell/blob/main/hexashell/kernel/sandbox.py"
click node_robotapi "https://github.com/hexafuture/hexashell/blob/main/hexashell/robot/host.py"
click node_motion "https://github.com/hexafuture/hexashell/blob/main/hexashell/motion/driver.py"
click node_ik "https://github.com/hexafuture/hexashell/blob/main/hexashell/motion/ik.py"
click node_kinematics "https://github.com/hexafuture/hexashell/blob/main/hexashell/motion/chain.py"
click node_state "https://github.com/hexafuture/hexashell/blob/main/hexashell/state.py"
click node_scene "https://github.com/hexafuture/hexashell/blob/main/hexashell/vision/scene.py"
click node_rgbd "https://github.com/hexafuture/hexashell/blob/main/hexashell/vision/rgbd.py"
click node_cameras "https://github.com/hexafuture/hexashell/blob/main/hexashell/vision/cameras.py"
click node_sam "https://github.com/hexafuture/hexashell/blob/main/services/sam3_server.py"
click node_benchsession "https://github.com/hexafuture/hexashell/blob/main/hexashell/benches/session.py"
click node_benchserver "https://github.com/hexafuture/hexashell/blob/main/hexashell/benches/armserver.py"
click node_libero "https://github.com/hexafuture/hexashell/blob/main/hexashell/benches/libero/server.py"
click node_robocasa "https://github.com/hexafuture/hexashell/blob/main/hexashell/benches/robocasa/server.py"
click node_robotwin "https://github.com/hexafuture/hexashell/blob/main/hexashell/benches/robotwin/server.py"
click node_xpl "https://github.com/hexafuture/hexashell/blob/main/hexashell/xpl/session.py"
click node_real "https://github.com/hexafuture/hexashell/blob/main/hexashell/real/urai/session.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_episode,node_conversation,node_llm,node_taskcard,node_records,node_session,node_kernelhost,node_worker,node_sandbox toneBlue
class node_robotapi,node_motion,node_ik,node_kinematics,node_state toneAmber
class node_scene,node_rgbd,node_cameras toneMint
class node_benchsession,node_benchserver,node_libero,node_robocasa,node_robotwin,node_xpl,node_real toneRose
class node_sam,node_user,node_simulator,node_robot toneIndigo
```

## Components

### Agent episode

- [Episode loop](https://github.com/hexafuture/hexashell/blob/main/hexashell/runtime/episode.py): `hexashell/runtime/episode.py`
- [Conversation](https://github.com/hexafuture/hexashell/blob/main/hexashell/runtime/conversation.py): `hexashell/runtime/conversation.py`
- [Model client](https://github.com/hexafuture/hexashell/blob/main/hexashell/runtime/llm.py): `hexashell/runtime/llm.py`
- [Task card](https://github.com/hexafuture/hexashell/blob/main/hexashell/runtime/card.py): `hexashell/runtime/card.py`
- [Episode records](https://github.com/hexafuture/hexashell/blob/main/hexashell/runtime/records.py): `hexashell/runtime/records.py`
- [Session contract](https://github.com/hexafuture/hexashell/blob/main/hexashell/runtime/session.py): `hexashell/runtime/session.py`
- [Cell host](https://github.com/hexafuture/hexashell/blob/main/hexashell/kernel/host.py): `hexashell/kernel/host.py`
- [Sandbox worker](https://github.com/hexafuture/hexashell/blob/main/hexashell/kernel/worker.py): `hexashell/kernel/worker.py`
- [Python sandbox](https://github.com/hexafuture/hexashell/blob/main/hexashell/kernel/sandbox.py): `hexashell/kernel/sandbox.py`

### Robot tools

- [Robot API host](https://github.com/hexafuture/hexashell/blob/main/hexashell/robot/host.py): `hexashell/robot/host.py`
- [Motion planner](https://github.com/hexafuture/hexashell/blob/main/hexashell/motion/driver.py): `hexashell/motion/driver.py`
- [Inverse kinematics](https://github.com/hexafuture/hexashell/blob/main/hexashell/motion/ik.py): `hexashell/motion/ik.py`
- [Arm kinematics](https://github.com/hexafuture/hexashell/blob/main/hexashell/motion/chain.py): `hexashell/motion/chain.py`
- [Observation state](https://github.com/hexafuture/hexashell/blob/main/hexashell/state.py): `hexashell/state.py`

### Perception

- [Scene perception](https://github.com/hexafuture/hexashell/blob/main/hexashell/vision/scene.py): `hexashell/vision/scene.py`
- [RGB-D geometry](https://github.com/hexafuture/hexashell/blob/main/hexashell/vision/rgbd.py): `hexashell/vision/rgbd.py`
- [Camera models](https://github.com/hexafuture/hexashell/blob/main/hexashell/vision/cameras.py): `hexashell/vision/cameras.py`

### Execution adapters

- [Simulator session](https://github.com/hexafuture/hexashell/blob/main/hexashell/benches/session.py): `hexashell/benches/session.py`
- [Benchmark server](https://github.com/hexafuture/hexashell/blob/main/hexashell/benches/armserver.py): `hexashell/benches/armserver.py`
- [LIBERO adapter](https://github.com/hexafuture/hexashell/blob/main/hexashell/benches/libero/server.py): `hexashell/benches/libero/server.py`
- [RoboCasa adapter](https://github.com/hexafuture/hexashell/blob/main/hexashell/benches/robocasa/server.py): `hexashell/benches/robocasa/server.py`
- [RoboTwin adapter](https://github.com/hexafuture/hexashell/blob/main/hexashell/benches/robotwin/server.py): `hexashell/benches/robotwin/server.py`
- [RoboDojo bridge](https://github.com/hexafuture/hexashell/blob/main/hexashell/xpl/session.py): `hexashell/xpl/session.py`
- [Real robot session](https://github.com/hexafuture/hexashell/blob/main/hexashell/real/urai/session.py): `hexashell/real/urai/session.py`

### External systems

- [SAM 3 service](https://github.com/hexafuture/hexashell/blob/main/services/sam3_server.py): `services/sam3_server.py`

### Other components

- **Coding agent**
- **Benchmark simulator**
- **PiPER-X robots**

## Connections

- Coding agent → Episode loop: starts episode (evidence: `README.md`)
- Episode loop → Conversation: runs conversation (evidence: `hexashell/runtime/episode.py`)
- Episode loop → Model client: queries model (evidence: `hexashell/runtime/episode.py`)
- Episode loop → Task card: builds task card (evidence: `hexashell/runtime/episode.py`)
- Episode loop → Episode records: writes records (evidence: `hexashell/runtime/episode.py`)
- Episode loop → Session contract: uses session (evidence: `hexashell/runtime/episode.py`)
- Episode loop → Cell host: runs cell (evidence: `hexashell/runtime/episode.py`)
- Cell host → Sandbox worker: dispatches cell (evidence: `hexashell/kernel/host.py`)
- Sandbox worker → Python sandbox: executes code (evidence: `hexashell/kernel/worker.py`)
- Robot API host → Motion planner: delegates motion (evidence: `hexashell/robot/host.py`)
- Motion planner → Inverse kinematics: plans paths (evidence: `hexashell/motion/driver.py`)
- Motion planner → Arm kinematics: uses arm chain (evidence: `hexashell/motion/driver.py`)
- Robot API host → Scene perception: serves perception (evidence: `hexashell/robot/host.py`)
- Scene perception → RGB-D geometry: uses RGB-D geometry (evidence: `hexashell/vision/scene.py`)
- Scene perception → Camera models: uses camera models (evidence: `hexashell/vision/scene.py`)
- Scene perception → SAM 3 service: requests segmentation (evidence: `README.md`)
- RoboDojo bridge → Robot API host: creates robot host (evidence: `hexashell/xpl/session.py`)
- RoboDojo bridge → Scene perception: configures perception (evidence: `hexashell/xpl/session.py`)
- Simulator session → Benchmark server: starts server (evidence: `hexashell/benches/session.py`)
- LIBERO adapter → Benchmark server: uses server base (evidence: `hexashell/benches/libero/server.py`)
- RoboCasa adapter → Benchmark server: uses server base (evidence: `hexashell/benches/robocasa/server.py`)
- RoboTwin adapter → Benchmark server: uses server base (evidence: `hexashell/benches/robotwin/server.py`)
- Benchmark simulator → Benchmark server: runs benchmark
- Real robot session → Robot API host: provides robot host
- PiPER-X robots → Real robot session: receives control
- Robot API host → Observation state: reads observations (evidence: `hexashell/robot/host.py`)