# innate-inc/innate-os architecture

> Architecture diagram of the GitHub repository innate-inc/innate-os, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-08-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/innate-inc/innate-os
- Repository: https://github.com/innate-inc/innate-os

## Overview

#### Purpose and Runtime Entry Points
Innate OS is a ROS 2 Humble-based operating system for MARS robots, combining physical control, autonomous multimodal agents, skills, training, cloud integration, a browser control surface, and a MuJoCo digital twin. Primary operator entry points are the robot web interface, `innate` CLI, ROS launch files, and simulator CLI:
- `innate-sim`
- `sim/launcher/main.py`
- `scripts/launch_ros_in_tmux.sh`
- `ros2_ws/src/mars_bot/mars_bringup/launch/mars_bringup.launch.py`

#### ROS Workspace and Shared Contracts
`ros2_ws` is the runtime boundary: independently built ROS packages communicate through topics, services, and actions. Message packages define the control contracts between brain, manipulation, cloud, and robot components, isolating callers from implementation details.
- `ros2_ws/src/brain/brain_messages/action/ExecuteSkill.action`
- `ros2_ws/src/brain/brain_messages/srv/ReloadSkillsAgents.srv`
- `ros2_ws/src/cloud/innate_cloud_msgs/action/NavigateInstruction.action`
- `ros2_ws/src/mars_bot/mars_msgs/srv/GotoJSTrajectory.srv`

#### Robot Hardware, Perception, and Motion
The MARS platform layer brings up motors, base, IMU, LiDAR, battery, cameras, arm/head servos, and TF. Control accepts teleoperation or navigation velocity commands; arm services and KDL-style IK provide manipulation primitives. Camera pipelines publish stereo/depth data and can stream WebRTC remotely.
- `ros2_ws/src/mars_bot/mars_bringup/mars_bringup/bringup.py`
- `ros2_ws/src/mars_bot/mars_control/mars_control/app.cpp`
- `ros2_ws/src/mars_bot/mars_arm/mars_arm/arm_node.cpp`
- `ros2_ws/src/mars_bot/mars_cam/mars_cam/stereo_depth_estimator.cpp`
- `ros2_ws/src/mars_bot/mars_cam/mars_cam/webrtc_streamer.cpp`

#### Navigation and Manipulation Execution
Navigation wraps Nav2, mapping/localization modes, costmaps, behavior trees, command multiplexing, and custom goal/footprint behavior. Manipulation records demonstrations, manages tasks, replays episodes, and runs learned ACT/TensorRT policies. These are the primary physical-action backends invoked by skills and agents.
- `ros2_ws/src/mars_bot/mars_nav/mars_nav/mode_manager.py`
- `ros2_ws/src/mars_bot/mars_nav/mars_nav/navigate_to_pose_router.py`
- `ros2_ws/src/brain/manipulation/manipulation/manipulation_server.py`
- `ros2_ws/src/brain/manipulation/src/recorder_node.cpp`
- `ros2_ws/src/brain/manipulation/manipulation/act_trt.py`

#### Brain, Agents, Skills, Inputs, and Memory
`brain_client` is the autonomy orchestrator. It loads workspace-defined agents, skill packs, and input devices; gathers robot/perception state and memory; runs a model-facing agent loop; and invokes or interrupts skills through ROS-facing servers. Skills are Python extensions with injected mobility/manipulation/perception interfaces, while agents constrain allowed skills, inputs, and prompts.
- `ros2_ws/src/brain/brain_client/brain_client/nodes/brain_client_node.py`
- `ros2_ws/src/brain/brain_client/brain_client/brain/loop.py`
- `ros2_ws/src/brain/brain_client/brain_client/skills/runner.py`
- `ros2_ws/src/brain/brain_client/brain_client/inputs/manager.py`
- `workspace/innate_agents/basic_agent.py`
- `workspace/innate_skills/navigate_to_position.py`

#### Cloud, Training, and Observability Boundaries
Cloud packages authenticate and proxy external model/TTS services, upload logs, transfer episodes, manage training jobs, and deploy resulting skills. The brain includes Gemini and vision integration; proxy adapters expose OpenAI and Cartesia. These integrations remain clients around ROS nodes rather than replacing local robot control.
- `ros2_ws/src/cloud/clients/proxy-client/innate_proxy/client.py`
- `ros2_ws/src/cloud/innate_training_node/innate_training_node/node.py`
- `ros2_ws/src/cloud/innate_logger/innate_logger/logger_node.py`
- `ros2_ws/src/brain/brain_client/innate/gemini.py`

#### User Interfaces, Simulation, and Deployment
The static webapp uses rosbridge, WebRTC, and HTTPS proxy routes to operate the same ROS services/actions as mobile clients. The simulator runs the real ROS stack with simulated hardware adapters, Dockerized provisioning, a Python launcher, and a TypeScript 3D viewer connected through rosbridge. Physical deployment is systemd-managed with DDS/Zenoh, Bluetooth, networking, and device configuration.
- `webapp/js/rosClient.js`
- `webapp/proxy/https_server.py`
- `sim/launcher/runtime.py`
- `ros2_ws/src/mars_bot/mars_sim_driver/mars_sim_driver/node.py`
- `sim/viewer/src/physics/rosbridgeController.ts`
- `config/systemd/ros-app.service`
- `config/systemd/zenoh-router.service`

Main entry points: [`ros2_ws`](https://github.com/innate-inc/innate-os/tree/main/ros2_ws) (ROS 2 workspace), [`ros2_ws/src/brain/brain_client/brain_client/nodes/brain_client_node.py`](https://github.com/innate-inc/innate-os/blob/main/ros2_ws/src/brain/brain_client/brain_client/nodes/brain_client_node.py) (Brain client), [`ros2_ws/src/brain/brain_client/brain_client/skills/runner.py`](https://github.com/innate-inc/innate-os/blob/main/ros2_ws/src/brain/brain_client/brain_client/skills/runner.py) (Skill runner), [`ros2_ws/src/mars_bot/mars_bringup/mars_bringup/bringup.py`](https://github.com/innate-inc/innate-os/blob/main/ros2_ws/src/mars_bot/mars_bringup/mars_bringup/bringup.py) (Hardware bringup)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["Operator & Runtime Entry Points"]
  node_webapp["Browser control surface<br/>web client<br/>[rosClient.js]"]
  node_web_proxy["HTTPS proxy<br/>web gateway<br/>[https_server.py]"]
  node_ros_launch["MARS ROS launch<br/>runtime entry"]
end

subgraph group_ros["ROS Runtime &amp; Contracts"]
  node_ros_workspace{{"ROS 2 workspace<br/>runtime boundary"}}
  node_skill_contract["ExecuteSkill action<br/>ROS action contract"]
end

subgraph group_autonomy["Brain &amp; Skills"]
  node_brain_client{{"Brain client<br/>autonomy orchestrator"}}
  node_agent_loop["Model-facing agent loop<br/>agent runtime<br/>[loop.py]"]
  node_skill_runner["Skill runner<br/>skill executor<br/>[runner.py]"]
  node_agent_workspace["Workspace agents<br/>agent definitions<br/>[basic_agent.py]"]
  node_skill_workspace["Workspace skills<br/>skill extensions"]
end

subgraph group_robot["Robot Action Backends"]
  node_bringup["Hardware bringup<br/>platform node<br/>[bringup.py]"]
  node_motion_control["Base motion control<br/>control node<br/>[app.cpp]"]
  node_camera_pipeline["Stereo depth &amp; WebRTC<br/>perception node"]
  node_navigation["Navigation backend<br/>Nav2 integration"]
  node_manipulation["Manipulation backend<br/>action backend"]
end

subgraph group_cloud["Cloud Services"]
  node_model_integration["Gemini integration<br/>model client<br/>[gemini.py]"]
  node_training["Training &amp; deployment<br/>cloud ROS node<br/>[node.py]"]
end

subgraph group_sim["Simulation &amp; Deployment"]
  node_sim_runtime["Simulator runtime<br/>simulation launcher<br/>[runtime.py]"]
  node_sim_driver["Simulated hardware driver<br/>ROS adapter<br/>[node.py]"]
  node_sim_viewer["3D simulator viewer<br/>TypeScript viewer"]
  node_systemd["Physical deployment service<br/>systemd unit<br/>[ros-app.service]"]
end

node_webapp -->|"HTTPS routes"| node_web_proxy
node_webapp -->|"rosbridge"| node_ros_workspace
node_webapp -->|"WebRTC video"| node_camera_pipeline
node_ros_launch -->|"starts"| node_ros_workspace
node_ros_workspace -->|"defines"| node_skill_contract
node_ros_workspace -->|"runs"| node_bringup
node_bringup -->|"hardware interfaces"| node_motion_control
node_bringup -->|"camera devices"| node_camera_pipeline
node_brain_client -->|"runs"| node_agent_loop
node_brain_client -->|"loads"| node_agent_workspace
node_brain_client -->|"loads"| node_skill_workspace
node_brain_client -->|"dispatches"| node_skill_runner
node_agent_loop -->|"model requests"| node_model_integration
node_skill_runner -->|"executes"| node_skill_contract
node_skill_runner -->|"mobility skills"| node_navigation
node_skill_runner -->|"manipulation skills"| node_manipulation
node_navigation -->|"velocity commands"| node_motion_control
node_camera_pipeline -->|"perception state"| node_brain_client
node_manipulation -->|"episodes &amp; policies"| node_training
node_sim_runtime -->|"runs real stack"| node_ros_workspace
node_sim_runtime -->|"starts"| node_sim_driver
node_sim_driver -.->|"simulated devices"| node_bringup
node_sim_viewer -->|"rosbridge"| node_ros_workspace
node_systemd -->|"manages"| node_ros_launch

click node_webapp "https://github.com/innate-inc/innate-os/blob/main/webapp/js/rosClient.js"
click node_web_proxy "https://github.com/innate-inc/innate-os/blob/main/webapp/proxy/https_server.py"
click node_ros_launch "https://github.com/innate-inc/innate-os/blob/main/ros2_ws/src/mars_bot/mars_bringup/launch/mars_bringup.launch.py"
click node_ros_workspace "https://github.com/innate-inc/innate-os/tree/main/ros2_ws"
click node_skill_contract "https://github.com/innate-inc/innate-os/blob/main/ros2_ws/src/brain/brain_messages/action/ExecuteSkill.action"
click node_bringup "https://github.com/innate-inc/innate-os/blob/main/ros2_ws/src/mars_bot/mars_bringup/mars_bringup/bringup.py"
click node_motion_control "https://github.com/innate-inc/innate-os/blob/main/ros2_ws/src/mars_bot/mars_control/mars_control/app.cpp"
click node_camera_pipeline "https://github.com/innate-inc/innate-os/blob/main/ros2_ws/src/mars_bot/mars_cam/mars_cam/stereo_depth_estimator.cpp"
click node_navigation "https://github.com/innate-inc/innate-os/blob/main/ros2_ws/src/mars_bot/mars_nav/mars_nav/navigate_to_pose_router.py"
click node_manipulation "https://github.com/innate-inc/innate-os/blob/main/ros2_ws/src/brain/manipulation/manipulation/manipulation_server.py"
click node_brain_client "https://github.com/innate-inc/innate-os/blob/main/ros2_ws/src/brain/brain_client/brain_client/nodes/brain_client_node.py"
click node_agent_loop "https://github.com/innate-inc/innate-os/blob/main/ros2_ws/src/brain/brain_client/brain_client/brain/loop.py"
click node_skill_runner "https://github.com/innate-inc/innate-os/blob/main/ros2_ws/src/brain/brain_client/brain_client/skills/runner.py"
click node_agent_workspace "https://github.com/innate-inc/innate-os/blob/main/workspace/innate_agents/basic_agent.py"
click node_skill_workspace "https://github.com/innate-inc/innate-os/blob/main/workspace/innate_skills/navigate_to_position.py"
click node_model_integration "https://github.com/innate-inc/innate-os/blob/main/ros2_ws/src/brain/brain_client/innate/gemini.py"
click node_training "https://github.com/innate-inc/innate-os/blob/main/ros2_ws/src/cloud/innate_training_node/innate_training_node/node.py"
click node_sim_runtime "https://github.com/innate-inc/innate-os/blob/main/sim/launcher/runtime.py"
click node_sim_driver "https://github.com/innate-inc/innate-os/blob/main/ros2_ws/src/mars_bot/mars_sim_driver/mars_sim_driver/node.py"
click node_sim_viewer "https://github.com/innate-inc/innate-os/blob/main/sim/viewer/src/physics/rosbridgeController.ts"
click node_systemd "https://github.com/innate-inc/innate-os/blob/main/config/systemd/ros-app.service"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_webapp,node_web_proxy,node_ros_launch toneBlue
class node_ros_workspace,node_skill_contract toneAmber
class node_brain_client,node_agent_loop,node_skill_runner,node_agent_workspace,node_skill_workspace toneMint
class node_bringup,node_motion_control,node_camera_pipeline,node_navigation,node_manipulation toneRose
class node_model_integration,node_training toneIndigo
class node_sim_runtime,node_sim_driver,node_sim_viewer,node_systemd toneTeal
```

## Components

### Operator & Runtime Entry Points

- [Browser control surface](https://github.com/innate-inc/innate-os/blob/main/webapp/js/rosClient.js): web client · Operates ROS through rosbridge, WebRTC, and proxy routes. · `webapp/js/rosClient.js`
- [HTTPS proxy](https://github.com/innate-inc/innate-os/blob/main/webapp/proxy/https_server.py): web gateway · `webapp/proxy/https_server.py`
- [MARS ROS launch](https://github.com/innate-inc/innate-os/blob/main/ros2_ws/src/mars_bot/mars_bringup/launch/mars_bringup.launch.py): runtime entry · `ros2_ws/src/mars_bot/mars_bringup/launch/mars_bringup.launch.py`

### ROS Runtime & Contracts

- [ROS 2 workspace](https://github.com/innate-inc/innate-os/tree/main/ros2_ws): runtime boundary · Independently built packages communicate via topics, services, and actions. · `ros2_ws`
- [ExecuteSkill action](https://github.com/innate-inc/innate-os/blob/main/ros2_ws/src/brain/brain_messages/action/ExecuteSkill.action): ROS action contract · Contract for brain-directed skill execution. · `ros2_ws/src/brain/brain_messages/action/ExecuteSkill.action`

### Brain & Skills

- [Brain client](https://github.com/innate-inc/innate-os/blob/main/ros2_ws/src/brain/brain_client/brain_client/nodes/brain_client_node.py): autonomy orchestrator · Loads agents and skills, collects state and memory, and drives the agent loop. · `ros2_ws/src/brain/brain_client/brain_client/nodes/brain_client_node.py`
- [Model-facing agent loop](https://github.com/innate-inc/innate-os/blob/main/ros2_ws/src/brain/brain_client/brain_client/brain/loop.py): agent runtime · `ros2_ws/src/brain/brain_client/brain_client/brain/loop.py`
- [Skill runner](https://github.com/innate-inc/innate-os/blob/main/ros2_ws/src/brain/brain_client/brain_client/skills/runner.py): skill executor · Invokes and interrupts ROS-facing skills. · `ros2_ws/src/brain/brain_client/brain_client/skills/runner.py`
- [Workspace agents](https://github.com/innate-inc/innate-os/blob/main/workspace/innate_agents/basic_agent.py): agent definitions · Define prompts and permitted skills and inputs. · `workspace/innate_agents/basic_agent.py`
- [Workspace skills](https://github.com/innate-inc/innate-os/blob/main/workspace/innate_skills/navigate_to_position.py): skill extensions · Python skills receive mobility, manipulation, and perception interfaces. · `workspace/innate_skills/navigate_to_position.py`

### Robot Action Backends

- [Hardware bringup](https://github.com/innate-inc/innate-os/blob/main/ros2_ws/src/mars_bot/mars_bringup/mars_bringup/bringup.py): platform node · Brings up motors, sensors, cameras, TF, and robot devices. · `ros2_ws/src/mars_bot/mars_bringup/mars_bringup/bringup.py`
- [Base motion control](https://github.com/innate-inc/innate-os/blob/main/ros2_ws/src/mars_bot/mars_control/mars_control/app.cpp): control node · Accepts teleoperation and navigation velocity commands. · `ros2_ws/src/mars_bot/mars_control/mars_control/app.cpp`
- [Stereo depth & WebRTC](https://github.com/innate-inc/innate-os/blob/main/ros2_ws/src/mars_bot/mars_cam/mars_cam/stereo_depth_estimator.cpp): perception node · `ros2_ws/src/mars_bot/mars_cam/mars_cam/stereo_depth_estimator.cpp`
- [Navigation backend](https://github.com/innate-inc/innate-os/blob/main/ros2_ws/src/mars_bot/mars_nav/mars_nav/navigate_to_pose_router.py): Nav2 integration · Coordinates mapping, localization, costmaps, behavior trees, and goal routing. · `ros2_ws/src/mars_bot/mars_nav/mars_nav/navigate_to_pose_router.py`
- [Manipulation backend](https://github.com/innate-inc/innate-os/blob/main/ros2_ws/src/brain/manipulation/manipulation/manipulation_server.py): action backend · Executes demonstrations, replay, and learned ACT/TensorRT policies. · `ros2_ws/src/brain/manipulation/manipulation/manipulation_server.py`

### Cloud Services

- [Gemini integration](https://github.com/innate-inc/innate-os/blob/main/ros2_ws/src/brain/brain_client/innate/gemini.py): model client · `ros2_ws/src/brain/brain_client/innate/gemini.py`
- [Training & deployment](https://github.com/innate-inc/innate-os/blob/main/ros2_ws/src/cloud/innate_training_node/innate_training_node/node.py): cloud ROS node · Transfers episodes, manages training jobs, and deploys resulting skills. · `ros2_ws/src/cloud/innate_training_node/innate_training_node/node.py`

### Simulation & Deployment

- [Simulator runtime](https://github.com/innate-inc/innate-os/blob/main/sim/launcher/runtime.py): simulation launcher · Runs the real ROS stack with simulated hardware adapters. · `sim/launcher/runtime.py`
- [Simulated hardware driver](https://github.com/innate-inc/innate-os/blob/main/ros2_ws/src/mars_bot/mars_sim_driver/mars_sim_driver/node.py): ROS adapter · `ros2_ws/src/mars_bot/mars_sim_driver/mars_sim_driver/node.py`
- [3D simulator viewer](https://github.com/innate-inc/innate-os/blob/main/sim/viewer/src/physics/rosbridgeController.ts): TypeScript viewer · Connects to ROS through rosbridge. · `sim/viewer/src/physics/rosbridgeController.ts`
- [Physical deployment service](https://github.com/innate-inc/innate-os/blob/main/config/systemd/ros-app.service): systemd unit · Manages the ROS application on deployed robots. · `config/systemd/ros-app.service`

## Connections

- Browser control surface → HTTPS proxy: HTTPS routes
- Browser control surface → ROS 2 workspace: rosbridge
- Browser control surface → Stereo depth & WebRTC: WebRTC video
- MARS ROS launch → ROS 2 workspace: starts
- ROS 2 workspace → ExecuteSkill action: defines
- ROS 2 workspace → Hardware bringup: runs
- Hardware bringup → Base motion control: hardware interfaces
- Hardware bringup → Stereo depth & WebRTC: camera devices
- Brain client → Model-facing agent loop: runs
- Brain client → Workspace agents: loads
- Brain client → Workspace skills: loads
- Brain client → Skill runner: dispatches
- Model-facing agent loop → Gemini integration: model requests
- Skill runner → ExecuteSkill action: executes
- Skill runner → Navigation backend: mobility skills
- Skill runner → Manipulation backend: manipulation skills
- Navigation backend → Base motion control: velocity commands
- Stereo depth & WebRTC → Brain client: perception state
- Manipulation backend → Training & deployment: episodes & policies
- Simulator runtime → ROS 2 workspace: runs real stack
- Simulator runtime → Simulated hardware driver: starts
- Simulated hardware driver → Hardware bringup: simulated devices
- 3D simulator viewer → ROS 2 workspace: rosbridge
- Physical deployment service → MARS ROS launch: manages