# qqh0618/autonomous-os architecture

> Architecture diagram of the GitHub repository qqh0618/autonomous-os, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/qqh0618/autonomous-os
- Repository: https://github.com/qqh0618/autonomous-os

## Overview

Autonomous OS is a robot operating system spanning setup and user-facing apps, agent runtimes, Go services, realtime voice, safety, and replaceable hardware drivers and boards. Its principal interaction is voice or chat input, intent handling or agent reasoning, safety validation, then a hardware action and spoken or visual response. The README documents the layered architecture and Harness, Buddy, and connector capabilities; source excerpts confirm selected setup, HAL, perception, and realtime-voice behavior, but do not establish wiring for every listed subsystem.

Main entry points: [`system/server`](https://github.com/qqh0618/autonomous-os/tree/main/system/server) (OS Server), [`hal/drivers/sensing/sensing_service.py`](https://github.com/qqh0618/autonomous-os/blob/main/hal/drivers/sensing/sensing_service.py) (Sensing Service), [`hal/server.py`](https://github.com/qqh0618/autonomous-os/blob/main/hal/server.py) (HAL Runtime), [`hal/routes`](https://github.com/qqh0618/autonomous-os/tree/main/hal/routes) (HAL Routes)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_apps["Apps and Integrations"]
  node_setup["Device Setup"]
  node_harness["Harness Bridge"]
  node_buddy["Autonomous Buddy"]
  node_connectors["Service Connectors"]
end

subgraph group_agent["Agent Services"]
  node_osapp["OS Server"]
  node_agent_runtime["Agent Runtime"]
  node_jev["Jev Intent<br/>[find_intent.py]"]
end

subgraph group_interaction["Voice and Perception"]
  node_realtime["Realtime Orchestrator<br/>[orchestrator.py]"]
  node_voice["Voice Pipeline<br/>[voice_service.py]"]
  node_sensing["Sensing Service<br/>[sensing_service.py]"]
  node_perception["Perception Pipeline<br/>[orchestrator.py]"]
  node_tracking["User Tracking<br/>[tracker_service.py]"]
end

subgraph group_robot["Robot Runtime"]
  node_safety["Safety Policy<br/>[policy.py]"]
  node_hal["HAL Runtime<br/>[server.py]"]
  node_routes["HAL Routes"]
  node_policy["Policy Service<br/>[service.py]"]
end

subgraph group_hardware["Hardware"]
  node_drivers["Hardware Drivers"]
  node_boards["Board Support"]
end

node_person(("Person"))
node_robot_body{{"Robot Hardware"}}
node_cloud(("Model Providers"))
node_computer(("Paired Computer"))

node_person -->|"configures"| node_setup
node_setup -->|"submits setup"| node_osapp
node_person -->|"voice or chat"| node_osapp
node_osapp -->|"checks intent"| node_jev
node_jev -->|"returns command"| node_osapp
node_osapp -->|"delegates request"| node_agent_runtime
node_agent_runtime -.->|"requests inference"| node_cloud
node_osapp -->|"routes realtime"| node_realtime
node_realtime -->|"uses voice service"| node_voice
node_voice -.->|"speech services"| node_cloud
node_hal -->|"starts sensing"| node_sensing
node_sensing -->|"processes sensor data"| node_perception
node_sensing -->|"tracks users"| node_tracking
node_osapp -->|"validates action"| node_safety
node_osapp -->|"sends hardware request"| node_hal
node_hal -->|"serves capabilities"| node_routes
node_routes -->|"checks policy"| node_policy
node_routes -->|"dispatches action"| node_drivers
node_drivers -->|"uses board support"| node_boards
node_boards -->|"controls hardware"| node_robot_body
node_agent_runtime -.->|"delegates digital tasks"| node_harness
node_harness -.->|"connects paired agents"| node_computer
node_agent_runtime -.->|"requests computer use"| node_buddy
node_buddy -.->|"operates Mac apps"| node_computer
node_agent_runtime -.->|"uses linked services"| node_connectors

click node_setup "https://github.com/qqh0618/autonomous-os/blob/main/system/web/src/pages/setup/useSetupController.ts"
click node_osapp "https://github.com/qqh0618/autonomous-os/tree/main/system/server"
click node_jev "https://github.com/qqh0618/autonomous-os/blob/main/hal/realtime/find_intent.py"
click node_harness "https://github.com/qqh0618/autonomous-os/tree/main/integrations/chat-bridges/autonomous-chat-hook"
click node_buddy "https://github.com/qqh0618/autonomous-os/tree/main/integrations/companions/autonomous-buddy"
click node_connectors "https://github.com/qqh0618/autonomous-os/tree/main/integrations"
click node_realtime "https://github.com/qqh0618/autonomous-os/blob/main/hal/realtime/orchestrator.py"
click node_voice "https://github.com/qqh0618/autonomous-os/blob/main/hal/drivers/voice/voice_service.py"
click node_sensing "https://github.com/qqh0618/autonomous-os/blob/main/hal/drivers/sensing/sensing_service.py"
click node_perception "https://github.com/qqh0618/autonomous-os/blob/main/hal/drivers/sensing/perceptions/orchestrator.py"
click node_tracking "https://github.com/qqh0618/autonomous-os/blob/main/hal/drivers/tracking/tracker_service.py"
click node_safety "https://github.com/qqh0618/autonomous-os/blob/main/hal/safety/policy.py"
click node_hal "https://github.com/qqh0618/autonomous-os/blob/main/hal/server.py"
click node_routes "https://github.com/qqh0618/autonomous-os/tree/main/hal/routes"
click node_policy "https://github.com/qqh0618/autonomous-os/blob/main/hal/policy/service.py"
click node_drivers "https://github.com/qqh0618/autonomous-os/tree/main/hal/drivers"
click node_boards "https://github.com/qqh0618/autonomous-os/tree/main/hal/board"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_setup,node_harness,node_buddy,node_connectors toneBlue
class node_osapp,node_agent_runtime,node_jev toneAmber
class node_realtime,node_voice,node_sensing,node_perception,node_tracking toneMint
class node_safety,node_hal,node_routes,node_policy toneRose
class node_drivers,node_boards,node_person,node_robot_body,node_cloud,node_computer toneIndigo
```

## Components

### Apps and Integrations

- [Device Setup](https://github.com/qqh0618/autonomous-os/blob/main/system/web/src/pages/setup/useSetupController.ts): `system/web/src/pages/setup/useSetupController.ts`
- [Harness Bridge](https://github.com/qqh0618/autonomous-os/tree/main/integrations/chat-bridges/autonomous-chat-hook): `integrations/chat-bridges/autonomous-chat-hook`
- [Autonomous Buddy](https://github.com/qqh0618/autonomous-os/tree/main/integrations/companions/autonomous-buddy): `integrations/companions/autonomous-buddy`
- [Service Connectors](https://github.com/qqh0618/autonomous-os/tree/main/integrations): `integrations`

### Agent Services

- [OS Server](https://github.com/qqh0618/autonomous-os/tree/main/system/server): `system/server`
- **Agent Runtime**
- [Jev Intent](https://github.com/qqh0618/autonomous-os/blob/main/hal/realtime/find_intent.py): `hal/realtime/find_intent.py`

### Voice and Perception

- [Realtime Orchestrator](https://github.com/qqh0618/autonomous-os/blob/main/hal/realtime/orchestrator.py): `hal/realtime/orchestrator.py`
- [Voice Pipeline](https://github.com/qqh0618/autonomous-os/blob/main/hal/drivers/voice/voice_service.py): `hal/drivers/voice/voice_service.py`
- [Sensing Service](https://github.com/qqh0618/autonomous-os/blob/main/hal/drivers/sensing/sensing_service.py): `hal/drivers/sensing/sensing_service.py`
- [Perception Pipeline](https://github.com/qqh0618/autonomous-os/blob/main/hal/drivers/sensing/perceptions/orchestrator.py): `hal/drivers/sensing/perceptions/orchestrator.py`
- [User Tracking](https://github.com/qqh0618/autonomous-os/blob/main/hal/drivers/tracking/tracker_service.py): `hal/drivers/tracking/tracker_service.py`

### Robot Runtime

- [Safety Policy](https://github.com/qqh0618/autonomous-os/blob/main/hal/safety/policy.py): `hal/safety/policy.py`
- [HAL Runtime](https://github.com/qqh0618/autonomous-os/blob/main/hal/server.py): `hal/server.py`
- [HAL Routes](https://github.com/qqh0618/autonomous-os/tree/main/hal/routes): `hal/routes`
- [Policy Service](https://github.com/qqh0618/autonomous-os/blob/main/hal/policy/service.py): `hal/policy/service.py`

### Hardware

- [Hardware Drivers](https://github.com/qqh0618/autonomous-os/tree/main/hal/drivers): `hal/drivers`
- [Board Support](https://github.com/qqh0618/autonomous-os/tree/main/hal/board): `hal/board`

### Other components

- **Person**
- **Robot Hardware**
- **Model Providers**
- **Paired Computer**

## Connections

- Person → Device Setup: configures
- Device Setup → OS Server: submits setup
- Person → OS Server: voice or chat
- OS Server → Jev Intent: checks intent
- Jev Intent → OS Server: returns command
- OS Server → Agent Runtime: delegates request
- Agent Runtime → Model Providers: requests inference
- OS Server → Realtime Orchestrator: routes realtime
- Realtime Orchestrator → Voice Pipeline: uses voice service
- Voice Pipeline → Model Providers: speech services
- HAL Runtime → Sensing Service: starts sensing
- Sensing Service → Perception Pipeline: processes sensor data
- Sensing Service → User Tracking: tracks users
- OS Server → Safety Policy: validates action
- OS Server → HAL Runtime: sends hardware request
- HAL Runtime → HAL Routes: serves capabilities
- HAL Routes → Policy Service: checks policy
- HAL Routes → Hardware Drivers: dispatches action
- Hardware Drivers → Board Support: uses board support
- Board Support → Robot Hardware: controls hardware
- Agent Runtime → Harness Bridge: delegates digital tasks
- Harness Bridge → Paired Computer: connects paired agents
- Agent Runtime → Autonomous Buddy: requests computer use
- Autonomous Buddy → Paired Computer: operates Mac apps
- Agent Runtime → Service Connectors: uses linked services