How qqh0618/autonomous-os works
Autonomous OS is a robot operating system spanning setup and user-facing apps, agent runtimes, Go services, realtime voice, safety, and replaceable hardware drivers and boards. Its principal interaction is voice or chat input, intent handling or agent reasoning, safety validation, then a hardware action and spoken or visual response. The README documents the layered architecture and Harness, Buddy, and connector capabilities; source excerpts confirm selected setup, HAL, perception, and realtime-voice behavior, but do not establish wiring for every listed subsystem.
Main entry points: server OS Server · sensing_service.py Sensing Service · server.py HAL Runtime · routes HAL Routes
Components 22
Apps and Integrations
- Device Setup
system/web/src/pages/setup/useSetupController.ts - Harness Bridge
integrations/chat-bridges/autonomous-chat-hook - Autonomous Buddy
integrations/companions/autonomous-buddy - Service Connectors
integrations
Agent Services
- OS Server
system/server - Agent Runtime
- Jev Intent
hal/realtime/find_intent.py
Voice and Perception
- Realtime Orchestrator
hal/realtime/orchestrator.py - Voice Pipeline
hal/drivers/voice/voice_service.py - Sensing Service
hal/drivers/sensing/sensing_service.py - Perception Pipeline
hal/drivers/sensing/perceptions/orchestrator.py - User Tracking
hal/drivers/tracking/tracker_service.py
Robot Runtime
- Safety Policy
hal/safety/policy.py - HAL Runtime
hal/server.py - HAL Routes
hal/routes - Policy Service
hal/policy/service.py
Hardware
- Hardware Drivers
hal/drivers - Board Support
hal/board
Other components
- Person
- Robot Hardware
- Model Providers
- Paired Computer
Connections 25
- Person → Device Setup: configures
- Device Setup → OS Server: submits setup
- Person → OS Server: voice or chat
- OS Server → Jev Intent: checks intent
- Jev Intent → OS Server: returns command
- OS Server → Agent Runtime: delegates request
- Agent Runtime → Model Providers: requests inference
- OS Server → Realtime Orchestrator: routes realtime
- Realtime Orchestrator → Voice Pipeline: uses voice service
- Voice Pipeline → Model Providers: speech services
- HAL Runtime → Sensing Service: starts sensing
- Sensing Service → Perception Pipeline: processes sensor data
- Sensing Service → User Tracking: tracks users
- OS Server → Safety Policy: validates action
- OS Server → HAL Runtime: sends hardware request
- HAL Runtime → HAL Routes: serves capabilities
- HAL Routes → Policy Service: checks policy
- HAL Routes → Hardware Drivers: dispatches action
- Hardware Drivers → Board Support: uses board support
- Board Support → Robot Hardware: controls hardware
- Agent Runtime → Harness Bridge: delegates digital tasks
- Harness Bridge → Paired Computer: connects paired agents
- Agent Runtime → Autonomous Buddy: requests computer use
- Autonomous Buddy → Paired Computer: operates Mac apps
- Agent Runtime → Service Connectors: uses linked services