# bhanot-99/semantic-neuroos architecture

> Architecture diagram of the GitHub repository bhanot-99/semantic-neuroos, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bhanot-99/semantic-neuroos
- Repository: https://github.com/bhanot-99/semantic-neuroos

## Overview

Semantic-NeuroOS is a planned, on-device Linux desktop intelligence system: it senses consented activity, builds local semantic memory, answers spoken context-aware questions, and gates actions and web research for safety. The principal workflow runs from desktop sensing through filtering, embedding and local storage, then context/evidence assembly, local inference and spoken playback. Separate branches cover safe actions, approved external research, and out-of-process health monitoring. The graph combines README capabilities with the sampled runtime sources; several feature connections are documented but their implementation wiring was not sampled.

Main entry points: [`cpp/neuroos-voice/src/conversation.cpp`](https://github.com/bhanot-99/semantic-neuroos/blob/main/cpp/neuroos-voice/src/conversation.cpp) (Voice interaction), [`crates/neuroos-storage/src/engine.rs`](https://github.com/bhanot-99/semantic-neuroos/blob/main/crates/neuroos-storage/src/engine.rs) (Ingest and query), [`crates/neuroos-fetcher/src/main.rs`](https://github.com/bhanot-99/semantic-neuroos/blob/main/crates/neuroos-fetcher/src/main.rs) (Approved URL fetch), [`crates/neuroosctl/src/main.rs`](https://github.com/bhanot-99/semantic-neuroos/blob/main/crates/neuroosctl/src/main.rs) (Operator CLI)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_sense["Desktop sensing"]
  node_monitor["Context monitor<br/>[main.rs]"]
  node_sensors["Desktop sensors"]
  node_event_bus["Telemetry bus<br/>[bus.rs]"]
end

subgraph group_memory["Private memory"]
  node_storage_engine["Ingest and query<br/>[engine.rs]"]
  node_ingest["Noise filtering"]
  node_embedder["Text embeddings<br/>[embed.rs]"]
  node_sqlite[("SQLite metadata<br/>[sqlite.rs]")]
  node_lance[("Vector memory<br/>[lance.rs]")]
  node_knowledge_background["Knowledge graph worker<br/>[main.py]"]
end

subgraph group_voice["Voice and reasoning"]
  node_knowledge_query["Evidence assembly<br/>[assemble.rs]"]
  node_voice["Voice interaction<br/>[conversation.cpp]"]
  node_voice_input["Voice capture and STT<br/>[stt.cpp]"]
  node_inference_server["Inference service<br/>[server.cpp]"]
  node_inference_engine["Local model engine<br/>[engine.cpp]"]
  node_voice_output["Speech playback<br/>[playback.cpp]"]
end

subgraph group_safety["Safety and research"]
  node_kernel["Skill safety kernel<br/>[main.rs]"]
  node_confirmation["User confirmation<br/>[main.rs]"]
  node_fetcher["Approved URL fetch<br/>[main.rs]"]
end

subgraph group_operations["Operations"]
  node_healthd["Health monitor<br/>[main.rs]"]
  node_health_server["Health status API<br/>[server.rs]"]
  node_cli["Operator CLI<br/>[main.rs]"]
end

node_user(("Desktop user"))
node_internet(("Internet"))
node_components(("NeuroOS services"))
node_operator(("System operator"))

node_user -->|"consents to sensing"| node_monitor
node_monitor -->|"starts sensors"| node_sensors
node_sensors -->|"publish events"| node_event_bus
node_event_bus -->|"telemetry events"| node_storage_engine
node_storage_engine -->|"filters events"| node_ingest
node_storage_engine -->|"writes metadata"| node_sqlite
node_storage_engine -->|"embeds chunks"| node_embedder
node_storage_engine -->|"stores vectors"| node_lance
node_knowledge_query -->|"reads context"| node_sqlite
node_knowledge_query -->|"retrieves evidence"| node_lance
node_user -->|"asks by voice"| node_voice
node_voice -->|"captures speech"| node_voice_input
node_voice_input -->|"transcript"| node_voice
node_voice -->|"assembles context"| node_knowledge_query
node_voice -->|"sends prompt"| node_inference_server
node_inference_server -->|"generates tokens"| node_inference_engine
node_voice -->|"speaks answer"| node_voice_output
node_voice_output -->|"audio answer"| node_user
node_voice -->|"requests skill"| node_kernel
node_kernel -->|"requests approval"| node_confirmation
node_user -->|"confirms action"| node_confirmation
node_user -->|"approves URL"| node_fetcher
node_fetcher -.->|"fetches URL"| node_internet
node_fetcher -.->|"returns tainted results"| node_voice
node_components -->|"health endpoints"| node_healthd
node_healthd -->|"serves snapshots"| node_health_server
node_operator -->|"requests status"| node_cli
node_cli -->|"queries status"| node_health_server
node_cli -->|"pause or resume"| node_monitor

click node_monitor "https://github.com/bhanot-99/semantic-neuroos/blob/main/crates/neuroos-monitor/src/main.rs"
click node_sensors "https://github.com/bhanot-99/semantic-neuroos/tree/main/crates/neuroos-monitor/src/sensors"
click node_event_bus "https://github.com/bhanot-99/semantic-neuroos/blob/main/crates/neuroos-monitor/src/bus.rs"
click node_storage_engine "https://github.com/bhanot-99/semantic-neuroos/blob/main/crates/neuroos-storage/src/engine.rs"
click node_ingest "https://github.com/bhanot-99/semantic-neuroos/tree/main/crates/neuroos-storage/src/ingest"
click node_embedder "https://github.com/bhanot-99/semantic-neuroos/blob/main/crates/neuroos-storage/src/embed.rs"
click node_sqlite "https://github.com/bhanot-99/semantic-neuroos/blob/main/crates/neuroos-storage/src/sqlite.rs"
click node_lance "https://github.com/bhanot-99/semantic-neuroos/blob/main/crates/neuroos-storage/src/lance.rs"
click node_knowledge_query "https://github.com/bhanot-99/semantic-neuroos/blob/main/crates/neuroos-knowledge-query/src/assemble.rs"
click node_knowledge_background "https://github.com/bhanot-99/semantic-neuroos/blob/main/python/neuroos-knowledge-background/src/neuroos_bg/main.py"
click node_voice "https://github.com/bhanot-99/semantic-neuroos/blob/main/cpp/neuroos-voice/src/conversation.cpp"
click node_voice_input "https://github.com/bhanot-99/semantic-neuroos/blob/main/cpp/neuroos-voice/src/stt.cpp"
click node_inference_server "https://github.com/bhanot-99/semantic-neuroos/blob/main/cpp/neuroos-inference/src/server.cpp"
click node_inference_engine "https://github.com/bhanot-99/semantic-neuroos/blob/main/cpp/neuroos-inference/src/engine.cpp"
click node_voice_output "https://github.com/bhanot-99/semantic-neuroos/blob/main/cpp/neuroos-voice/src/playback.cpp"
click node_kernel "https://github.com/bhanot-99/semantic-neuroos/blob/main/crates/neuroos-kernel/src/main.rs"
click node_confirmation "https://github.com/bhanot-99/semantic-neuroos/blob/main/crates/neuroos-confirm/src/main.rs"
click node_fetcher "https://github.com/bhanot-99/semantic-neuroos/blob/main/crates/neuroos-fetcher/src/main.rs"
click node_healthd "https://github.com/bhanot-99/semantic-neuroos/blob/main/crates/neuroos-healthd/src/main.rs"
click node_health_server "https://github.com/bhanot-99/semantic-neuroos/blob/main/crates/neuroos-healthd/src/server.rs"
click node_cli "https://github.com/bhanot-99/semantic-neuroos/blob/main/crates/neuroosctl/src/main.rs"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_monitor,node_sensors,node_event_bus,node_user toneBlue
class node_storage_engine,node_ingest,node_embedder,node_sqlite,node_lance,node_knowledge_background toneAmber
class node_knowledge_query,node_voice,node_voice_input,node_inference_server,node_inference_engine,node_voice_output toneMint
class node_kernel,node_confirmation,node_fetcher toneRose
class node_healthd,node_health_server,node_cli,node_internet,node_components,node_operator toneIndigo
```

## Components

### Desktop sensing

- [Context monitor](https://github.com/bhanot-99/semantic-neuroos/blob/main/crates/neuroos-monitor/src/main.rs): `crates/neuroos-monitor/src/main.rs`
- [Desktop sensors](https://github.com/bhanot-99/semantic-neuroos/tree/main/crates/neuroos-monitor/src/sensors): `crates/neuroos-monitor/src/sensors`
- [Telemetry bus](https://github.com/bhanot-99/semantic-neuroos/blob/main/crates/neuroos-monitor/src/bus.rs): `crates/neuroos-monitor/src/bus.rs`

### Private memory

- [Ingest and query](https://github.com/bhanot-99/semantic-neuroos/blob/main/crates/neuroos-storage/src/engine.rs): `crates/neuroos-storage/src/engine.rs`
- [Noise filtering](https://github.com/bhanot-99/semantic-neuroos/tree/main/crates/neuroos-storage/src/ingest): `crates/neuroos-storage/src/ingest`
- [Text embeddings](https://github.com/bhanot-99/semantic-neuroos/blob/main/crates/neuroos-storage/src/embed.rs): `crates/neuroos-storage/src/embed.rs`
- [SQLite metadata](https://github.com/bhanot-99/semantic-neuroos/blob/main/crates/neuroos-storage/src/sqlite.rs): `crates/neuroos-storage/src/sqlite.rs`
- [Vector memory](https://github.com/bhanot-99/semantic-neuroos/blob/main/crates/neuroos-storage/src/lance.rs): `crates/neuroos-storage/src/lance.rs`
- [Knowledge graph worker](https://github.com/bhanot-99/semantic-neuroos/blob/main/python/neuroos-knowledge-background/src/neuroos_bg/main.py): `python/neuroos-knowledge-background/src/neuroos_bg/main.py`

### Voice and reasoning

- [Evidence assembly](https://github.com/bhanot-99/semantic-neuroos/blob/main/crates/neuroos-knowledge-query/src/assemble.rs): `crates/neuroos-knowledge-query/src/assemble.rs`
- [Voice interaction](https://github.com/bhanot-99/semantic-neuroos/blob/main/cpp/neuroos-voice/src/conversation.cpp): `cpp/neuroos-voice/src/conversation.cpp`
- [Voice capture and STT](https://github.com/bhanot-99/semantic-neuroos/blob/main/cpp/neuroos-voice/src/stt.cpp): `cpp/neuroos-voice/src/stt.cpp`
- [Inference service](https://github.com/bhanot-99/semantic-neuroos/blob/main/cpp/neuroos-inference/src/server.cpp): `cpp/neuroos-inference/src/server.cpp`
- [Local model engine](https://github.com/bhanot-99/semantic-neuroos/blob/main/cpp/neuroos-inference/src/engine.cpp): `cpp/neuroos-inference/src/engine.cpp`
- [Speech playback](https://github.com/bhanot-99/semantic-neuroos/blob/main/cpp/neuroos-voice/src/playback.cpp): `cpp/neuroos-voice/src/playback.cpp`

### Safety and research

- [Skill safety kernel](https://github.com/bhanot-99/semantic-neuroos/blob/main/crates/neuroos-kernel/src/main.rs): `crates/neuroos-kernel/src/main.rs`
- [User confirmation](https://github.com/bhanot-99/semantic-neuroos/blob/main/crates/neuroos-confirm/src/main.rs): `crates/neuroos-confirm/src/main.rs`
- [Approved URL fetch](https://github.com/bhanot-99/semantic-neuroos/blob/main/crates/neuroos-fetcher/src/main.rs): `crates/neuroos-fetcher/src/main.rs`

### Operations

- [Health monitor](https://github.com/bhanot-99/semantic-neuroos/blob/main/crates/neuroos-healthd/src/main.rs): `crates/neuroos-healthd/src/main.rs`
- [Health status API](https://github.com/bhanot-99/semantic-neuroos/blob/main/crates/neuroos-healthd/src/server.rs): `crates/neuroos-healthd/src/server.rs`
- [Operator CLI](https://github.com/bhanot-99/semantic-neuroos/blob/main/crates/neuroosctl/src/main.rs): `crates/neuroosctl/src/main.rs`

### Other components

- **Desktop user**
- **Internet**
- **NeuroOS services**
- **System operator**

## Connections

- Desktop user → Context monitor: consents to sensing
- Context monitor → Desktop sensors: starts sensors
- Desktop sensors → Telemetry bus: publish events
- Telemetry bus → Ingest and query: telemetry events
- Ingest and query → Noise filtering: filters events
- Ingest and query → SQLite metadata: writes metadata
- Ingest and query → Text embeddings: embeds chunks
- Ingest and query → Vector memory: stores vectors
- Evidence assembly → SQLite metadata: reads context
- Evidence assembly → Vector memory: retrieves evidence
- Desktop user → Voice interaction: asks by voice
- Voice interaction → Voice capture and STT: captures speech
- Voice capture and STT → Voice interaction: transcript
- Voice interaction → Evidence assembly: assembles context
- Voice interaction → Inference service: sends prompt
- Inference service → Local model engine: generates tokens
- Voice interaction → Speech playback: speaks answer
- Speech playback → Desktop user: audio answer
- Voice interaction → Skill safety kernel: requests skill
- Skill safety kernel → User confirmation: requests approval
- Desktop user → User confirmation: confirms action
- Desktop user → Approved URL fetch: approves URL
- Approved URL fetch → Internet: fetches URL
- Approved URL fetch → Voice interaction: returns tainted results
- NeuroOS services → Health monitor: health endpoints
- Health monitor → Health status API: serves snapshots
- System operator → Operator CLI: requests status
- Operator CLI → Health status API: queries status
- Operator CLI → Context monitor: pause or resume