How bhanot-99/semantic-neuroos works
Semantic-NeuroOS is a planned, on-device Linux desktop intelligence system: it senses consented activity, builds local semantic memory, answers spoken context-aware questions, and gates actions and web research for safety. The principal workflow runs from desktop sensing through filtering, embedding and local storage, then context/evidence assembly, local inference and spoken playback. Separate branches cover safe actions, approved external research, and out-of-process health monitoring. The graph combines README capabilities with the sampled runtime sources; several feature connections are documented but their implementation wiring was not sampled.
Main entry points: conversation.cpp Voice interaction · engine.rs Ingest and query · src/main.rs Approved URL fetch · src/main.rs Operator CLI
Components 25
Desktop sensing
- Context monitor
crates/neuroos-monitor/src/main.rs - Desktop sensors
crates/neuroos-monitor/src/sensors - Telemetry bus
crates/neuroos-monitor/src/bus.rs
Private memory
- Ingest and query
crates/neuroos-storage/src/engine.rs - Noise filtering
crates/neuroos-storage/src/ingest - Text embeddings
crates/neuroos-storage/src/embed.rs - SQLite metadata
crates/neuroos-storage/src/sqlite.rs - Vector memory
crates/neuroos-storage/src/lance.rs - Knowledge graph worker
python/neuroos-knowledge-background/src/neuroos_bg/main.py
Voice and reasoning
- Evidence assembly
crates/neuroos-knowledge-query/src/assemble.rs - Voice interaction
cpp/neuroos-voice/src/conversation.cpp - Voice capture and STT
cpp/neuroos-voice/src/stt.cpp - Inference service
cpp/neuroos-inference/src/server.cpp - Local model engine
cpp/neuroos-inference/src/engine.cpp - Speech playback
cpp/neuroos-voice/src/playback.cpp
Safety and research
- Skill safety kernel
crates/neuroos-kernel/src/main.rs - User confirmation
crates/neuroos-confirm/src/main.rs - Approved URL fetch
crates/neuroos-fetcher/src/main.rs
Operations
- Health monitor
crates/neuroos-healthd/src/main.rs - Health status API
crates/neuroos-healthd/src/server.rs - Operator CLI
crates/neuroosctl/src/main.rs
Other components
- Desktop user
- Internet
- NeuroOS services
- System operator
Connections 29
- Desktop user → Context monitor: consents to sensing
- Context monitor → Desktop sensors: starts sensors
- Desktop sensors → Telemetry bus: publish events
- Telemetry bus → Ingest and query: telemetry events
- Ingest and query → Noise filtering: filters events
- Ingest and query → SQLite metadata: writes metadata
- Ingest and query → Text embeddings: embeds chunks
- Ingest and query → Vector memory: stores vectors
- Evidence assembly → SQLite metadata: reads context
- Evidence assembly → Vector memory: retrieves evidence
- Desktop user → Voice interaction: asks by voice
- Voice interaction → Voice capture and STT: captures speech
- Voice capture and STT → Voice interaction: transcript
- Voice interaction → Evidence assembly: assembles context
- Voice interaction → Inference service: sends prompt
- Inference service → Local model engine: generates tokens
- Voice interaction → Speech playback: speaks answer
- Speech playback → Desktop user: audio answer
- Voice interaction → Skill safety kernel: requests skill
- Skill safety kernel → User confirmation: requests approval
- Desktop user → User confirmation: confirms action
- Desktop user → Approved URL fetch: approves URL
- Approved URL fetch → Internet: fetches URL
- Approved URL fetch → Voice interaction: returns tainted results
- NeuroOS services → Health monitor: health endpoints
- Health monitor → Health status API: serves snapshots
- System operator → Operator CLI: requests status
- Operator CLI → Health status API: queries status
- Operator CLI → Context monitor: pause or resume