# insightos-community/semantic-framework architecture

> Architecture diagram of the GitHub repository insightos-community/semantic-framework, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/insightos-community/semantic-framework
- Repository: https://github.com/insightos-community/semantic-framework

## Overview

InsightOS Semantic is a framework for embodied AI and robotic applications. Its principal workflow is a user request entering the Semantic Server, where an agent interprets and plans the task, workflow orchestration coordinates execution, and robot services dispatch work to Pilots and robot runtimes. Execution feedback and operational records support monitoring and continued capability development. The README documents major capabilities including collaboration, semantic world models, simulation, and skill evolution. Some internal relationships are documented only at a high level; the graph avoids asserting unsampled call paths.

Main entry points: [`internal/agent/runtime/service.go`](https://github.com/insightos-community/semantic-framework/blob/main/internal/agent/runtime/service.go) (Agent runtime), [`internal/server/http`](https://github.com/insightos-community/semantic-framework/tree/main/internal/server/http) (HTTP and WS API), [`internal/robot/service.go`](https://github.com/insightos-community/semantic-framework/blob/main/internal/robot/service.go) (Robot service), [`internal/workflow/scheduler.go`](https://github.com/insightos-community/semantic-framework/blob/main/internal/workflow/scheduler.go) (Workflow scheduler)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_server["Application interaction"]
  node_http["HTTP and WS API"]
  node_auth["Authentication"]
end

subgraph group_orchestration["Task orchestration"]
  node_agent["Agent runtime<br/>[service.go]"]
  node_kernel["Agent kernel"]
  node_workflow["Workflow scheduler<br/>[scheduler.go]"]
  node_interaction["Human interaction<br/>[service.go]"]
end

subgraph group_robot["Robot execution"]
  node_robotservice["Robot service<br/>[service.go]"]
  node_pilot["Semantic Pilot<br/>[main.go]"]
  node_robotruntime["Robot runtime<br/>[orchestrator.go]"]
  node_skills["Robot Skills<br/>[store.go]"]
end

subgraph group_world["World and simulation"]
  node_map["Semantic world model<br/>[map_query.go]"]
  node_simulation["Simulation services<br/>[service.go]"]
end

subgraph group_platform["Shared platform"]
  node_mcp["MCP integrations<br/>[registry.go]"]
  node_store[("Persistent state")]
end

node_user(("Studio user"))
node_studio["Semantic Studio"]
node_models["Model providers"]
node_devices["Robots and devices"]

node_user -->|"describes goal"| node_studio
node_studio -->|"sends requests"| node_http
node_http -->|"routes auth"| node_auth
node_http -.->|"routes requests"| node_agent
node_agent -->|"runs agent"| node_kernel
node_agent -->|"requests reasoning"| node_models
node_agent -->|"organizes tasks"| node_workflow
node_agent -.->|"requests approval"| node_interaction
node_workflow -->|"coordinates robot work"| node_robotservice
node_robotservice -->|"dispatches actions"| node_pilot
node_pilot -->|"runs capabilities"| node_robotruntime
node_robotruntime -->|"connects devices"| node_devices
node_robotservice -->|"manages skills"| node_skills
node_agent -.->|"queries semantics"| node_map
node_http -->|"routes simulation"| node_simulation
node_agent -->|"uses tools"| node_mcp
node_agent -->|"persists runtime state"| node_store
node_workflow -->|"reads and writes"| node_store
node_robotservice -->|"records execution"| node_store

click node_http "https://github.com/insightos-community/semantic-framework/tree/main/internal/server/http"
click node_auth "https://github.com/insightos-community/semantic-framework/tree/main/internal/server/auth"
click node_agent "https://github.com/insightos-community/semantic-framework/blob/main/internal/agent/runtime/service.go"
click node_kernel "https://github.com/insightos-community/semantic-framework/tree/main/internal/agent/kernel"
click node_workflow "https://github.com/insightos-community/semantic-framework/blob/main/internal/workflow/scheduler.go"
click node_interaction "https://github.com/insightos-community/semantic-framework/blob/main/internal/interaction/service.go"
click node_robotservice "https://github.com/insightos-community/semantic-framework/blob/main/internal/robot/service.go"
click node_pilot "https://github.com/insightos-community/semantic-framework/blob/main/cmd/semantic-pilot/main.go"
click node_robotruntime "https://github.com/insightos-community/semantic-framework/blob/main/internal/robotruntime/orchestrator.go"
click node_skills "https://github.com/insightos-community/semantic-framework/blob/main/internal/skill/store.go"
click node_map "https://github.com/insightos-community/semantic-framework/blob/main/internal/tool/builtin/map_query.go"
click node_simulation "https://github.com/insightos-community/semantic-framework/blob/main/internal/simulation/service.go"
click node_mcp "https://github.com/insightos-community/semantic-framework/blob/main/internal/mcpregistry/registry.go"
click node_store "https://github.com/insightos-community/semantic-framework/tree/main/internal/store"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_http,node_auth,node_user toneBlue
class node_agent,node_kernel,node_workflow,node_interaction toneAmber
class node_robotservice,node_pilot,node_robotruntime,node_skills toneMint
class node_map,node_simulation toneRose
class node_mcp,node_store,node_studio,node_models,node_devices toneIndigo
```

## Components

### Application interaction

- [HTTP and WS API](https://github.com/insightos-community/semantic-framework/tree/main/internal/server/http): `internal/server/http`
- [Authentication](https://github.com/insightos-community/semantic-framework/tree/main/internal/server/auth): `internal/server/auth`

### Task orchestration

- [Agent runtime](https://github.com/insightos-community/semantic-framework/blob/main/internal/agent/runtime/service.go): `internal/agent/runtime/service.go`
- [Agent kernel](https://github.com/insightos-community/semantic-framework/tree/main/internal/agent/kernel): `internal/agent/kernel`
- [Workflow scheduler](https://github.com/insightos-community/semantic-framework/blob/main/internal/workflow/scheduler.go): `internal/workflow/scheduler.go`
- [Human interaction](https://github.com/insightos-community/semantic-framework/blob/main/internal/interaction/service.go): `internal/interaction/service.go`

### Robot execution

- [Robot service](https://github.com/insightos-community/semantic-framework/blob/main/internal/robot/service.go): `internal/robot/service.go`
- [Semantic Pilot](https://github.com/insightos-community/semantic-framework/blob/main/cmd/semantic-pilot/main.go): `cmd/semantic-pilot/main.go`
- [Robot runtime](https://github.com/insightos-community/semantic-framework/blob/main/internal/robotruntime/orchestrator.go): `internal/robotruntime/orchestrator.go`
- [Robot Skills](https://github.com/insightos-community/semantic-framework/blob/main/internal/skill/store.go): `internal/skill/store.go`

### World and simulation

- [Semantic world model](https://github.com/insightos-community/semantic-framework/blob/main/internal/tool/builtin/map_query.go): `internal/tool/builtin/map_query.go`
- [Simulation services](https://github.com/insightos-community/semantic-framework/blob/main/internal/simulation/service.go): `internal/simulation/service.go`

### Shared platform

- [MCP integrations](https://github.com/insightos-community/semantic-framework/blob/main/internal/mcpregistry/registry.go): `internal/mcpregistry/registry.go`
- [Persistent state](https://github.com/insightos-community/semantic-framework/tree/main/internal/store): `internal/store`

### Other components

- **Studio user**
- **Semantic Studio**
- **Model providers**
- **Robots and devices**

## Connections

- Studio user → Semantic Studio: describes goal (evidence: `README.md`)
- Semantic Studio → HTTP and WS API: sends requests (evidence: `README.md`)
- HTTP and WS API → Authentication: routes auth (evidence: `internal/server/http/router.go`)
- HTTP and WS API → Agent runtime: routes requests
- Agent runtime → Agent kernel: runs agent (evidence: `internal/agent/runtime/service.go`)
- Agent runtime → Model providers: requests reasoning (evidence: `README.md`)
- Agent runtime → Workflow scheduler: organizes tasks (evidence: `README.md`)
- Agent runtime → Human interaction: requests approval
- Workflow scheduler → Robot service: coordinates robot work (evidence: `README.md`)
- Robot service → Semantic Pilot: dispatches actions (evidence: `README.md`)
- Semantic Pilot → Robot runtime: runs capabilities (evidence: `README.md`)
- Robot runtime → Robots and devices: connects devices (evidence: `README.md`)
- Robot service → Robot Skills: manages skills (evidence: `internal/robot/service.go`)
- Agent runtime → Semantic world model: queries semantics
- HTTP and WS API → Simulation services: routes simulation (evidence: `internal/server/http/router.go`)
- Agent runtime → MCP integrations: uses tools (evidence: `internal/agent/runtime/service.go`)
- Agent runtime → Persistent state: persists runtime state (evidence: `internal/agent/runtime/service.go`)
- Workflow scheduler → Persistent state: reads and writes (evidence: `internal/workflow/scheduler.go`)
- Robot service → Persistent state: records execution (evidence: `internal/robot/service.go`)
