# osaurus-ai/osaurus architecture

> Architecture diagram of the GitHub repository osaurus-ai/osaurus, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/osaurus-ai/osaurus
- Repository: https://github.com/osaurus-ai/osaurus

## Overview

1. **Top-level shape**
   - This is a macOS-native Swift system with one app target (`App/osaurus`), one command-line entrypoint (`Packages/OsaurusCLI`), and a large shared core library (`Packages/OsaurusCore`). Most runtime behavior lives in the core package and is reused by both the GUI and CLI.

2. **User-facing entry points**
   - The macOS app is the primary interactive shell: SwiftUI views, app intents, management panels, and system-wide integrations.
   - The CLI exposes operational commands like `serve`, `ui`, `status`, `mcp`, and `tools ...`; it is the operational gateway for headless/server use and for external harnesses.

3. **Core runtime boundary**
   - `OsaurusCore` is the main architecture boundary. It contains models, services, managers, storage, networking, tools, identity, sandbox, memory, privacy filtering, and views.
   - The codebase is organized so “services” hold business logic, “managers” hold UI-facing state, and “views” render feature surfaces on top of them.

4. **Conversation and agent loop**
   - Chat is the central orchestration flow. A session collects user messages, attachments, tool results, memory, and model responses.
   - The agent loop composes prompts, selects tools, streams model output, handles tool calls, and persists the resulting transcript/state. This loop is reused across the UI, server API, and evaluation harnesses.

5. **Model/inference subsystem**
   - Model access is abstracted behind `ModelService` / `Inference` services and supports several backends: local MLX, Apple Foundation Models, and remote providers such as OpenAI-compatible, Anthropic, Gemini, xAI/OpenRouter/Ollama-like endpoints.
   - The model layer is not just transport: it also handles model selection, runtime policy, token budgeting, batching, residency/lease management, and capability-specific routing.

6. **Tooling and plugin system**
   - Tools are a first-class layer beneath the agent loop. Core tool types include built-ins, folder tools, sandbox tools, database tools, capability tools, and MCP/provider-backed tools.
   - There is a plugin ABI plus a repository/install layer (`OsaurusRepository`) for native plugins and Claude-style plugin bundles. The tool registry decides exposure, permissions, and routing into these capabilities.

7. **Sandbox execution boundary**
   - The sandbox is a hard trust boundary: agents can run code in an isolated Linux VM managed by Apple Containerization, with host/guest communication via vsock and shared workspace mounts.
   - Host-side sandbox services provision the VM, register tools, stream logs, enforce security policies, and mediate file/artifact access. This is where “real code execution” is kept separate from the macOS host.

8. **Identity and secure communication**
   - Identity is cryptographic and per-participant: master keys, agent keys, pairing keys, revocation, and secure session stores live in `Identity`.
   - This supports local access keys, pairings, and secure channel connections. The secure-channel layer encrypts agent-to-agent or relay-mediated traffic end to end, while the relay itself stays blind to plaintext.

9. **Networking and server surfaces**
   - `Networking` contains the local HTTP server, request parsing/validation, OpenAI/Anthropic/Ollama-compatible endpoints, MCP endpoints, relay tunneling, Bonjour discovery, and auth gates.
   - The same server layer powers local API consumption, remote exposure, and the stdio MCP bridge used by external MCP clients.

10. **Storage and persistence**
   - State is persisted through multiple SQLite-backed stores: chat history, agents, tools, plugins, schedules, memory, billing/credits, and server/runtime settings.
   - There is also encrypted storage for secrets and sensitive per-agent data. Storage is a shared persistence layer underneath chat, identity, memory, plugins, and tooling.

11. **Memory and privacy**
   - Memory is a distinct subsystem with embeddings, search, consolidation, relevance gating, and per-session context assembly.
   - Privacy filtering sits on the outbound path to cloud models: content is redacted, reviewed, then streamed back with placeholders unscrubbed for display. The design is fail-closed, so leakage blocks send.

12. **Knowledge and reuse layers**
   - Skills, methods, slash commands, watchers, schedules, themes, and document adapters are all separate feature layers that feed into the agent experience.
   - These are discovered and selected through search/routing services rather than manual wiring, which means the agent can dynamically pick relevant capability bundles.

13. **Document and attachment pipeline**
   - File attachments are not treated as plain blobs when structure matters. Dedicated adapters parse CSV, XLSX, PPTX, PDF, and rich documents into structured representations before they reach the model.
   - This keeps the agent’s context higher-fidelity and lets tools operate on structured content instead of only raw text.

14. **Observability, quality, and release plumbing**
   - Telemetry and crash reporting are optional, privacy-conscious, and separated from product data. The repo also contains extensive test suites, eval harnesses, live-proof scripts, benchmark scripts, and CI/release workflows.
   - These don’t drive runtime behavior, but they strongly shape the architecture by enforcing stability across inference, sandboxing, privacy, plugin ABI, and provider compatibility.

Main entry points: [`Packages/OsaurusCore/Services/Chat/ChatEngine.swift`](https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCore/Services/Chat/ChatEngine.swift) (Chat Engine), [`Packages/OsaurusCLI/Sources/OsaurusCLICore/Commands/Serve.swift`](https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCLI/Sources/OsaurusCLICore/Commands/Serve.swift) (CLI Commands), [`Packages/OsaurusCore/Services/Sandbox/SandboxManager.swift`](https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCore/Services/Sandbox/SandboxManager.swift) (Sandbox), [`Packages/OsaurusCore/Tools/ToolRegistry.swift`](https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCore/Tools/ToolRegistry.swift) (Tool Registry)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_app["macOS App"]
  node_app_shell["App Shell<br/>swiftui app<br/>[osaurusApp.swift]"]
  node_app_intents["App Intents"]
end

subgraph group_cli["CLI"]
  node_cli_entry["CLI Entry<br/>command entrypoint<br/>[OsaurusCLI.swift]"]
  node_cli_commands["CLI Commands<br/>command suite<br/>[Serve.swift]"]
end

subgraph group_core["OsaurusCore"]
  node_chat_engine["Chat Engine<br/>orchestrator<br/>[ChatEngine.swift]"]
  node_chat_models["Chat Models<br/>transcript models"]
  node_prompt_builder["Prompt Builder<br/>prompting"]
  node_model_service["Model Service<br/>inference abstraction<br/>[ModelService.swift]"]
  node_model_runtime["Model Runtime<br/>runtime policy"]
  node_tool_registry["Tool Registry<br/>tool router<br/>[ToolRegistry.swift]"]
  node_tooling["Tool Layers<br/>tool implementations"]
  node_sandbox["Sandbox<br/>trust boundary"]
  node_networking["Networking<br/>server surface"]
  node_identity["Identity"]
  node_storage[("Storage<br/>sqlite persistence")]
  node_memory["Memory<br/>retrieval"]
  node_privacy_filter["Privacy Filter<br/>redaction pipeline"]
  node_documents["Documents<br/>adapters"]
  node_managers["Managers<br/>ui state"]
  node_views["Views<br/>ui surfaces<br/>[ChatView.swift]"]
  node_computer_use["Computer Use<br/>desktop agent"]
end

subgraph group_evals["Evals"]
  node_eval_harness["Eval Harness<br/>regression runner<br/>[EvalRunner.swift]"]
end

subgraph group_plugins["Plugins"]
  node_plugin_repo["Plugin Repo<br/>plugin install"]
  node_plugin_abi["Plugin ABI<br/>interop ABI<br/>[osaurus_plugin.h]"]
end

node_app_shell -->|"uses"| node_managers
node_app_shell -->|"drives"| node_chat_engine
node_app_intents -->|"invokes"| node_chat_engine
node_cli_entry -->|"dispatches"| node_cli_commands
node_cli_commands -->|"boots"| node_networking
node_cli_commands -->|"manages"| node_tool_registry
node_cli_commands -->|"uses"| node_chat_engine
node_chat_engine -->|"composes"| node_prompt_builder
node_chat_engine -->|"calls"| node_model_service
node_chat_engine -->|"selects"| node_tool_registry
node_chat_engine -->|"retrieves"| node_memory
node_chat_engine -->|"filters"| node_privacy_filter
node_chat_engine -->|"persists"| node_chat_models
node_model_service -->|"governs"| node_model_runtime
node_model_service -->|"streams through"| node_privacy_filter
node_tool_registry -->|"exposes"| node_tooling
node_tooling -->|"executes in"| node_sandbox
node_sandbox -->|"reads/writes"| node_storage
node_sandbox -->|"serves"| node_networking
node_identity -->|"authenticates"| node_networking
node_identity -->|"stores"| node_storage
node_documents -->|"feeds"| node_chat_engine
node_managers -->|"renders"| node_views
node_computer_use -->|"extends"| node_chat_engine
node_plugin_repo -->|"conforms to"| node_plugin_abi
node_plugin_repo -->|"registers"| node_tool_registry
node_eval_harness -.->|"regresses"| node_chat_engine
node_eval_harness -.->|"regresses"| node_computer_use
node_eval_harness -.->|"regresses"| node_sandbox

click node_app_shell "https://github.com/osaurus-ai/osaurus/blob/main/App/osaurus/osaurusApp.swift"
click node_app_intents "https://github.com/osaurus-ai/osaurus/blob/main/App/osaurus/AppIntents/OsaurusIntents.swift"
click node_cli_entry "https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCLI/Sources/OsaurusCLI/OsaurusCLI.swift"
click node_cli_commands "https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCLI/Sources/OsaurusCLICore/Commands/Serve.swift"
click node_chat_engine "https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCore/Services/Chat/ChatEngine.swift"
click node_chat_models "https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCore/Models/Chat/ChatSessionData.swift"
click node_prompt_builder "https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCore/Services/Chat/SystemPromptComposer.swift"
click node_model_service "https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCore/Services/Inference/ModelService.swift"
click node_model_runtime "https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCore/Services/ModelRuntime/ModelResidencyManager.swift"
click node_tool_registry "https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCore/Tools/ToolRegistry.swift"
click node_tooling "https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCore/Tools/AgentLoopTools.swift"
click node_sandbox "https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCore/Services/Sandbox/SandboxManager.swift"
click node_networking "https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCore/Networking/OsaurusServer.swift"
click node_identity "https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCore/Identity/OsaurusIdentity.swift"
click node_storage "https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCore/Storage/StorageDatabaseCatalog.swift"
click node_memory "https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCore/Services/Memory/MemoryService.swift"
click node_privacy_filter "https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCore/PrivacyFilter/Core/PrivacyFilterPipeline.swift"
click node_documents "https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCore/Services/Documents/DocumentFileInspector.swift"
click node_managers "https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCore/Managers/ManagementStateManager.swift"
click node_views "https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCore/Views/Chat/ChatView.swift"
click node_computer_use "https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCore/ComputerUse/Loop/ComputerUseLoop.swift"
click node_plugin_repo "https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusRepository/PluginInstallManager.swift"
click node_plugin_abi "https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCore/Tools/PluginABI/osaurus_plugin.h"
click node_eval_harness "https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusEvals/Sources/OsaurusEvalsKit/EvalRunner.swift"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_app_shell,node_app_intents toneBlue
class node_cli_entry,node_cli_commands toneAmber
class node_chat_engine,node_chat_models,node_prompt_builder,node_model_service,node_model_runtime,node_tool_registry,node_tooling,node_sandbox,node_networking,node_identity,node_storage,node_memory,node_privacy_filter,node_documents,node_managers,node_views,node_computer_use toneMint
class node_eval_harness toneRose
class node_plugin_repo,node_plugin_abi toneIndigo
```

## Components

### macOS App

SwiftUI shell and app-intents entrypoint.

- [App Shell](https://github.com/osaurus-ai/osaurus/blob/main/App/osaurus/osaurusApp.swift): swiftui app · Primary macOS interactive entrypoint and lifecycle host. · `App/osaurus/osaurusApp.swift`
- [App Intents](https://github.com/osaurus-ai/osaurus/blob/main/App/osaurus/AppIntents/OsaurusIntents.swift): intents · Shortcuts and app-intent integrations for macOS control. · `App/osaurus/AppIntents/OsaurusIntents.swift`

### CLI

Headless operational commands and server bootstrap.

- [CLI Entry](https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCLI/Sources/OsaurusCLI/OsaurusCLI.swift): command entrypoint · Command-line front door for server and operations. · `Packages/OsaurusCLI/Sources/OsaurusCLI/OsaurusCLI.swift`
- [CLI Commands](https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCLI/Sources/OsaurusCLICore/Commands/Serve.swift): command suite · serve, ui, status, mcp, tools, and related operations. · `Packages/OsaurusCLI/Sources/OsaurusCLICore/Commands/Serve.swift`

### OsaurusCore

Shared runtime boundary used by app, CLI, and server.

- [Chat Engine](https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCore/Services/Chat/ChatEngine.swift): orchestrator · Core agent loop that composes prompts, streams output, and handles tool calls. · `Packages/OsaurusCore/Services/Chat/ChatEngine.swift`
- [Chat Models](https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCore/Models/Chat/ChatSessionData.swift): transcript models · Session turns, attachments, transcripts, and chat configuration state. · `Packages/OsaurusCore/Models/Chat/ChatSessionData.swift`
- [Prompt Builder](https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCore/Services/Chat/SystemPromptComposer.swift): prompting · System prompt composition, tool resolution, and token budgeting. · `Packages/OsaurusCore/Services/Chat/SystemPromptComposer.swift`
- [Model Service](https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCore/Services/Inference/ModelService.swift): inference abstraction · Backend-agnostic model access and selection boundary. · `Packages/OsaurusCore/Services/Inference/ModelService.swift`
- [Model Runtime](https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCore/Services/ModelRuntime/ModelResidencyManager.swift): runtime policy · Residency, leases, batching, load coordination, and runtime validation. · `Packages/OsaurusCore/Services/ModelRuntime/ModelResidencyManager.swift`
- [Tool Registry](https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCore/Tools/ToolRegistry.swift): tool router · Exposure, permissions, discovery, and routing for tools. · `Packages/OsaurusCore/Tools/ToolRegistry.swift`
- [Tool Layers](https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCore/Tools/AgentLoopTools.swift): tool implementations · Built-ins, folder, database, sandbox, capability, and MCP-backed tools. · `Packages/OsaurusCore/Tools/AgentLoopTools.swift`
- [Sandbox](https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCore/Services/Sandbox/SandboxManager.swift): trust boundary · Host-side provisioning and isolated execution for real code execution. · `Packages/OsaurusCore/Services/Sandbox/SandboxManager.swift`
- [Networking](https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCore/Networking/OsaurusServer.swift): server surface · Local HTTP server, compatible APIs, relay tunneling, and MCP bridges. · `Packages/OsaurusCore/Networking/OsaurusServer.swift`
- [Identity](https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCore/Identity/OsaurusIdentity.swift): crypto identity · Keys, pairing, secure sessions, revocation, and access-key management. · `Packages/OsaurusCore/Identity/OsaurusIdentity.swift`
- [Storage](https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCore/Storage/StorageDatabaseCatalog.swift): sqlite persistence · SQLite-backed stores, encrypted files, and mutation gates. · `Packages/OsaurusCore/Storage/StorageDatabaseCatalog.swift`
- [Memory](https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCore/Services/Memory/MemoryService.swift): retrieval · Embeddings, relevance gating, consolidation, and context assembly. · `Packages/OsaurusCore/Services/Memory/MemoryService.swift`
- [Privacy Filter](https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCore/PrivacyFilter/Core/PrivacyFilterPipeline.swift): redaction pipeline · Outbound scrubbing, review, placeholders, and fail-closed leakage prevention. · `Packages/OsaurusCore/PrivacyFilter/Core/PrivacyFilterPipeline.swift`
- [Documents](https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCore/Services/Documents/DocumentFileInspector.swift): adapters · Structured parsing and emission for PDF, CSV, XLSX, PPTX, and rich documents. · `Packages/OsaurusCore/Services/Documents/DocumentFileInspector.swift`
- [Managers](https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCore/Managers/ManagementStateManager.swift): ui state · UI-facing state, windows, model selection, plugins, schedules, and orchestration helpers. · `Packages/OsaurusCore/Managers/ManagementStateManager.swift`
- [Views](https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCore/Views/Chat/ChatView.swift): ui surfaces · SwiftUI and representable feature surfaces over the shared managers/services. · `Packages/OsaurusCore/Views/Chat/ChatView.swift`
- [Computer Use](https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCore/ComputerUse/Loop/ComputerUseLoop.swift): desktop agent · macOS perception, control, policy, and loop for GUI automation. · `Packages/OsaurusCore/ComputerUse/Loop/ComputerUseLoop.swift`

### Evals

Regression and capability harnesses for core flows.

- [Eval Harness](https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusEvals/Sources/OsaurusEvalsKit/EvalRunner.swift): regression runner · Agent-loop, computer-use, and capability regression suites. · `Packages/OsaurusEvals/Sources/OsaurusEvalsKit/EvalRunner.swift`

### Plugins

Native plugin packaging and installation boundary.

- [Plugin Repo](https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusRepository/PluginInstallManager.swift): plugin install · Native plugin repository, install flow, and manifest verification. · `Packages/OsaurusRepository/PluginInstallManager.swift`
- [Plugin ABI](https://github.com/osaurus-ai/osaurus/blob/main/Packages/OsaurusCore/Tools/PluginABI/osaurus_plugin.h): interop ABI · Shared host/plugin ABI and test kit used by plugin packages. · `Packages/OsaurusCore/Tools/PluginABI/osaurus_plugin.h`

## Connections

- App Shell → Managers: uses · The app shell hosts the shared UI state and windows.
- App Shell → Chat Engine: drives · Interactive chat ultimately flows through the shared agent loop.
- App Intents → Chat Engine: invokes · App intents route into the same core chat orchestration.
- CLI Entry → CLI Commands: dispatches · The CLI entrypoint exposes command families.
- CLI Commands → Networking: boots · serve and mcp commands expose the server surfaces.
- CLI Commands → Tool Registry: manages · tool subcommands inspect and modify tool exposure.
- CLI Commands → Chat Engine: uses · ui/status/serve flows share the core agent runtime.
- Chat Engine → Prompt Builder: composes · The agent loop builds prompts and resolves tool context.
- Chat Engine → Model Service: calls · Chat delegates generation and streaming to model backends.
- Chat Engine → Tool Registry: selects · Tool calls are routed through the registry and permissions layer.
- Chat Engine → Memory: retrieves · Session context can be assembled from long-term memory.
- Chat Engine → Privacy Filter: filters · Outbound cloud traffic is redacted before sending.
- Chat Engine → Chat Models: persists · The loop writes session turns, attachments, and transcript state.
- Model Service → Model Runtime: governs · Backends share runtime policy, residency, and batching control.
- Model Service → Privacy Filter: streams through · Remote-provider output passes through privacy safeguards.
- Tool Registry → Tool Layers: exposes · The registry surfaces built-in and provider-backed tools.
- Tool Layers → Sandbox: executes in · Shell and code-execution tools are mediated by the sandbox.
- Sandbox → Storage: reads/writes · Workspace artifacts and logs are persisted through storage.
- Sandbox → Networking: serves · Sandbox bridge and stdio/host communication connect to server surfaces.
- Identity → Networking: authenticates · Secure channels and access keys gate local and remote traffic.
- Identity → Storage: stores · Keys, revocations, and secure session state are persisted.
- Documents → Chat Engine: feeds · Structured attachments are normalized before prompting.
- Managers → Views: renders · Managers supply state used by SwiftUI and representable views.
- Computer Use → Chat Engine: extends · Desktop automation is coordinated by the same agent loop.
- Plugin Repo → Plugin ABI: conforms to · Repository installs are validated against the shared plugin ABI.
- Plugin Repo → Tool Registry: registers · Installed plugins become tool-capable surfaces in the registry.
- Eval Harness → Chat Engine: regresses · Evals exercise the agent loop across scenarios and budgets.
- Eval Harness → Computer Use: regresses · Evals also cover computer-use behavior and policies.
- Eval Harness → Sandbox: regresses · Sandbox scenarios are validated by the harness.