How sankhya-ai/dhee works
1. Overall shape
Dhee is a Python-first local memory platform for agents, with a few adjacent packages around it: a Rust acceleration crate (dhee-accel), a small pub/sub service (engram-bus), a web UI, MCP/server entrypoints, and optional plugins/hooks for agent runtimes. The central design is: capture agent activity locally, turn it into structured memory, store it durably, then compile the smallest useful context back into the next interaction.
2. Primary control surfaces
The main ways into the system are the CLI (dhee/cli*.py), MCP server/tools (dhee/mcp_server.py, dhee/mcp_slim.py), HTTP/runtime servers (dhee/runtime_daemon.py, dhee/ui/server.py, dhee/agent_runtime/server.py), and provider-specific adapters. These are the boundaries where external agents, shells, or UIs enter the memory system.
3. Context compilation core
The conceptual center is in dhee/core, dhee/context_ir.py, dhee/context_kernel.py, and dhee/context_state.py. This layer turns raw events, transcripts, files, and prior sessions into a compact context packet. It includes the story model the README describes: series/season/episode/scene/scene-card, plus intent, policy, belief, conflict, decay, retrieval, and handoff logic.
4. Memory pipeline
The memory subsystem in dhee/memory is the main ingest/search/write path. It has admission, scoping, quality, reranking, episodic memory, search/write pipelines, and orchestration. The data flow is: incoming events and evidence are admitted, scored, possibly enriched or distilled, then written as durable items and later retrieved as scene cards or context packs.
5. Persistence and storage
Durable state is backed primarily by SQLite through dhee/db, with separate modules for common tables, analytics, artifacts, and domain-specific storage. Vector search is abstracted through dhee/vector_stores with implementations such as SQLite vectors and zvec; the README’s production stack points to dhee-accel for accelerated vector/score operations. Secrets and repo-scoped state are isolated through dhee/secret_store.py, workspace helpers, and local .dhee/... storage.
6. Agent runtime layer
dhee/agent_runtime, dhee/runtime.py, dhee/runtime_io.py, and dhee/contract_runtime.py form the runtime boundary that wraps an agent session. This layer handles before-run context injection, tool exposure, event capture, checkpointing, and policy/proof checks. It is where Dhee can behave as a sidecar memory runtime rather than a standalone app.
7. Provider and harness integrations
Provider adapters live in dhee/providers, dhee/llms, dhee/embeddings, and dhee/profiles. The repo supports multiple external model vendors and embedding backends, including OpenAI, Gemini, Nvidia, Ollama, Qwen, Anthropic, and ElevenLabs. For execution environments, dhee/harness and dhee/hooks integrate with agent shells such as Claude Code and Codex, capturing transcripts and tool calls as memory events.
8. MCP and tool surface
Dhee exposes a tool-oriented interface through MCP and related routers. The dhee/router package contains digests, policy, intent handling, pre-tool gating, and quality reporting, while dhee/mcp_registry.py, dhee/mcp_server.py, and dhee/cli_mcp.py publish the actual tool surface. This is the boundary that lets third-party MCP clients query or mutate memory without knowing the internal storage model.
9. UI and operator views
The web UI under dhee/ui/web is a React/Vite frontend backed by dhee/ui/server.py and the Python backend APIs. Its views are organized around operational concepts like memory, router, tasks, workspace, conflict, evolution, and canvas. Architecturally, it is an निरीक्षण/control plane: it visualizes memory state, scene graphs, routing decisions, and workspace-level entities rather than serving as the primary execution engine.
10. Repo intelligence and graph analysis
dhee/repo_intelligence.py, dhee/repo_link.py, dhee/graph_proof_bundle.py, dhee/graph_projection_hardening.py, and related dhee/core/graph.py modules form the “repo brain.” This subsystem maps files, symbols, routes, tests, and failure evidence into a graph-backed impact model so the agent can answer “if I touch this, what breaks?” It is a separate analysis boundary from generic memory: it is git/repo-scoped and optimized for change impact and proof.
11. Scene, temporal, and world-memory subsystems
Several packages model time and narrative state: dhee/temporal_*, dhee/world_memory, dhee/core/scene.py, episode.py, session_tracker.py, and handoff_snapshot.py. These components preserve continuity across sessions by tracking scenes, causal graphs, session graphs, and checkpoint/handoff artifacts. They are the bridge between short-lived agent runs and long-lived memory.
12. Shared task and multi-agent coordination
dhee/engram-bus, dhee/core/workspace_line_bus.py, dhee/core/shared_tasks.py, and dhee/core/task_state.py provide coordination primitives for shared workspace state and task progress. This is the cross-agent boundary: memory is not only per-session, but can be published, replayed, or synchronized across workers and tools.
13. Evaluation, training, and quality gates
The dhee/training, dhee/mini, dhee/benchmarks, benchmarks/longmemeval, and the extensive test suite act as the quality subsystem. They validate retrieval quality, memory admission, routing, runtime behavior, and provider integrations. This matters architecturally because Dhee treats memory quality as an enforced contract, not just a best-effort cache.
14. Packaging and deployment model
The repo is built as a distributable Python package with optional extras, a Docker image, and CLI install flows. The Rust crate is an internal performance boundary; the UI is prebuilt and served alongside Python services; and the optional plugin/package directories show that Dhee can be embedded into existing agent ecosystems rather than requiring a single monolithic deployment.
Main entry points: main.py Memory core · context_kernel.py Kernel · sqlite.py SQLite DB · search_pipeline.py Search/write
Components 23
Entry points
User and machine entry surfaces into the memory system
- CLI · Primary shell entry for setup, updates, and memory operations
dhee/cli.py - MCP tools · MCP server and registry for external clients
dhee/mcp_server.py - Runtime daemon · Long-running service entry for local runtime orchestration
dhee/runtime_daemon.py
Context core
Context compilation and session-state machinery
- Context IR · Intermediate representation for compact context packets
dhee/context_ir.py - Kernel · Core logic that compiles events and evidence into usable context
dhee/context_kernel.py - State · Mutable context and session state tracked across runs
dhee/context_state.py
Memory system
Ingest, retrieve, score, and write durable memory
- Memory core · Main orchestration for ingest, search, and write flows
dhee/memory/main.py - Admission · Filters and scopes incoming evidence before persistence
dhee/memory/admission.py - Search/write · Retrieval and durable write pipelines for memory items
dhee/memory/search_pipeline.py - Quality · Quality checks, reranking, and relevance shaping
dhee/memory/quality.py
Storage layer
Persistence, vector search, and acceleration backends
- SQLite DB · Durable relational storage for common, analytics, artifact, and domain data
dhee/db/sqlite.py - Vector store · Abstracted vector search layer with SQLite and zvec backends
dhee/vector_stores/base.py - Rust accel · Rust acceleration for scoring, decay, and vector operations
dhee-accel/src/lib.rs
Runtime boundary
Agent session wrapping, tool exposure, and provider integration
- Agent runtime · Wraps agent sessions with context injection, event capture, and policy checks
dhee/runtime.py - Agent server · HTTP/server boundary for agent-runtime workflows and tools
dhee/agent_runtime/server.py - Providers · Model and embedding adapters for multiple external vendors
dhee/providers/base.py
Ops and analysis
UI, repo intelligence, coordination, and evaluation
- UI server · Backend for the operator UI and local web app
dhee/ui/server.py - Web UI · React/Vite frontend for memory, router, workspace, and canvas views
dhee/ui/web/src/App.tsx - Repo brain · Repo-scoped impact, graph, and proof analysis for change reasoning
dhee/repo_intelligence.py - World memory · Session graphs, causal graphs, and world-state capture across runs
dhee/world_memory/service.py - Engram bus · Small pub/sub coordination service for workspace and task signals
engram-bus/engram_bus/server.py - Quality loop · Benchmarks, replay gates, and trainer loops that enforce memory quality
dhee/training/train.py - Plugins · Optional shell/runtime integrations and command packs
plugins/engram-memory/README.md
Connections 27
- CLI → Memory core: operate · CLI drives memory operations and maintenance
- CLI → MCP tools: expose tools · CLI publishes or reuses MCP-oriented tool entrypoints
- Runtime daemon → Agent runtime: host · Daemon hosts runtime wrapping and session services
- Agent runtime → Kernel: compile · Runtime injects compiled context into sessions
- Agent runtime → Agent server: serve · Runtime boundary is surfaced through the agent server
- Kernel → Context IR: shape · Kernel shapes the intermediate context representation
- Kernel → State: update · Kernel updates session and context state
- Memory core → Admission: ingest · Memory core admits and scopes incoming evidence
- Memory core → Quality: score · Memory core ranks and quality-controls candidates
- Memory core → Search/write: retrieve/write · Memory core routes work into search and write pipelines
- Search/write → SQLite DB: persist · Memory writes durable items into SQLite-backed storage
- Search/write → Vector store: index · Memory indexes and retrieves similarity data through vector stores
- Vector store → Rust accel: accelerate · Vector operations can use native Rust acceleration
- SQLite DB → Vector store: support · Relational state supports vector-backed retrieval and metadata
- Agent server → Providers: invoke · Agent runtime relies on provider adapters for model calls
- Providers → Memory core: feed events · Provider interactions become memory events and transcripts
- UI server → Web UI: serve · Python backend serves the web frontend
- Web UI → Memory core: inspect · UI reads and visualizes memory state and routing outputs
- Web UI → State: display · UI surfaces current context and session state
- Repo brain → Kernel: analyze · Repo intelligence consumes graph-like context and proof artifacts
- Repo brain → SQLite DB: record · Repo intelligence stores scoped analysis and evidence
- World memory → State: continuity · World memory preserves continuity across sessions
- World memory → SQLite DB: store · Temporal graphs and session artifacts are persisted durably
- Engram bus → Memory core: publish · Bus broadcasts workspace and task signals into memory flows
- Plugins → CLI: extend · Plugin packs extend the CLI and shell workflows
- Quality loop → Memory core: evaluate · Benchmarks and replay gates evaluate memory behavior
- Quality loop → Repo brain: verify · Quality loops validate repo-impact and proof workflows