# jackmannx/sawtooth-memory architecture

> Architecture diagram of the GitHub repository jackmannx/sawtooth-memory, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jackmannx/sawtooth-memory
- Repository: https://github.com/jackmannx/sawtooth-memory

## Overview

Sawtooth Memory is a hierarchical memory framework for LLM agents. An application adds conversation messages to a context manager, which returns promptly while local monitoring, entity extraction, and folding prepare work for background compression. A worker updates archival memory; prompt compilation retrieves useful context from the working, entity, archival, and optional semantic layers for the agent’s next LLM call. The README documents sync operation, provider backends, adapters, and storage options. Some connections are based on documented behavior because the corresponding implementation excerpts were not provided.

Main entry points: [`sawtooth_memory/middleware.py`](https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/middleware.py) (Async context manager), [`sawtooth_memory/storage/base.py`](https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/storage/base.py) (Storage interface), [`sawtooth_memory/state.py`](https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/state.py) (Hierarchical state), [`sawtooth_memory/worker.py`](https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/worker.py) (Background worker)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_api["Memory API"]
  node_context_manager["Async context manager<br/>[middleware.py]"]
  node_sync_manager["Sync context manager<br/>[sync_manager.py]"]
end

subgraph group_processing["Ingest and Compression"]
  node_monitor["Token monitor<br/>[monitor.py]"]
  node_ner["Deterministic NER<br/>[ner.py]"]
  node_guard["Entity guard<br/>[entity_guard.py]"]
  node_planner["Intent planner<br/>[intent_planner.py]"]
  node_observation_crush["Observation compaction"]
  node_fold["History folding<br/>[fold_unit.py]"]
  node_worker["Background worker<br/>[worker.py]"]
  node_compressor["Compression engine<br/>[compressor.py]"]
  node_provider_adapters["LLM provider adapters<br/>[adapters.py]"]
end

subgraph group_context["Context and Retrieval"]
  node_state["Hierarchical state<br/>[state.py]"]
  node_prompt_compiler["Prompt compiler<br/>[prompt_compiler.py]"]
  node_semantic_indexer["Semantic indexer<br/>[l3_indexer.py]"]
  node_embedding_provider["Embedding provider<br/>[factory.py]"]
end

subgraph group_integrations["Adapters and Events"]
  node_langchain["LangChain adapter"]
  node_langgraph_adapter["LangGraph adapter<br/>[adapter.py]"]
  node_langgraph["LangGraph nodes<br/>[graph.py]"]
  node_event_bus["Event bus<br/>[bus.py]"]
  node_journal["Compression journal<br/>[journal.py]"]
end

subgraph group_storage["Persistence"]
  node_storage_base["Storage interface<br/>[base.py]"]
  node_postgres["Postgres adapter"]
  node_redis["Redis adapter<br/>[redis_adapter.py]"]
end

node_application(("Agent application"))
node_llm_services(("LLM services"))

node_application -->|"adds messages"| node_context_manager
node_application -->|"adds messages"| node_sync_manager
node_context_manager -->|"updates"| node_state
node_sync_manager -->|"updates"| node_state
node_context_manager -->|"checks thresholds"| node_monitor
node_context_manager -->|"scans messages"| node_ner
node_context_manager -->|"protects entities"| node_guard
node_context_manager -->|"plans retrieval"| node_planner
node_context_manager -->|"compacts observations"| node_observation_crush
node_context_manager -->|"queues compression"| node_worker
node_worker -->|"compresses history"| node_compressor
node_compressor -->|"formats requests"| node_provider_adapters
node_provider_adapters -.->|"calls provider"| node_llm_services
node_worker -->|"merges results"| node_state
node_context_manager -->|"compiles context"| node_prompt_compiler
node_prompt_compiler -->|"reads memory layers"| node_state
node_context_manager -.->|"indexes archive"| node_semantic_indexer
node_semantic_indexer -->|"creates embeddings"| node_embedding_provider
node_semantic_indexer -->|"stores vectors"| node_storage_base
node_context_manager -.->|"loads and saves state"| node_storage_base
node_storage_base -.->|"persists state"| node_postgres
node_storage_base -.->|"persists state"| node_redis
node_application -.->|"uses history adapter"| node_langchain
node_langchain -->|"adds and reads messages"| node_context_manager
node_langgraph -->|"syncs and compiles"| node_langgraph_adapter
node_langgraph_adapter -->|"adds messages"| node_context_manager
node_context_manager -.->|"emits events"| node_event_bus
node_event_bus -.->|"dispatches journal events"| node_journal

click node_context_manager "https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/middleware.py"
click node_sync_manager "https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/sync_manager.py"
click node_state "https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/state.py"
click node_monitor "https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/monitor.py"
click node_ner "https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/ner.py"
click node_guard "https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/entity_guard.py"
click node_planner "https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/intent_planner.py"
click node_observation_crush "https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/observation_crush.py"
click node_fold "https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/fold_unit.py"
click node_worker "https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/worker.py"
click node_compressor "https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/compressor.py"
click node_provider_adapters "https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/providers/adapters.py"
click node_prompt_compiler "https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/prompt_compiler.py"
click node_semantic_indexer "https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/l3_indexer.py"
click node_embedding_provider "https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/embeddings/factory.py"
click node_storage_base "https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/storage/base.py"
click node_postgres "https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/storage/postgres_adapter.py"
click node_redis "https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/storage/redis_adapter.py"
click node_langchain "https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/integrations/langchain_adapter.py"
click node_langgraph_adapter "https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/integrations/langgraph/adapter.py"
click node_langgraph "https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/integrations/langgraph/graph.py"
click node_event_bus "https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/events/bus.py"
click node_journal "https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/journal.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_context_manager,node_sync_manager toneBlue
class node_monitor,node_ner,node_guard,node_planner,node_observation_crush,node_fold,node_worker,node_compressor,node_provider_adapters toneAmber
class node_state,node_prompt_compiler,node_semantic_indexer,node_embedding_provider toneMint
class node_langchain,node_langgraph_adapter,node_langgraph,node_event_bus,node_journal toneRose
class node_storage_base,node_postgres,node_redis,node_application,node_llm_services toneIndigo
```

## Components

### Memory API

- [Async context manager](https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/middleware.py): `sawtooth_memory/middleware.py`
- [Sync context manager](https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/sync_manager.py): `sawtooth_memory/sync_manager.py`

### Ingest and Compression

- [Token monitor](https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/monitor.py): `sawtooth_memory/monitor.py`
- [Deterministic NER](https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/ner.py): `sawtooth_memory/ner.py`
- [Entity guard](https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/entity_guard.py): `sawtooth_memory/entity_guard.py`
- [Intent planner](https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/intent_planner.py): `sawtooth_memory/intent_planner.py`
- [Observation compaction](https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/observation_crush.py): `sawtooth_memory/observation_crush.py`
- [History folding](https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/fold_unit.py): `sawtooth_memory/fold_unit.py`
- [Background worker](https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/worker.py): `sawtooth_memory/worker.py`
- [Compression engine](https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/compressor.py): `sawtooth_memory/compressor.py`
- [LLM provider adapters](https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/providers/adapters.py): `sawtooth_memory/providers/adapters.py`

### Context and Retrieval

- [Hierarchical state](https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/state.py): `sawtooth_memory/state.py`
- [Prompt compiler](https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/prompt_compiler.py): `sawtooth_memory/prompt_compiler.py`
- [Semantic indexer](https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/l3_indexer.py): `sawtooth_memory/l3_indexer.py`
- [Embedding provider](https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/embeddings/factory.py): `sawtooth_memory/embeddings/factory.py`

### Adapters and Events

- [LangChain adapter](https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/integrations/langchain_adapter.py): `sawtooth_memory/integrations/langchain_adapter.py`
- [LangGraph adapter](https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/integrations/langgraph/adapter.py): `sawtooth_memory/integrations/langgraph/adapter.py`
- [LangGraph nodes](https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/integrations/langgraph/graph.py): `sawtooth_memory/integrations/langgraph/graph.py`
- [Event bus](https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/events/bus.py): `sawtooth_memory/events/bus.py`
- [Compression journal](https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/journal.py): `sawtooth_memory/journal.py`

### Persistence

- [Storage interface](https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/storage/base.py): `sawtooth_memory/storage/base.py`
- [Postgres adapter](https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/storage/postgres_adapter.py): `sawtooth_memory/storage/postgres_adapter.py`
- [Redis adapter](https://github.com/jackmannx/sawtooth-memory/blob/main/sawtooth_memory/storage/redis_adapter.py): `sawtooth_memory/storage/redis_adapter.py`

### Other components

- **Agent application**
- **LLM services**

## Connections

- Agent application → Async context manager: adds messages
- Agent application → Sync context manager: adds messages
- Async context manager → Hierarchical state: updates
- Sync context manager → Hierarchical state: updates
- Async context manager → Token monitor: checks thresholds
- Async context manager → Deterministic NER: scans messages
- Async context manager → Entity guard: protects entities
- Async context manager → Intent planner: plans retrieval
- Async context manager → Observation compaction: compacts observations
- Async context manager → Background worker: queues compression
- Background worker → Compression engine: compresses history
- Compression engine → LLM provider adapters: formats requests
- LLM provider adapters → LLM services: calls provider
- Background worker → Hierarchical state: merges results
- Async context manager → Prompt compiler: compiles context
- Prompt compiler → Hierarchical state: reads memory layers
- Async context manager → Semantic indexer: indexes archive
- Semantic indexer → Embedding provider: creates embeddings
- Semantic indexer → Storage interface: stores vectors
- Async context manager → Storage interface: loads and saves state
- Storage interface → Postgres adapter: persists state
- Storage interface → Redis adapter: persists state
- Agent application → LangChain adapter: uses history adapter
- LangChain adapter → Async context manager: adds and reads messages
- LangGraph nodes → LangGraph adapter: syncs and compiles
- LangGraph adapter → Async context manager: adds messages
- Async context manager → Event bus: emits events
- Event bus → Compression journal: dispatches journal events