# ayushpani/engram architecture

> Architecture diagram of the GitHub repository ayushpani/engram, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-05-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ayushpani/engram
- Repository: https://github.com/ayushpani/engram

## Overview

1. **The repo is a monorepo for an “agentic memory” platform, not a single app.**  
   The main product surface is the Next.js web app, but the system also includes browser/desktop capture clients, an MCP server for agent runtimes, edge reranking workers, shared SDKs, WASM performance modules, docs, and benchmark/test tooling.

2. **The core runtime flow is ingestion → deduplication → storage → consolidation → retrieval.**  
   Memories can be created from the web app, browser extension, Raycast, or third-party agent SDKs. Before being persisted, some clients use local WASM deduplication; later, background consolidation rebuilds memory as a graph and compresses clusters into higher-density facts; retrieval then reorders candidates with an edge reranker.

3. **`apps/web` is the main user-facing control plane.**  
   It is a Next.js app with auth, onboarding, settings, integrations, chat, document views, and memory graph views. It also exposes server routes for onboarding flows, email generation, and OG assets, and is configured for Cloudflare deployment via Wrangler/OpenNext.

4. **`apps/browser-extension` is a capture layer at the browser edge.**  
   It uses WXT and is split into background, content-script, popup, and welcome entrypoints. Its content scripts are specialized per site (ChatGPT, Claude, Twitter, T3), so the extension can detect context, collect page data, deduplicate it, and push it into the memory system.

5. **`apps/raycast-extension` is a desktop capture/search surface for macOS.**  
   It provides quick actions for adding memories and searching memories/projects directly from Raycast, acting as another ingestion and retrieval client outside the browser and web UI.

6. **`apps/mcp` exposes Engram over the Model Context Protocol.**  
   This is the integration boundary for external AI tools like Claude Desktop or Cursor. The app includes auth, search/rerank logic, a small UI, and Cloudflare Worker deployment config, making Engram usable as a local/remote memory provider for MCP-capable agents.

7. **`apps/reranker-worker` is the edge retrieval accelerator.**  
   It is the Cloudflare Worker responsible for reranking candidate memories/documents using edge AI infrastructure. The repository also has `packages/edge-reranker`, which appears to hold shared reranking logic used by that worker.

8. **WASM packages handle the heavy local compute pieces.**  
   `packages/client-dedup-wasm` supports client-side similarity/deduplication. `packages/memory-consolidator` handles graph clustering/consolidation. `packages/reranker-wasm` and `packages/wasm-math` provide lower-level compute helpers, but the readme notes production reranking is done at the edge with Cloudflare AI rather than pure WASM.

9. **`packages/memory-graph` is the interactive graph visualization subsystem.**  
   It contains canvas rendering, hit testing, simulation, viewport, hover popovers, and graph hooks. This package is reused by the web app and also has a standalone playground app, so it is a distinct visualization boundary rather than just a component.

10. **`packages/tools` is the agent integration layer.**  
   It contains adapters for AI SDK, OpenAI, Vercel, Mastra, VoltAgent, and shared prompt/context/memory-client utilities. This is the part of the repo that turns Engram memory into callable tools or middleware for external agent frameworks.

11. **There are first-class SDK packages for Python and TypeScript consumers.**  
   The Python packages target specific ecosystems such as OpenAI, agent frameworks, Cartesia, and Pipecat. The TypeScript SDKs and helpers live alongside them, giving external apps a narrow integration boundary without needing the full monorepo.

12. **Shared internal packages keep UI, auth, validation, and client behavior consistent.**  
   `packages/lib` centralizes auth, query client, similarity, IDs, and common types. `packages/ui` is the design system; `packages/hooks` holds reusable React hooks; `packages/validation` defines shared schemas and API shapes. These packages reduce coupling between the apps.

13. **Docs, examples, and benchmarks are treated as part of the product architecture.**  
   `apps/docs` is a large MDX documentation app covering concepts, APIs, integrations, deployment, and cookbooks. `packages/docs-test` and `packages/benchmark` validate the public docs/SDK behavior and measure retrieval/reranking performance against live systems.

14. **Deployment is edge-first and package publishing is automated.**  
   The readme and workflows show Cloudflare Workers, Workers AI, Vectorize, Hyperdrive, and Durable Objects as the main infrastructure targets. GitHub Actions publish SDK/package releases and deploy the reranker worker, so the repo’s architecture includes both runtime services and a release pipeline.

Main entry points: [`apps/web`](https://github.com/ayushpani/engram/tree/main/apps/web) (Web app), [`packages/tools`](https://github.com/ayushpani/engram/tree/main/packages/tools) (Tools SDK), [`apps/reranker-worker`](https://github.com/ayushpani/engram/tree/main/apps/reranker-worker) (Reranker worker), [`apps/browser-extension`](https://github.com/ayushpani/engram/tree/main/apps/browser-extension) (Browser extension)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_apps["Apps"]
  node_web["Web app<br/>Next.js control plane"]
  node_browser_extension["Browser extension<br/>WXT capture client"]
  node_raycast_extension["Raycast app<br/>macOS capture/search"]
  node_mcp_app["MCP server<br/>MCP integration"]
  node_reranker_worker["Reranker worker<br/>Cloudflare Worker"]
  node_docs_app["Docs site<br/>MDX docs"]
  node_playground["Graph playground<br/>Next.js demo"]
end

subgraph group_packages["Packages"]
  node_memory_graph(("Memory graph<br/>visualization library"))
  node_tools["Tools SDK<br/>agent tooling"]
  node_agent_py["Python agents<br/>Python SDKs"]
  node_ai_sdk["AI SDK<br/>TypeScript SDK"]
  node_lib["Core lib<br/>shared runtime"]
  node_ui["UI kit<br/>design system"]
  node_hooks["Hooks"]
  node_validation["Validation<br/>schemas"]
  node_dedup_wasm{{"Dedup WASM<br/>local similarity compute"}}
  node_consolidator{{"Consolidator<br/>graph clustering"}}
  node_edge_reranker_pkg{{"Rerank logic<br/>shared reranking"}}
  node_reranker_wasm{{"Reranker WASM<br/>compute helper"}}
  node_wasm_math{{"WASM math<br/>compute helper"}}
  node_benchmark["Benchmarking<br/>perf tooling"]
  node_docs_test["Docs tests<br/>integration checks"]
end

subgraph group_infra["Infra"]
end

node_browser_extension -->|"dedups"| node_dedup_wasm
node_browser_extension -->|"sends memories"| node_web
node_raycast_extension -->|"adds/searches"| node_web
node_mcp_app -->|"queries memory"| node_web
node_mcp_app -->|"reranks"| node_edge_reranker_pkg
node_reranker_worker -->|"uses"| node_edge_reranker_pkg
node_reranker_worker -->|"serves ranking"| node_web
node_web -->|"visualizes"| node_memory_graph
node_playground -->|"hosts"| node_memory_graph
node_web -->|"agent tools"| node_tools
node_web -->|"shared core"| node_lib
node_web -->|"renders with"| node_ui
node_web -->|"uses hooks"| node_hooks
node_web -->|"validates APIs"| node_validation
node_tools -->|"adapts"| node_ai_sdk
node_tools -->|"mirrors"| node_agent_py
node_agent_py -->|"consumes API"| node_web
node_docs_app -->|"documents"| node_validation
node_docs_test -->|"verifies"| node_docs_app
node_benchmark -->|"measures"| node_reranker_worker
node_consolidator -->|"feeds retrieval"| node_web
node_reranker_wasm -->|"depends on"| node_wasm_math

click node_web "https://github.com/ayushpani/engram/tree/main/apps/web"
click node_browser_extension "https://github.com/ayushpani/engram/tree/main/apps/browser-extension"
click node_raycast_extension "https://github.com/ayushpani/engram/tree/main/apps/raycast-extension"
click node_mcp_app "https://github.com/ayushpani/engram/tree/main/apps/mcp"
click node_reranker_worker "https://github.com/ayushpani/engram/tree/main/apps/reranker-worker"
click node_docs_app "https://github.com/ayushpani/engram/tree/main/apps/docs"
click node_playground "https://github.com/ayushpani/engram/tree/main/apps/memory-graph-playground"
click node_memory_graph "https://github.com/ayushpani/engram/tree/main/packages/memory-graph"
click node_tools "https://github.com/ayushpani/engram/tree/main/packages/tools"
click node_agent_py "https://github.com/ayushpani/engram/tree/main/packages/agent-framework-python"
click node_ai_sdk "https://github.com/ayushpani/engram/tree/main/packages/ai-sdk"
click node_lib "https://github.com/ayushpani/engram/tree/main/packages/lib"
click node_ui "https://github.com/ayushpani/engram/tree/main/packages/ui"
click node_hooks "https://github.com/ayushpani/engram/tree/main/packages/hooks"
click node_validation "https://github.com/ayushpani/engram/tree/main/packages/validation"
click node_dedup_wasm "https://github.com/ayushpani/engram/tree/main/packages/client-dedup-wasm"
click node_consolidator "https://github.com/ayushpani/engram/tree/main/packages/memory-consolidator"
click node_edge_reranker_pkg "https://github.com/ayushpani/engram/tree/main/packages/edge-reranker"
click node_reranker_wasm "https://github.com/ayushpani/engram/tree/main/packages/reranker-wasm"
click node_wasm_math "https://github.com/ayushpani/engram/tree/main/packages/wasm-math"
click node_benchmark "https://github.com/ayushpani/engram/tree/main/packages/benchmark"
click node_docs_test "https://github.com/ayushpani/engram/tree/main/packages/docs-test"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_web,node_browser_extension,node_raycast_extension,node_mcp_app,node_reranker_worker,node_docs_app,node_playground toneBlue
class node_memory_graph,node_tools,node_agent_py,node_ai_sdk,node_lib,node_ui,node_hooks,node_validation,node_dedup_wasm,node_consolidator,node_edge_reranker_pkg,node_reranker_wasm,node_wasm_math,node_benchmark,node_docs_test toneAmber
```

## Components

### Apps

User-facing surfaces and runtime services

- [Web app](https://github.com/ayushpani/engram/tree/main/apps/web): Next.js control plane · Main product UI for auth, onboarding, settings, chat, documents, and memory graph views. · `apps/web`
- [Browser extension](https://github.com/ayushpani/engram/tree/main/apps/browser-extension): WXT capture client · Browser-edge ingestion surface with site-specific content scripts, popup, background, and welcome flows. · `apps/browser-extension`
- [Raycast app](https://github.com/ayushpani/engram/tree/main/apps/raycast-extension): macOS capture/search · Desktop quick actions for adding memories and searching memories or projects. · `apps/raycast-extension`
- [MCP server](https://github.com/ayushpani/engram/tree/main/apps/mcp): MCP integration · Model Context Protocol boundary exposing Engram search and rerank to external agents. · `apps/mcp`
- [Reranker worker](https://github.com/ayushpani/engram/tree/main/apps/reranker-worker): Cloudflare Worker · Edge retrieval accelerator that reranks candidates during search. · `apps/reranker-worker`
- [Docs site](https://github.com/ayushpani/engram/tree/main/apps/docs): MDX docs · Product documentation, SDK guides, concepts, cookbooks, and deployment docs. · `apps/docs`
- [Graph playground](https://github.com/ayushpani/engram/tree/main/apps/memory-graph-playground): Next.js demo · Standalone playground for the memory graph subsystem. · `apps/memory-graph-playground`

### Packages

Shared libraries, SDKs, compute modules, and tooling

- [Memory graph](https://github.com/ayushpani/engram/tree/main/packages/memory-graph): visualization library · Interactive canvas-based memory graph renderer with simulation, hit testing, viewport, and hover UI. · `packages/memory-graph`
- [Tools SDK](https://github.com/ayushpani/engram/tree/main/packages/tools): agent tooling · Adapters and shared utilities for AI SDK, OpenAI, Vercel, Mastra, and VoltAgent integrations. · `packages/tools`
- [Python agents](https://github.com/ayushpani/engram/tree/main/packages/agent-framework-python): Python SDKs · Python packages for agent-framework, OpenAI, Cartesia, and Pipecat integrations. · `packages/agent-framework-python`
- [AI SDK](https://github.com/ayushpani/engram/tree/main/packages/ai-sdk): TypeScript SDK · TypeScript SDK surface for tools and agent integration. · `packages/ai-sdk`
- [Core lib](https://github.com/ayushpani/engram/tree/main/packages/lib): shared runtime · Shared auth, API client, query, similarity, ID, and utility code used across apps. · `packages/lib`
- [UI kit](https://github.com/ayushpani/engram/tree/main/packages/ui): design system · Shared components, typography, and visual primitives used by the web app and related surfaces. · `packages/ui`
- [Hooks](https://github.com/ayushpani/engram/tree/main/packages/hooks): shared React hooks · Reusable React hooks for onboarding, storage, and interaction behavior. · `packages/hooks`
- [Validation](https://github.com/ayushpani/engram/tree/main/packages/validation): schemas · Shared API and schema definitions for consistent client/server shapes. · `packages/validation`
- [Dedup WASM](https://github.com/ayushpani/engram/tree/main/packages/client-dedup-wasm): local similarity compute · Client-side deduplication and similarity compute used before persistence. · `packages/client-dedup-wasm`
- [Consolidator](https://github.com/ayushpani/engram/tree/main/packages/memory-consolidator): graph clustering · WASM/Rust memory consolidation worker that rebuilds and compresses memory clusters. · `packages/memory-consolidator`
- [Rerank logic](https://github.com/ayushpani/engram/tree/main/packages/edge-reranker): shared reranking · Shared reranking logic used by the edge reranker worker. · `packages/edge-reranker`
- [Reranker WASM](https://github.com/ayushpani/engram/tree/main/packages/reranker-wasm): compute helper · Lower-level WASM reranking helpers and tokenizer assets. · `packages/reranker-wasm`
- [WASM math](https://github.com/ayushpani/engram/tree/main/packages/wasm-math): compute helper · Low-level numerical helpers shared by WASM compute modules. · `packages/wasm-math`
- [Benchmarking](https://github.com/ayushpani/engram/tree/main/packages/benchmark): perf tooling · Dataset, runner, metrics, and reporting for retrieval and reranking evaluation. · `packages/benchmark`
- [Docs tests](https://github.com/ayushpani/engram/tree/main/packages/docs-test): integration checks · Validation harness for public docs and SDK examples across TypeScript and Python. · `packages/docs-test`

## Connections

- Browser extension → Dedup WASM: dedups · Performs local similarity filtering before sending captured memories.
- Browser extension → Web app: sends memories · Pushes captured content into the main Engram system.
- Raycast app → Web app: adds/searches · Uses Engram as a desktop capture and retrieval surface.
- MCP server → Web app: queries memory · Acts as an external agent-facing access path to the core memory system.
- MCP server → Rerank logic: reranks · Applies search reranking inside the MCP boundary.
- Reranker worker → Rerank logic: uses · Worker shares reranking logic with the package layer.
- Reranker worker → Web app: serves ranking · Accelerates retrieval for the product search flow.
- Web app → Memory graph: visualizes · Embeds the graph experience for memory exploration.
- Graph playground → Memory graph: hosts · Provides a standalone sandbox for the graph renderer.
- Web app → Tools SDK: agent tools · Uses shared agent integration utilities for chat and MCP-related features.
- Web app → Core lib: shared core · Relies on common auth, queries, IDs, and client helpers.
- Web app → UI kit: renders with · Builds the product UI from shared design system components.
- Web app → Hooks: uses hooks · Shares onboarding and interaction behavior through common hooks.
- Web app → Validation: validates APIs · Shares schemas for consistent client/server data contracts.
- Tools SDK → AI SDK: adapts · Provides Engram memory tools for the AI SDK ecosystem.
- Tools SDK → Python agents: mirrors · Aligns TypeScript and Python agent integration patterns.
- Python agents → Web app: consumes API · Python SDKs integrate against the Engram backend surface.
- Docs site → Validation: documents · Tracks API shapes and product contracts through docs.
- Docs tests → Docs site: verifies · Checks docs and examples against expected SDK behavior.
- Benchmarking → Reranker worker: measures · Benchmarks retrieval and reranking behavior against live systems.
- Consolidator → Web app: feeds retrieval · Background consolidation improves the memory structures queried by the app.
- Reranker WASM → WASM math: depends on · Shares lower-level compute helpers for WASM execution.
