# deepseek-ai/engram architecture

> Architecture diagram of the GitHub repository deepseek-ai/engram, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/deepseek-ai/engram
- Repository: https://github.com/deepseek-ai/engram

## Overview

This repository is a standalone Engram demonstration, not a production model-serving system. Its intended workflow is to run the demo, form token n-grams, hash them into static memory, and fuse retrieved embeddings with hidden states in selected transformer blocks. The implementation also includes a compressed tokenizer and short convolution; attention and MoE are explicitly mocked. The available excerpts do not show the full script entry-point wiring or Engram forward pass, so those internal links are limited to relationships visible in the excerpts or documented in the README.

Main entry points: [`engram_demo_v1.py`](https://github.com/deepseek-ai/engram/blob/main/engram_demo_v1.py) (Transformer block)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_demo["Demo"]
  node_demo_script["Demo script<br/>[engram_demo_v1.py]"]
end

subgraph group_memory["Engram Memory"]
  node_tokenizer["Compressed tokenizer<br/>[engram_demo_v1.py]"]
  node_hash_mapping["N-gram hash mapping<br/>[engram_demo_v1.py]"]
  node_embedding["Multi-head embedding<br/>[engram_demo_v1.py]"]
  node_short_conv["Short convolution<br/>[engram_demo_v1.py]"]
  node_engram["Engram module<br/>[engram_demo_v1.py]"]
end

subgraph group_backbone["Backbone"]
  node_transformer_block["Transformer block<br/>[engram_demo_v1.py]"]
  node_attention["Mock attention<br/>[engram_demo_v1.py]"]
  node_moe["Mock MoE<br/>[engram_demo_v1.py]"]
end

node_user(("Demo user"))
node_tokenizer_source[("Pretrained tokenizer source")]

node_user -->|"runs"| node_demo_script
node_demo_script -.->|"runs demo"| node_transformer_block
node_engram -->|"constructs"| node_hash_mapping
node_engram -->|"constructs"| node_embedding
node_engram -->|"constructs"| node_short_conv
node_hash_mapping -->|"uses"| node_tokenizer
node_tokenizer -->|"loads tokenizer"| node_tokenizer_source
node_transformer_block -->|"adds memory output"| node_engram
node_transformer_block -->|"calls"| node_attention
node_transformer_block -->|"calls"| node_moe

click node_demo_script "https://github.com/deepseek-ai/engram/blob/main/engram_demo_v1.py"
click node_tokenizer "https://github.com/deepseek-ai/engram/blob/main/engram_demo_v1.py"
click node_hash_mapping "https://github.com/deepseek-ai/engram/blob/main/engram_demo_v1.py"
click node_embedding "https://github.com/deepseek-ai/engram/blob/main/engram_demo_v1.py"
click node_short_conv "https://github.com/deepseek-ai/engram/blob/main/engram_demo_v1.py"
click node_engram "https://github.com/deepseek-ai/engram/blob/main/engram_demo_v1.py"
click node_transformer_block "https://github.com/deepseek-ai/engram/blob/main/engram_demo_v1.py"
click node_attention "https://github.com/deepseek-ai/engram/blob/main/engram_demo_v1.py"
click node_moe "https://github.com/deepseek-ai/engram/blob/main/engram_demo_v1.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_demo_script,node_user toneBlue
class node_tokenizer,node_hash_mapping,node_embedding,node_short_conv,node_engram,node_tokenizer_source toneAmber
class node_transformer_block,node_attention,node_moe toneMint
```

## Components

### Demo

- [Demo script](https://github.com/deepseek-ai/engram/blob/main/engram_demo_v1.py): `engram_demo_v1.py`

### Engram Memory

- [Compressed tokenizer](https://github.com/deepseek-ai/engram/blob/main/engram_demo_v1.py): `engram_demo_v1.py`
- [N-gram hash mapping](https://github.com/deepseek-ai/engram/blob/main/engram_demo_v1.py): `engram_demo_v1.py`
- [Multi-head embedding](https://github.com/deepseek-ai/engram/blob/main/engram_demo_v1.py): `engram_demo_v1.py`
- [Short convolution](https://github.com/deepseek-ai/engram/blob/main/engram_demo_v1.py): `engram_demo_v1.py`
- [Engram module](https://github.com/deepseek-ai/engram/blob/main/engram_demo_v1.py): `engram_demo_v1.py`

### Backbone

- [Transformer block](https://github.com/deepseek-ai/engram/blob/main/engram_demo_v1.py): `engram_demo_v1.py`
- [Mock attention](https://github.com/deepseek-ai/engram/blob/main/engram_demo_v1.py): `engram_demo_v1.py`
- [Mock MoE](https://github.com/deepseek-ai/engram/blob/main/engram_demo_v1.py): `engram_demo_v1.py`

### Other components

- **Demo user**
- **Pretrained tokenizer source**

## Connections

- Demo user → Demo script: runs (evidence: `README.md`)
- Demo script → Transformer block: runs demo (evidence: `engram_demo_v1.py`)
- Engram module → N-gram hash mapping: constructs (evidence: `engram_demo_v1.py`)
- Engram module → Multi-head embedding: constructs (evidence: `engram_demo_v1.py`)
- Engram module → Short convolution: constructs (evidence: `engram_demo_v1.py`)
- N-gram hash mapping → Compressed tokenizer: uses (evidence: `engram_demo_v1.py`)
- Compressed tokenizer → Pretrained tokenizer source: loads tokenizer (evidence: `engram_demo_v1.py`)
- Transformer block → Engram module: adds memory output (evidence: `engram_demo_v1.py`)
- Transformer block → Mock attention: calls (evidence: `engram_demo_v1.py`)
- Transformer block → Mock MoE: calls (evidence: `engram_demo_v1.py`)