How deepseek-ai/engram works
This repository is a standalone Engram demonstration, not a production model-serving system. Its intended workflow is to run the demo, form token n-grams, hash them into static memory, and fuse retrieved embeddings with hidden states in selected transformer blocks. The implementation also includes a compressed tokenizer and short convolution; attention and MoE are explicitly mocked. The available excerpts do not show the full script entry-point wiring or Engram forward pass, so those internal links are limited to relationships visible in the excerpts or documented in the README.
Main entry points: engram_demo_v1.py Transformer block
Components 11
Demo
- Demo script
engram_demo_v1.py
Engram Memory
- Compressed tokenizer
engram_demo_v1.py - N-gram hash mapping
engram_demo_v1.py - Multi-head embedding
engram_demo_v1.py - Short convolution
engram_demo_v1.py - Engram module
engram_demo_v1.py
Backbone
- Transformer block
engram_demo_v1.py - Mock attention
engram_demo_v1.py - Mock MoE
engram_demo_v1.py
Other components
- Demo user
- Pretrained tokenizer source
Connections 10
- Demo user → Demo script: runs
README.md - Demo script → Transformer block: runs demo
engram_demo_v1.py - Engram module → N-gram hash mapping: constructs
engram_demo_v1.py - Engram module → Multi-head embedding: constructs
engram_demo_v1.py - Engram module → Short convolution: constructs
engram_demo_v1.py - N-gram hash mapping → Compressed tokenizer: uses
engram_demo_v1.py - Compressed tokenizer → Pretrained tokenizer source: loads tokenizer
engram_demo_v1.py - Transformer block → Engram module: adds memory output
engram_demo_v1.py - Transformer block → Mock attention: calls
engram_demo_v1.py - Transformer block → Mock MoE: calls
engram_demo_v1.py