# snap-stanford/memorilla architecture

> Architecture diagram of the GitHub repository snap-stanford/memorilla, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/snap-stanford/memorilla
- Repository: https://github.com/snap-stanford/memorilla

## Overview

Memorilla trains a question-conditioned memory module that compresses document embeddings into a fixed set of tokens for a frozen LLM. Its main use path loads a collection and question, encodes them, reads memory tokens, and generates an answer; separate training and benchmark workflows prepare data, train the module, and score results. The repository also includes PersonalizationV4 data generation, TextWorld reinforcement-learning tools, and memory-cache profiling. Some of these are represented by directory-level nodes because their source was not sampled; their internal call relationships are therefore not asserted.

Main entry points: [`memorilla/vllm.py`](https://github.com/snap-stanford/memorilla/blob/main/memorilla/vllm.py) (Inference wrapper), [`memorilla/data.py`](https://github.com/snap-stanford/memorilla/blob/main/memorilla/data.py) (Dataset access), [`memorilla/memory.py`](https://github.com/snap-stanford/memorilla/blob/main/memorilla/memory.py) (Memory module), [`evaluate.py`](https://github.com/snap-stanford/memorilla/blob/main/evaluate.py) (Benchmark runner)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Memory and inference"]
  node_vllm["Inference wrapper<br/>[vllm.py]"]
  node_encoder["Embedding encoder<br/>[embeddings.py]"]
  node_memory["Memory module<br/>[memory.py]"]
  node_cache[("Memory cache<br/>[memory.py]")]
  node_training_llm["Training wrapper<br/>[llm.py]"]
end

subgraph group_training["Training and data"]
  node_train["Training entrypoint<br/>[train.py]"]
  node_data["Dataset access<br/>[data.py]"]
  node_embedding_store[("Embedding store<br/>[embeddings.py]")]
  node_paths["Data paths<br/>[paths.py]"]
end

subgraph group_evaluation["Evaluation"]
  node_evaluate["Benchmark runner<br/>[evaluate.py]"]
  node_baseline["Baseline runner"]
  node_benchmarks["Benchmark protocols<br/>[benchmarks.py]"]
  node_metrics["Scoring<br/>[metrics.py]"]
end

subgraph group_extensions["Data generation and tools"]
  node_personalization["PersonalizationV4 generation<br/>[generate.py]"]
  node_personalization_prompts["Generation prompts<br/>[prompts.py]"]
  node_personalization_build["Personalization dataset build<br/>[build_dataset.py]"]
  node_rl["TextWorld RL tools"]
  node_profiling["Memory profiling<br/>[run.py]"]
  node_profile_cache["FP8 cache profiling<br/>[fp8_cache.py]"]
end

node_user(("User"))
node_dataset[("Benchmark data")]
node_decoder["Frozen LLM"]

node_user -->|"asks question"| node_vllm
node_vllm -->|"encodes text"| node_encoder
node_vllm -->|"reads memory"| node_memory
node_vllm -->|"generates answer"| node_decoder
node_vllm -->|"writes collection"| node_cache
node_train -->|"prepares batches"| node_data
node_train -->|"loads embeddings"| node_embedding_store
node_train -->|"trains through"| node_training_llm
node_training_llm -->|"computes tokens"| node_memory
node_training_llm -->|"runs decoder"| node_decoder
node_data -->|"resolves data"| node_paths
node_embedding_store -->|"uses data root"| node_paths
node_evaluate -->|"loads examples"| node_data
node_evaluate -->|"runs inference"| node_vllm
node_evaluate -->|"selects protocol"| node_benchmarks
node_evaluate -->|"scores results"| node_metrics
node_baseline -->|"loads examples"| node_data
node_baseline -->|"uses protocol"| node_benchmarks
node_baseline -->|"scores results"| node_metrics
node_dataset -->|"supplies splits"| node_data
node_personalization -->|"uses prompts"| node_personalization_prompts
node_personalization_build -->|"uses generation"| node_personalization
node_profiling -->|"profiles module"| node_memory
node_profile_cache -->|"profiles cache"| node_memory
node_rl -->|"uses memory"| node_memory

click node_vllm "https://github.com/snap-stanford/memorilla/blob/main/memorilla/vllm.py"
click node_encoder "https://github.com/snap-stanford/memorilla/blob/main/memorilla/embeddings.py"
click node_memory "https://github.com/snap-stanford/memorilla/blob/main/memorilla/memory.py"
click node_cache "https://github.com/snap-stanford/memorilla/blob/main/memorilla/memory.py"
click node_training_llm "https://github.com/snap-stanford/memorilla/blob/main/memorilla/llm.py"
click node_train "https://github.com/snap-stanford/memorilla/blob/main/train.py"
click node_data "https://github.com/snap-stanford/memorilla/blob/main/memorilla/data.py"
click node_embedding_store "https://github.com/snap-stanford/memorilla/blob/main/memorilla/embeddings.py"
click node_paths "https://github.com/snap-stanford/memorilla/blob/main/memorilla/paths.py"
click node_evaluate "https://github.com/snap-stanford/memorilla/blob/main/evaluate.py"
click node_baseline "https://github.com/snap-stanford/memorilla/blob/main/evaluate_baselines.py"
click node_benchmarks "https://github.com/snap-stanford/memorilla/blob/main/memorilla/benchmarks.py"
click node_metrics "https://github.com/snap-stanford/memorilla/blob/main/memorilla/metrics.py"
click node_personalization "https://github.com/snap-stanford/memorilla/blob/main/personalizationv4/generate.py"
click node_personalization_prompts "https://github.com/snap-stanford/memorilla/blob/main/personalizationv4/prompts.py"
click node_personalization_build "https://github.com/snap-stanford/memorilla/blob/main/personalizationv4/build_dataset.py"
click node_rl "https://github.com/snap-stanford/memorilla/tree/main/rl"
click node_profiling "https://github.com/snap-stanford/memorilla/blob/main/profiling/run.py"
click node_profile_cache "https://github.com/snap-stanford/memorilla/blob/main/profiling/fp8_cache.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_vllm,node_encoder,node_memory,node_cache,node_training_llm,node_user toneBlue
class node_train,node_data,node_embedding_store,node_paths,node_dataset toneAmber
class node_evaluate,node_baseline,node_benchmarks,node_metrics toneMint
class node_personalization,node_personalization_prompts,node_personalization_build,node_rl,node_profiling,node_profile_cache toneRose
class node_decoder toneIndigo
```

## Components

### Memory and inference

- [Inference wrapper](https://github.com/snap-stanford/memorilla/blob/main/memorilla/vllm.py): `memorilla/vllm.py`
- [Embedding encoder](https://github.com/snap-stanford/memorilla/blob/main/memorilla/embeddings.py): `memorilla/embeddings.py`
- [Memory module](https://github.com/snap-stanford/memorilla/blob/main/memorilla/memory.py): `memorilla/memory.py`
- [Memory cache](https://github.com/snap-stanford/memorilla/blob/main/memorilla/memory.py): `memorilla/memory.py`
- [Training wrapper](https://github.com/snap-stanford/memorilla/blob/main/memorilla/llm.py): `memorilla/llm.py`

### Training and data

- [Training entrypoint](https://github.com/snap-stanford/memorilla/blob/main/train.py): `train.py`
- [Dataset access](https://github.com/snap-stanford/memorilla/blob/main/memorilla/data.py): `memorilla/data.py`
- [Embedding store](https://github.com/snap-stanford/memorilla/blob/main/memorilla/embeddings.py): `memorilla/embeddings.py`
- [Data paths](https://github.com/snap-stanford/memorilla/blob/main/memorilla/paths.py): `memorilla/paths.py`

### Evaluation

- [Benchmark runner](https://github.com/snap-stanford/memorilla/blob/main/evaluate.py): `evaluate.py`
- [Baseline runner](https://github.com/snap-stanford/memorilla/blob/main/evaluate_baselines.py): `evaluate_baselines.py`
- [Benchmark protocols](https://github.com/snap-stanford/memorilla/blob/main/memorilla/benchmarks.py): `memorilla/benchmarks.py`
- [Scoring](https://github.com/snap-stanford/memorilla/blob/main/memorilla/metrics.py): `memorilla/metrics.py`

### Data generation and tools

- [PersonalizationV4 generation](https://github.com/snap-stanford/memorilla/blob/main/personalizationv4/generate.py): `personalizationv4/generate.py`
- [Generation prompts](https://github.com/snap-stanford/memorilla/blob/main/personalizationv4/prompts.py): `personalizationv4/prompts.py`
- [Personalization dataset build](https://github.com/snap-stanford/memorilla/blob/main/personalizationv4/build_dataset.py): `personalizationv4/build_dataset.py`
- [TextWorld RL tools](https://github.com/snap-stanford/memorilla/tree/main/rl): `rl`
- [Memory profiling](https://github.com/snap-stanford/memorilla/blob/main/profiling/run.py): `profiling/run.py`
- [FP8 cache profiling](https://github.com/snap-stanford/memorilla/blob/main/profiling/fp8_cache.py): `profiling/fp8_cache.py`

### Other components

- **User**
- **Benchmark data**
- **Frozen LLM**

## Connections

- User → Inference wrapper: asks question (evidence: `README.md`)
- Inference wrapper → Embedding encoder: encodes text (evidence: `memorilla/vllm.py`)
- Inference wrapper → Memory module: reads memory (evidence: `memorilla/vllm.py`)
- Inference wrapper → Frozen LLM: generates answer (evidence: `README.md`)
- Inference wrapper → Memory cache: writes collection (evidence: `README.md`)
- Training entrypoint → Dataset access: prepares batches (evidence: `train.py`)
- Training entrypoint → Embedding store: loads embeddings (evidence: `train.py`)
- Training entrypoint → Training wrapper: trains through (evidence: `train.py`)
- Training wrapper → Memory module: computes tokens (evidence: `memorilla/llm.py`)
- Training wrapper → Frozen LLM: runs decoder (evidence: `memorilla/llm.py`)
- Dataset access → Data paths: resolves data (evidence: `memorilla/data.py`)
- Embedding store → Data paths: uses data root (evidence: `memorilla/embeddings.py`)
- Benchmark runner → Dataset access: loads examples (evidence: `evaluate.py`)
- Benchmark runner → Inference wrapper: runs inference (evidence: `evaluate.py`)
- Benchmark runner → Benchmark protocols: selects protocol (evidence: `evaluate.py`)
- Benchmark runner → Scoring: scores results (evidence: `evaluate.py`)
- Baseline runner → Dataset access: loads examples (evidence: `evaluate_baselines.py`)
- Baseline runner → Benchmark protocols: uses protocol (evidence: `evaluate_baselines.py`)
- Baseline runner → Scoring: scores results (evidence: `evaluate_baselines.py`)
- Benchmark data → Dataset access: supplies splits (evidence: `README.md`)
- PersonalizationV4 generation → Generation prompts: uses prompts (evidence: `personalizationv4/generate.py`)
- Personalization dataset build → PersonalizationV4 generation: uses generation (evidence: `personalizationv4/build_dataset.py`)
- Memory profiling → Memory module: profiles module (evidence: `profiling/run.py`)
- FP8 cache profiling → Memory module: profiles cache (evidence: `profiling/fp8_cache.py`)
- TextWorld RL tools → Memory module: uses memory (evidence: `rl/export_memory.py`)
