# mohebabdelhafidh/memory-in-action-conditioned-world-models-gru-vs.-transformer-vs.-xlstm architecture

> Architecture diagram of the GitHub repository mohebabdelhafidh/memory-in-action-conditioned-world-models-gru-vs.-transformer-vs.-xlstm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mohebabdelhafidh/memory-in-action-conditioned-world-models-gru-vs.-transformer-vs.-xlstm
- Repository: https://github.com/mohebabdelhafidh/memory-in-action-conditioned-world-models-gru-vs.-transformer-vs.-xlstm

## Overview

This is a small research system for comparing recurrent sequence cores in action-conditioned world models on a partially observable cue-recall task. Episodes are generated from the simulator and loaded as padded sequences; training fits the model to predict next frames and rewards. The trained model is then evaluated and used for imagination-based planning/visualization. The shared model encodes frames and actions, runs a selectable dynamics core, and decodes predictions. Some evaluation and planning modules are unsampled, so their internal connections are omitted or marked unverified.

Main entry points: [`models/world_model.py`](https://github.com/mohebabdelhafidh/memory-in-action-conditioned-world-models-gru-vs.-transformer-vs.-xlstm/blob/main/models/world_model.py) (Action-conditioned model), [`eval/visualize.py`](https://github.com/mohebabdelhafidh/memory-in-action-conditioned-world-models-gru-vs.-transformer-vs.-xlstm/blob/main/eval/visualize.py) (Imagination visualization), [`train.py`](https://github.com/mohebabdelhafidh/memory-in-action-conditioned-world-models-gru-vs.-transformer-vs.-xlstm/blob/main/train.py) (Model training), [`data/dataset.py`](https://github.com/mohebabdelhafidh/memory-in-action-conditioned-world-models-gru-vs.-transformer-vs.-xlstm/blob/main/data/dataset.py) (Episode dataset)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_task_data["Task and data"]
  node_simulator["Cue-recall simulator<br/>[cue_recall.py]"]
  node_collection["Episode collection<br/>[collect.py]"]
  node_episode_data["Episode dataset<br/>[dataset.py]"]
end

subgraph group_model["World model"]
  node_world_model["Action-conditioned model<br/>[world_model.py]"]
  node_encoder["Frame encoder<br/>[encoder.py]"]
  node_decoder["Frame decoder<br/>[decoder.py]"]
  node_core_interface["Dynamics core interface<br/>[base.py]"]
  node_gru["GRU core<br/>[gru.py]"]
  node_memoryless["Memoryless control<br/>[memoryless.py]"]
  node_transformer["Transformer core<br/>[transformer.py]"]
  node_xlstm["xLSTM core<br/>[xlstm_core.py]"]
end

subgraph group_experiments["Training and evaluation"]
  node_trainer["Model training<br/>[train.py]"]
  node_evaluation["Model evaluation"]
  node_efficiency["Efficiency tests<br/>[efficiency.py]"]
  node_gap_sweep["Gap sweeps<br/>[gap_sweep.py]"]
end

subgraph group_planning["Planning and analysis"]
  node_planning["CEM model planning<br/>[cem_mpc.py]"]
  node_visualization["Imagination visualization<br/>[visualize.py]"]
end

node_researcher(("Researcher"))

node_collection -->|"runs episodes"| node_simulator
node_episode_data -->|"loads episodes"| node_collection
node_trainer -->|"loads batches"| node_episode_data
node_trainer -->|"trains"| node_world_model
node_world_model -->|"encodes frames"| node_encoder
node_world_model -->|"decodes predictions"| node_decoder
node_world_model -->|"runs dynamics"| node_core_interface
node_world_model -->|"builds core"| node_gru
node_world_model -->|"builds core"| node_memoryless
node_world_model -->|"builds core"| node_transformer
node_world_model -->|"builds core"| node_xlstm
node_evaluation -.->|"evaluates model"| node_world_model
node_efficiency -->|"tests efficiency"| node_world_model
node_gap_sweep -->|"loads episodes"| node_episode_data
node_gap_sweep -->|"sweeps gaps"| node_world_model
node_researcher -.->|"starts training"| node_trainer
node_planning -.->|"plans by imagination"| node_world_model
node_visualization -->|"builds context"| node_simulator
node_visualization -->|"imagines outcomes"| node_world_model
node_visualization -->|"converts frames"| node_decoder

click node_simulator "https://github.com/mohebabdelhafidh/memory-in-action-conditioned-world-models-gru-vs.-transformer-vs.-xlstm/blob/main/envs/cue_recall.py"
click node_collection "https://github.com/mohebabdelhafidh/memory-in-action-conditioned-world-models-gru-vs.-transformer-vs.-xlstm/blob/main/data/collect.py"
click node_episode_data "https://github.com/mohebabdelhafidh/memory-in-action-conditioned-world-models-gru-vs.-transformer-vs.-xlstm/blob/main/data/dataset.py"
click node_trainer "https://github.com/mohebabdelhafidh/memory-in-action-conditioned-world-models-gru-vs.-transformer-vs.-xlstm/blob/main/train.py"
click node_world_model "https://github.com/mohebabdelhafidh/memory-in-action-conditioned-world-models-gru-vs.-transformer-vs.-xlstm/blob/main/models/world_model.py"
click node_encoder "https://github.com/mohebabdelhafidh/memory-in-action-conditioned-world-models-gru-vs.-transformer-vs.-xlstm/blob/main/models/encoder.py"
click node_decoder "https://github.com/mohebabdelhafidh/memory-in-action-conditioned-world-models-gru-vs.-transformer-vs.-xlstm/blob/main/models/decoder.py"
click node_core_interface "https://github.com/mohebabdelhafidh/memory-in-action-conditioned-world-models-gru-vs.-transformer-vs.-xlstm/blob/main/models/cores/base.py"
click node_gru "https://github.com/mohebabdelhafidh/memory-in-action-conditioned-world-models-gru-vs.-transformer-vs.-xlstm/blob/main/models/cores/gru.py"
click node_memoryless "https://github.com/mohebabdelhafidh/memory-in-action-conditioned-world-models-gru-vs.-transformer-vs.-xlstm/blob/main/models/cores/memoryless.py"
click node_transformer "https://github.com/mohebabdelhafidh/memory-in-action-conditioned-world-models-gru-vs.-transformer-vs.-xlstm/blob/main/models/cores/transformer.py"
click node_xlstm "https://github.com/mohebabdelhafidh/memory-in-action-conditioned-world-models-gru-vs.-transformer-vs.-xlstm/blob/main/models/cores/xlstm_core.py"
click node_evaluation "https://github.com/mohebabdelhafidh/memory-in-action-conditioned-world-models-gru-vs.-transformer-vs.-xlstm/tree/main/eval"
click node_efficiency "https://github.com/mohebabdelhafidh/memory-in-action-conditioned-world-models-gru-vs.-transformer-vs.-xlstm/blob/main/eval/efficiency.py"
click node_gap_sweep "https://github.com/mohebabdelhafidh/memory-in-action-conditioned-world-models-gru-vs.-transformer-vs.-xlstm/blob/main/eval/gap_sweep.py"
click node_planning "https://github.com/mohebabdelhafidh/memory-in-action-conditioned-world-models-gru-vs.-transformer-vs.-xlstm/blob/main/planning/cem_mpc.py"
click node_visualization "https://github.com/mohebabdelhafidh/memory-in-action-conditioned-world-models-gru-vs.-transformer-vs.-xlstm/blob/main/eval/visualize.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_simulator,node_collection,node_episode_data toneBlue
class node_world_model,node_encoder,node_decoder,node_core_interface,node_gru,node_memoryless,node_transformer,node_xlstm toneAmber
class node_trainer,node_evaluation,node_efficiency,node_gap_sweep toneMint
class node_planning,node_visualization toneRose
class node_researcher toneIndigo
```

## Components

### Task and data

- [Cue-recall simulator](https://github.com/mohebabdelhafidh/memory-in-action-conditioned-world-models-gru-vs.-transformer-vs.-xlstm/blob/main/envs/cue_recall.py): `envs/cue_recall.py`
- [Episode collection](https://github.com/mohebabdelhafidh/memory-in-action-conditioned-world-models-gru-vs.-transformer-vs.-xlstm/blob/main/data/collect.py): `data/collect.py`
- [Episode dataset](https://github.com/mohebabdelhafidh/memory-in-action-conditioned-world-models-gru-vs.-transformer-vs.-xlstm/blob/main/data/dataset.py): `data/dataset.py`

### World model

- [Action-conditioned model](https://github.com/mohebabdelhafidh/memory-in-action-conditioned-world-models-gru-vs.-transformer-vs.-xlstm/blob/main/models/world_model.py): `models/world_model.py`
- [Frame encoder](https://github.com/mohebabdelhafidh/memory-in-action-conditioned-world-models-gru-vs.-transformer-vs.-xlstm/blob/main/models/encoder.py): `models/encoder.py`
- [Frame decoder](https://github.com/mohebabdelhafidh/memory-in-action-conditioned-world-models-gru-vs.-transformer-vs.-xlstm/blob/main/models/decoder.py): `models/decoder.py`
- [Dynamics core interface](https://github.com/mohebabdelhafidh/memory-in-action-conditioned-world-models-gru-vs.-transformer-vs.-xlstm/blob/main/models/cores/base.py): `models/cores/base.py`
- [GRU core](https://github.com/mohebabdelhafidh/memory-in-action-conditioned-world-models-gru-vs.-transformer-vs.-xlstm/blob/main/models/cores/gru.py): `models/cores/gru.py`
- [Memoryless control](https://github.com/mohebabdelhafidh/memory-in-action-conditioned-world-models-gru-vs.-transformer-vs.-xlstm/blob/main/models/cores/memoryless.py): `models/cores/memoryless.py`
- [Transformer core](https://github.com/mohebabdelhafidh/memory-in-action-conditioned-world-models-gru-vs.-transformer-vs.-xlstm/blob/main/models/cores/transformer.py): `models/cores/transformer.py`
- [xLSTM core](https://github.com/mohebabdelhafidh/memory-in-action-conditioned-world-models-gru-vs.-transformer-vs.-xlstm/blob/main/models/cores/xlstm_core.py): `models/cores/xlstm_core.py`

### Training and evaluation

- [Model training](https://github.com/mohebabdelhafidh/memory-in-action-conditioned-world-models-gru-vs.-transformer-vs.-xlstm/blob/main/train.py): `train.py`
- [Model evaluation](https://github.com/mohebabdelhafidh/memory-in-action-conditioned-world-models-gru-vs.-transformer-vs.-xlstm/tree/main/eval): `eval`
- [Efficiency tests](https://github.com/mohebabdelhafidh/memory-in-action-conditioned-world-models-gru-vs.-transformer-vs.-xlstm/blob/main/eval/efficiency.py): `eval/efficiency.py`
- [Gap sweeps](https://github.com/mohebabdelhafidh/memory-in-action-conditioned-world-models-gru-vs.-transformer-vs.-xlstm/blob/main/eval/gap_sweep.py): `eval/gap_sweep.py`

### Planning and analysis

- [CEM model planning](https://github.com/mohebabdelhafidh/memory-in-action-conditioned-world-models-gru-vs.-transformer-vs.-xlstm/blob/main/planning/cem_mpc.py): `planning/cem_mpc.py`
- [Imagination visualization](https://github.com/mohebabdelhafidh/memory-in-action-conditioned-world-models-gru-vs.-transformer-vs.-xlstm/blob/main/eval/visualize.py): `eval/visualize.py`

### Other components

- **Researcher**

## Connections

- Episode collection → Cue-recall simulator: runs episodes (evidence: `data/collect.py`)
- Episode dataset → Episode collection: loads episodes (evidence: `data/dataset.py`)
- Model training → Episode dataset: loads batches (evidence: `train.py`)
- Model training → Action-conditioned model: trains (evidence: `train.py`)
- Action-conditioned model → Frame encoder: encodes frames (evidence: `models/world_model.py`)
- Action-conditioned model → Frame decoder: decodes predictions (evidence: `models/world_model.py`)
- Action-conditioned model → Dynamics core interface: runs dynamics (evidence: `models/world_model.py`)
- Action-conditioned model → GRU core: builds core (evidence: `models/world_model.py`)
- Action-conditioned model → Memoryless control: builds core (evidence: `models/world_model.py`)
- Action-conditioned model → Transformer core: builds core (evidence: `models/world_model.py`)
- Action-conditioned model → xLSTM core: builds core (evidence: `models/world_model.py`)
- Model evaluation → Action-conditioned model: evaluates model (evidence: `eval/visualize.py`)
- Efficiency tests → Action-conditioned model: tests efficiency (evidence: `eval/efficiency.py`)
- Gap sweeps → Episode dataset: loads episodes (evidence: `eval/gap_sweep.py`)
- Gap sweeps → Action-conditioned model: sweeps gaps (evidence: `eval/gap_sweep.py`)
- Researcher → Model training: starts training
- CEM model planning → Action-conditioned model: plans by imagination
- Imagination visualization → Cue-recall simulator: builds context (evidence: `eval/visualize.py`)
- Imagination visualization → Action-conditioned model: imagines outcomes (evidence: `eval/visualize.py`)
- Imagination visualization → Frame decoder: converts frames (evidence: `eval/visualize.py`)
