# konpep-dev/aptgm architecture

> Architecture diagram of the GitHub repository konpep-dev/aptgm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/konpep-dev/aptgm
- Repository: https://github.com/konpep-dev/aptgm

## Overview

APTGM is a research implementation of a hybrid sequence language model, evaluated on Multi-Query Associative Recall (MQAR). The main workflow generates MQAR batches, trains a selected model, and measures language-model loss and recall accuracy. In the APTGM architecture, each layer blends selective SSM and causal grouped-query attention outputs using a per-token gate. The repository also contains a separate configurable BPE tokenizer and dataset-source definitions. Training scripts and model wiring are partly unsampled; edges for those links rely on README statements or SOURCE INDEX references.

Main entry points: [`aptgm/models/model.py`](https://github.com/konpep-dev/aptgm/blob/main/aptgm/models/model.py) (LM Backbone), [`aptgm/data/mqar.py`](https://github.com/konpep-dev/aptgm/blob/main/aptgm/data/mqar.py) (MQAR Generator), [`aptgm/models/block.py`](https://github.com/konpep-dev/aptgm/blob/main/aptgm/models/block.py) (APTGM Block), [`aptgm/train_aptgm.py`](https://github.com/konpep-dev/aptgm/blob/main/aptgm/train_aptgm.py) (APTGM Training)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Task Data"]
  node_mqar["MQAR Generator<br/>[mqar.py]"]
end

subgraph group_model["Model Architecture"]
  node_lm["LM Backbone<br/>[model.py]"]
  node_aptgm_block["APTGM Block<br/>[block.py]"]
  node_ssm["Selective SSM<br/>[ssm.py]"]
  node_attention["Grouped Attention<br/>[attention.py]"]
  node_gate["Token Gate<br/>[gate.py]"]
end

subgraph group_training["Training"]
  node_train_aptgm["APTGM Training<br/>[train_aptgm.py]"]
  node_train_ssm["SSM Training<br/>[train.py]"]
  node_train_attention["Attention Training<br/>[train_attention.py]"]
  node_train_baselines["Baseline Training<br/>[train_baselines.py]"]
  node_train_plot["Training With Plots<br/>[train_with_plot.py]"]
end

subgraph group_tokenizer["Tokenizer"]
  node_tokenizer_data["Dataset Sources<br/>[datasets.py]"]
  node_tokenizer_training["Tokenizer Training<br/>[training.py]"]
  node_tokenizer_core["BPE Tokenizer<br/>[core.py]"]
  node_tokenizer_config["Tokenizer Config<br/>[config.py]"]
end

node_user(("Researcher"))

node_user -->|"starts training"| node_train_aptgm
node_user -->|"starts training"| node_train_ssm
node_user -->|"starts training"| node_train_attention
node_train_aptgm -->|"generates batches"| node_mqar
node_train_ssm -->|"uses batches"| node_mqar
node_train_attention -->|"uses batches"| node_mqar
node_train_baselines -->|"uses batches"| node_mqar
node_train_plot -->|"generates batches"| node_mqar
node_train_aptgm -->|"trains model"| node_lm
node_train_ssm -->|"trains model"| node_lm
node_train_attention -->|"trains model"| node_lm
node_train_baselines -->|"trains model"| node_lm
node_train_plot -->|"trains model"| node_lm
node_lm -->|"stacks blocks"| node_aptgm_block
node_aptgm_block -->|"computes branch"| node_ssm
node_aptgm_block -->|"computes branch"| node_attention
node_aptgm_block -->|"mixes with gate"| node_gate
node_user -.->|"trains tokenizer"| node_tokenizer_training
node_tokenizer_training -->|"trains and saves"| node_tokenizer_core
node_tokenizer_core -->|"uses configuration"| node_tokenizer_config

click node_mqar "https://github.com/konpep-dev/aptgm/blob/main/aptgm/data/mqar.py"
click node_lm "https://github.com/konpep-dev/aptgm/blob/main/aptgm/models/model.py"
click node_aptgm_block "https://github.com/konpep-dev/aptgm/blob/main/aptgm/models/block.py"
click node_ssm "https://github.com/konpep-dev/aptgm/blob/main/aptgm/models/ssm.py"
click node_attention "https://github.com/konpep-dev/aptgm/blob/main/aptgm/models/attention.py"
click node_gate "https://github.com/konpep-dev/aptgm/blob/main/aptgm/models/gate.py"
click node_train_aptgm "https://github.com/konpep-dev/aptgm/blob/main/aptgm/train_aptgm.py"
click node_train_ssm "https://github.com/konpep-dev/aptgm/blob/main/aptgm/train.py"
click node_train_attention "https://github.com/konpep-dev/aptgm/blob/main/aptgm/train_attention.py"
click node_train_baselines "https://github.com/konpep-dev/aptgm/blob/main/aptgm/train_baselines.py"
click node_train_plot "https://github.com/konpep-dev/aptgm/blob/main/aptgm/train_with_plot.py"
click node_tokenizer_data "https://github.com/konpep-dev/aptgm/blob/main/aptgm_tokenizer/datasets.py"
click node_tokenizer_training "https://github.com/konpep-dev/aptgm/blob/main/aptgm_tokenizer/training.py"
click node_tokenizer_core "https://github.com/konpep-dev/aptgm/blob/main/aptgm_tokenizer/core.py"
click node_tokenizer_config "https://github.com/konpep-dev/aptgm/blob/main/aptgm_tokenizer/config.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_mqar toneBlue
class node_lm,node_aptgm_block,node_ssm,node_attention,node_gate toneAmber
class node_train_aptgm,node_train_ssm,node_train_attention,node_train_baselines,node_train_plot toneMint
class node_tokenizer_data,node_tokenizer_training,node_tokenizer_core,node_tokenizer_config toneRose
class node_user toneIndigo
```

## Components

### Task Data

- [MQAR Generator](https://github.com/konpep-dev/aptgm/blob/main/aptgm/data/mqar.py): `aptgm/data/mqar.py`

### Model Architecture

- [LM Backbone](https://github.com/konpep-dev/aptgm/blob/main/aptgm/models/model.py): `aptgm/models/model.py`
- [APTGM Block](https://github.com/konpep-dev/aptgm/blob/main/aptgm/models/block.py): `aptgm/models/block.py`
- [Selective SSM](https://github.com/konpep-dev/aptgm/blob/main/aptgm/models/ssm.py): `aptgm/models/ssm.py`
- [Grouped Attention](https://github.com/konpep-dev/aptgm/blob/main/aptgm/models/attention.py): `aptgm/models/attention.py`
- [Token Gate](https://github.com/konpep-dev/aptgm/blob/main/aptgm/models/gate.py): `aptgm/models/gate.py`

### Training

- [APTGM Training](https://github.com/konpep-dev/aptgm/blob/main/aptgm/train_aptgm.py): `aptgm/train_aptgm.py`
- [SSM Training](https://github.com/konpep-dev/aptgm/blob/main/aptgm/train.py): `aptgm/train.py`
- [Attention Training](https://github.com/konpep-dev/aptgm/blob/main/aptgm/train_attention.py): `aptgm/train_attention.py`
- [Baseline Training](https://github.com/konpep-dev/aptgm/blob/main/aptgm/train_baselines.py): `aptgm/train_baselines.py`
- [Training With Plots](https://github.com/konpep-dev/aptgm/blob/main/aptgm/train_with_plot.py): `aptgm/train_with_plot.py`

### Tokenizer

- [Dataset Sources](https://github.com/konpep-dev/aptgm/blob/main/aptgm_tokenizer/datasets.py): `aptgm_tokenizer/datasets.py`
- [Tokenizer Training](https://github.com/konpep-dev/aptgm/blob/main/aptgm_tokenizer/training.py): `aptgm_tokenizer/training.py`
- [BPE Tokenizer](https://github.com/konpep-dev/aptgm/blob/main/aptgm_tokenizer/core.py): `aptgm_tokenizer/core.py`
- [Tokenizer Config](https://github.com/konpep-dev/aptgm/blob/main/aptgm_tokenizer/config.py): `aptgm_tokenizer/config.py`

### Other components

- **Researcher**

## Connections

- Researcher → APTGM Training: starts training (evidence: `README.md`)
- Researcher → SSM Training: starts training (evidence: `README.md`)
- Researcher → Attention Training: starts training (evidence: `README.md`)
- APTGM Training → MQAR Generator: generates batches (evidence: `aptgm/train_aptgm.py`)
- SSM Training → MQAR Generator: uses batches (evidence: `aptgm/train.py`)
- Attention Training → MQAR Generator: uses batches (evidence: `aptgm/train_attention.py`)
- Baseline Training → MQAR Generator: uses batches (evidence: `aptgm/train_baselines.py`)
- Training With Plots → MQAR Generator: generates batches (evidence: `aptgm/train_with_plot.py`)
- APTGM Training → LM Backbone: trains model (evidence: `aptgm/train_aptgm.py`)
- SSM Training → LM Backbone: trains model (evidence: `aptgm/train.py`)
- Attention Training → LM Backbone: trains model (evidence: `aptgm/train_attention.py`)
- Baseline Training → LM Backbone: trains model (evidence: `aptgm/train_baselines.py`)
- Training With Plots → LM Backbone: trains model (evidence: `aptgm/train_with_plot.py`)
- LM Backbone → APTGM Block: stacks blocks (evidence: `README.md`)
- APTGM Block → Selective SSM: computes branch (evidence: `aptgm/models/block.py`)
- APTGM Block → Grouped Attention: computes branch (evidence: `aptgm/models/block.py`)
- APTGM Block → Token Gate: mixes with gate (evidence: `aptgm/models/block.py`)
- Researcher → Tokenizer Training: trains tokenizer
- Tokenizer Training → BPE Tokenizer: trains and saves (evidence: `aptgm_tokenizer/training.py`)
- BPE Tokenizer → Tokenizer Config: uses configuration (evidence: `aptgm_tokenizer/core.py`)
