How konpep-dev/aptgm works
APTGM is a research implementation of a hybrid sequence language model, evaluated on Multi-Query Associative Recall (MQAR). The main workflow generates MQAR batches, trains a selected model, and measures language-model loss and recall accuracy. In the APTGM architecture, each layer blends selective SSM and causal grouped-query attention outputs using a per-token gate. The repository also contains a separate configurable BPE tokenizer and dataset-source definitions. Training scripts and model wiring are partly unsampled; edges for those links rely on README statements or SOURCE INDEX references.
Main entry points: model.py LM Backbone · mqar.py MQAR Generator · block.py APTGM Block · train_aptgm.py APTGM Training
Components 16
Task Data
- MQAR Generator
aptgm/data/mqar.py
Model Architecture
- LM Backbone
aptgm/models/model.py - APTGM Block
aptgm/models/block.py - Selective SSM
aptgm/models/ssm.py - Grouped Attention
aptgm/models/attention.py - Token Gate
aptgm/models/gate.py
Training
- APTGM Training
aptgm/train_aptgm.py - SSM Training
aptgm/train.py - Attention Training
aptgm/train_attention.py - Baseline Training
aptgm/train_baselines.py - Training With Plots
aptgm/train_with_plot.py
Tokenizer
- Dataset Sources
aptgm_tokenizer/datasets.py - Tokenizer Training
aptgm_tokenizer/training.py - BPE Tokenizer
aptgm_tokenizer/core.py - Tokenizer Config
aptgm_tokenizer/config.py
Other components
- Researcher
Connections 20
- Researcher → APTGM Training: starts training
README.md - Researcher → SSM Training: starts training
README.md - Researcher → Attention Training: starts training
README.md - APTGM Training → MQAR Generator: generates batches
aptgm/train_aptgm.py - SSM Training → MQAR Generator: uses batches
aptgm/train.py - Attention Training → MQAR Generator: uses batches
aptgm/train_attention.py - Baseline Training → MQAR Generator: uses batches
aptgm/train_baselines.py - Training With Plots → MQAR Generator: generates batches
aptgm/train_with_plot.py - APTGM Training → LM Backbone: trains model
aptgm/train_aptgm.py - SSM Training → LM Backbone: trains model
aptgm/train.py - Attention Training → LM Backbone: trains model
aptgm/train_attention.py - Baseline Training → LM Backbone: trains model
aptgm/train_baselines.py - Training With Plots → LM Backbone: trains model
aptgm/train_with_plot.py - LM Backbone → APTGM Block: stacks blocks
README.md - APTGM Block → Selective SSM: computes branch
aptgm/models/block.py - APTGM Block → Grouped Attention: computes branch
aptgm/models/block.py - APTGM Block → Token Gate: mixes with gate
aptgm/models/block.py - Researcher → Tokenizer Training: trains tokenizer
- Tokenizer Training → BPE Tokenizer: trains and saves
aptgm_tokenizer/training.py - BPE Tokenizer → Tokenizer Config: uses configuration
aptgm_tokenizer/core.py