How ynkling/clankers works
BDH is a byte-level language-model implementation of a biologically inspired neural architecture. Its main workflow trains on token sequences, predicts next-token logits, and can autoregressively generate text. The core model combines sparse latent neuron activations, causal attention, multiplicative interactions, and residual decoding; separate files explore context-routed channels and explicit Hebbian recurrence. The map reflects the README and sampled source; broader documented claims such as scale-free topology are not independently confirmed by these excerpts.
Main entry points: bdh.py BDH model · train.py Training script · bdh_mc.py Multi-channel BDH · train_two_task.py Two-task training
Components 16
BDH model
- BDH model
bdh.py - Token embedding
bdh.py - Sparse neurons
bdh.py - Causal attention
bdh.py - Neuron interaction
bdh.py - Language readout
bdh.py - Text generation
bdh.py
Memory variants
- Multi-channel BDH
bdh_mc.py - Context gate
bdh_mc.py - Channel weights
bdh_mc.py - Hebbian recurrence
bdh_recurrent.py - Gated memory model
bdh_multichannel.py
Training workflows
- Training script
train.py - Two-task training
train_two_task.py
Other components
- User
- Byte corpus
Connections 18
- User → Training script: starts training
- Byte corpus → Training script: provides bytes
- Training script → BDH model: trains
- BDH model → Token embedding: embeds tokens
- Token embedding → Sparse neurons: projects
- Sparse neurons → Causal attention: queries keys
- Causal attention → Neuron interaction: returns context
- Neuron interaction → Sparse neurons: updates residual
- BDH model → Language readout: projects logits
- BDH model → Text generation: generates tokens
- Text generation → User: returns text
- Training script → Multi-channel BDH: trains variant
- Multi-channel BDH → Context gate: projects context
- Context gate → Channel weights: blends channels
- Channel weights → Multi-channel BDH: supplies weights
- Two-task training → Multi-channel BDH: trains tasks
- Two-task training → Byte corpus: loads task bytes
- Hebbian recurrence → Gated memory model: implements recurrence