How samamabile/irontorch works
IronTorch is a C++ text-classification and next-token-prediction system that can train embeddings through NTP. The main workflow accepts text or labeled CSV data, builds BPE tokens, trains a CNN, and uses saved model/tokenizer files for classification or generation. The README documents the network variants and pretraining option; sampled code confirms the API, tokenizer, training, and inference responsibilities. Some training and architecture wiring is inferred from documented capabilities, so uncertain call paths are omitted.
Main entry points: model.cpp Model · train.cpp Trainer · inference.cpp Inference · irontorch.cpp IronTorch API
Components 22
API and inference
- CLI entry
api.cpp - IronTorch API
irontorch.cpp - Inference
inference.cpp
Text preparation
- Dataset tokenizer
tokenizer.cpp - BPE tokenizer
bpe.cpp
Training workflows
- Trainer
train.cpp - Batcher
batcher.cpp - Optimizer
optimizer.hpp - Loss functions
loss.cpp
Neural network
- Architecture setup
arch.cpp - Model
model.cpp - Embedding layer
embedding.cpp - Conv1D layer
conv1d.cpp - Parallel Conv1D
doubleConv1d.cpp - Activations
relu.cpp - Pooling
pool.cpp - Dense layer
dense.cpp - Output heads
sigmoid.cpp
Numerical runtime
- Matrix operations
matrix.cpp
Other components
- User
- Text or CSV
- Model and BPE files
Connections 30
- User → CLI entry: provides input
- Text or CSV → CLI entry: supplies data
- CLI entry → IronTorch API: dispatches
- IronTorch API → Trainer: trains
- IronTorch API → Inference: loads or predicts
- Trainer → Dataset tokenizer: prepares tokens
- Dataset tokenizer → BPE tokenizer: fits and tokenizes
- Trainer → Batcher: batches data
- Trainer → Architecture setup: sets up layers
- Architecture setup → Model: initializes
- Trainer → Model: trains
- Trainer → Loss functions: computes loss
- Trainer → Optimizer: updates parameters
- Model → Embedding layer: dispatches layers
- Model → Parallel Conv1D: dispatches layers
- Model → Conv1D layer: dispatches layers
- Model → Activations: dispatches layers
- Model → Pooling: dispatches layers
- Model → Dense layer: dispatches layers
- Model → Output heads: dispatches layers
- Embedding layer → Matrix operations: uses operations
- Conv1D layer → Matrix operations: uses operations
- IronTorch API → Text or CSV: reads data
- Trainer → Model and BPE files: saves weights
- Dataset tokenizer → Model and BPE files: saves BPE rules
- Inference → Model and BPE files: loads artifacts
- Inference → Dataset tokenizer: loads and tokenizes
- Inference → Model: runs forward pass
- Inference → Embedding layer: creates embedding
- Inference → User: returns prediction or text