How ifm-ai/xllm works
xLLM is a PyTorch framework for training and evaluating long-context language models. The principal workflow prepares text online, feeds packed batches to a configurable model, trains it with distributed parallelism and optimization, then saves checkpoints; evaluation can score task datasets using model inference. The graph preserves README capabilities including dense/MoE/MoVA architectures, custom fused computation, and optional deployment integration. The tree and README support subsystem coverage, but sampled source is limited, so detailed wiring within unsampled stages is intentionally conservative.
Main entry points: train.py Training entry · dataset_streamer.py Dataset streamer · transformer.py Transformer, MoE, MoVA · launch_eval.py Evaluation runner
Components 25
Online data
- Dataset streamer
xllm/data/dataset_streamer/dataset_streamer.py - Tokenizer backends
xllm/data/dataset_streamer/tokenizer/tokenizer.py - Feature builder
xllm/data/dataset_streamer/feature_builder/feature_builder.py - Buffer packing
xllm/data/buffer_assembler/buffer_assembler.py - Training loader
xllm/data/dataloader.py
Model runtime
- Model architectures
xllm/models/build.py - Base model
xllm/models/xllm.py - Transformer, MoE, MoVA
xllm/models/transformer.py - Attention modules
xllm/modules/multihead_attention.py - Expert routing
xllm/modules/moe/moe.py - Moving-average layers
xllm/modules/complex_exponential_moving_average.py - Fused operations
xllm/modules/fused_ops/__init__.py - Native CUDA ops
xllm/csrc/xllm_extension.cc
Training runtime
- Training entry
train.py - Run configuration
xllm/config.py - Parallel training
xllm/distributed/initialize.py - Optimizer and schedule
xllm/optim/optimizer.py - Checkpoint and resume
xllm/checkpointing.py
Evaluation
- Evaluation entry
eval.py - Evaluation runner
xllm/eval/launch_eval.py - Benchmark tasks
xllm/eval/task/base.py - Model inference
xllm/inference.py
Other components
- Training operator
- Text corpus
- xBridges export and serving
Connections 26
- Training operator → Training entry: launches
- Training entry → Run configuration: loads settings
- Text corpus → Dataset streamer: provides text
- Dataset streamer → Tokenizer backends: tokenizes
- Dataset streamer → Feature builder: builds features
- Dataset streamer → Buffer packing: prepares buffers
- Buffer packing → Training loader: packs batches
- Training entry → Training loader: consumes batches
- Training entry → Model architectures: builds model
- Model architectures → Base model: constructs
- Base model → Transformer, MoE, MoVA: implements
- Transformer, MoE, MoVA → Attention modules: uses
- Transformer, MoE, MoVA → Expert routing: uses
- Transformer, MoE, MoVA → Moving-average layers: uses
- Attention modules → Fused operations: dispatches ops
- Moving-average layers → Fused operations: calls scan
- Fused operations → Native CUDA ops: dispatches kernels
- Training entry → Parallel training: initializes parallelism
- Training entry → Optimizer and schedule: trains with
- Training entry → Checkpoint and resume: saves and resumes
- Training operator → Evaluation entry: launches evaluation
- Evaluation entry → Evaluation runner: starts runner
- Evaluation runner → Benchmark tasks: selects tasks
- Evaluation runner → Model inference: scores model
- Model inference → Base model: runs
- Checkpoint and resume → xBridges export and serving: exports or serves