# ifm-ai/xllm architecture

> Architecture diagram of the GitHub repository ifm-ai/xllm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ifm-ai/xllm
- Repository: https://github.com/ifm-ai/xllm

## Overview

xLLM is a PyTorch framework for training and evaluating long-context language models. The principal workflow prepares text online, feeds packed batches to a configurable model, trains it with distributed parallelism and optimization, then saves checkpoints; evaluation can score task datasets using model inference. The graph preserves README capabilities including dense/MoE/MoVA architectures, custom fused computation, and optional deployment integration. The tree and README support subsystem coverage, but sampled source is limited, so detailed wiring within unsampled stages is intentionally conservative.

Main entry points: [`train.py`](https://github.com/ifm-ai/xllm/blob/main/train.py) (Training entry), [`xllm/data/dataset_streamer/dataset_streamer.py`](https://github.com/ifm-ai/xllm/blob/main/xllm/data/dataset_streamer/dataset_streamer.py) (Dataset streamer), [`xllm/models/transformer.py`](https://github.com/ifm-ai/xllm/blob/main/xllm/models/transformer.py) (Transformer, MoE, MoVA), [`xllm/eval/launch_eval.py`](https://github.com/ifm-ai/xllm/blob/main/xllm/eval/launch_eval.py) (Evaluation runner)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Online data"]
  node_streamer["Dataset streamer"]
  node_tokenizers["Tokenizer backends<br/>[tokenizer.py]"]
  node_features["Feature builder<br/>[feature_builder.py]"]
  node_assembler["Buffer packing"]
  node_dataloader["Training loader<br/>[dataloader.py]"]
end

subgraph group_model["Model runtime"]
  node_models["Model architectures<br/>[build.py]"]
  node_xllmodel["Base model<br/>[xllm.py]"]
  node_architectures["Transformer, MoE, MoVA<br/>[transformer.py]"]
  node_attention["Attention modules"]
  node_moe["Expert routing<br/>[moe.py]"]
  node_cema["Moving-average layers"]
  node_fused["Fused operations<br/>[__init__.py]"]
  node_native["Native CUDA ops<br/>[xllm_extension.cc]"]
end

subgraph group_training["Training runtime"]
  node_train["Training entry<br/>[train.py]"]
  node_config["Run configuration<br/>[config.py]"]
  node_distributed["Parallel training<br/>[initialize.py]"]
  node_optimizer["Optimizer and schedule<br/>[optimizer.py]"]
  node_checkpoint["Checkpoint and resume<br/>[checkpointing.py]"]
end

subgraph group_evaluation["Evaluation"]
  node_evalentry["Evaluation entry<br/>[eval.py]"]
  node_evalrunner["Evaluation runner<br/>[launch_eval.py]"]
  node_tasks["Benchmark tasks<br/>[base.py]"]
  node_inference["Model inference<br/>[inference.py]"]
end

node_operator(("Training operator"))
node_corpus["Text corpus"]
node_xbridges["xBridges export and serving"]

node_operator -->|"launches"| node_train
node_train -->|"loads settings"| node_config
node_corpus -->|"provides text"| node_streamer
node_streamer -->|"tokenizes"| node_tokenizers
node_streamer -->|"builds features"| node_features
node_streamer -->|"prepares buffers"| node_assembler
node_assembler -->|"packs batches"| node_dataloader
node_train -->|"consumes batches"| node_dataloader
node_train -->|"builds model"| node_models
node_models -->|"constructs"| node_xllmodel
node_xllmodel -->|"implements"| node_architectures
node_architectures -->|"uses"| node_attention
node_architectures -->|"uses"| node_moe
node_architectures -->|"uses"| node_cema
node_attention -->|"dispatches ops"| node_fused
node_cema -->|"calls scan"| node_fused
node_fused -->|"dispatches kernels"| node_native
node_train -->|"initializes parallelism"| node_distributed
node_train -->|"trains with"| node_optimizer
node_train -->|"saves and resumes"| node_checkpoint
node_operator -->|"launches evaluation"| node_evalentry
node_evalentry -->|"starts runner"| node_evalrunner
node_evalrunner -->|"selects tasks"| node_tasks
node_evalrunner -->|"scores model"| node_inference
node_inference -->|"runs"| node_xllmodel
node_checkpoint -.->|"exports or serves"| node_xbridges

click node_train "https://github.com/ifm-ai/xllm/blob/main/train.py"
click node_config "https://github.com/ifm-ai/xllm/blob/main/xllm/config.py"
click node_streamer "https://github.com/ifm-ai/xllm/blob/main/xllm/data/dataset_streamer/dataset_streamer.py"
click node_tokenizers "https://github.com/ifm-ai/xllm/blob/main/xllm/data/dataset_streamer/tokenizer/tokenizer.py"
click node_features "https://github.com/ifm-ai/xllm/blob/main/xllm/data/dataset_streamer/feature_builder/feature_builder.py"
click node_assembler "https://github.com/ifm-ai/xllm/blob/main/xllm/data/buffer_assembler/buffer_assembler.py"
click node_dataloader "https://github.com/ifm-ai/xllm/blob/main/xllm/data/dataloader.py"
click node_models "https://github.com/ifm-ai/xllm/blob/main/xllm/models/build.py"
click node_xllmodel "https://github.com/ifm-ai/xllm/blob/main/xllm/models/xllm.py"
click node_architectures "https://github.com/ifm-ai/xllm/blob/main/xllm/models/transformer.py"
click node_attention "https://github.com/ifm-ai/xllm/blob/main/xllm/modules/multihead_attention.py"
click node_moe "https://github.com/ifm-ai/xllm/blob/main/xllm/modules/moe/moe.py"
click node_cema "https://github.com/ifm-ai/xllm/blob/main/xllm/modules/complex_exponential_moving_average.py"
click node_fused "https://github.com/ifm-ai/xllm/blob/main/xllm/modules/fused_ops/__init__.py"
click node_native "https://github.com/ifm-ai/xllm/blob/main/xllm/csrc/xllm_extension.cc"
click node_distributed "https://github.com/ifm-ai/xllm/blob/main/xllm/distributed/initialize.py"
click node_optimizer "https://github.com/ifm-ai/xllm/blob/main/xllm/optim/optimizer.py"
click node_checkpoint "https://github.com/ifm-ai/xllm/blob/main/xllm/checkpointing.py"
click node_evalentry "https://github.com/ifm-ai/xllm/blob/main/eval.py"
click node_evalrunner "https://github.com/ifm-ai/xllm/blob/main/xllm/eval/launch_eval.py"
click node_tasks "https://github.com/ifm-ai/xllm/blob/main/xllm/eval/task/base.py"
click node_inference "https://github.com/ifm-ai/xllm/blob/main/xllm/inference.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_streamer,node_tokenizers,node_features,node_assembler,node_dataloader toneBlue
class node_models,node_xllmodel,node_architectures,node_attention,node_moe,node_cema,node_fused,node_native toneAmber
class node_train,node_config,node_distributed,node_optimizer,node_checkpoint toneMint
class node_evalentry,node_evalrunner,node_tasks,node_inference toneRose
class node_operator,node_corpus,node_xbridges toneIndigo
```

## Components

### Online data

- [Dataset streamer](https://github.com/ifm-ai/xllm/blob/main/xllm/data/dataset_streamer/dataset_streamer.py): `xllm/data/dataset_streamer/dataset_streamer.py`
- [Tokenizer backends](https://github.com/ifm-ai/xllm/blob/main/xllm/data/dataset_streamer/tokenizer/tokenizer.py): `xllm/data/dataset_streamer/tokenizer/tokenizer.py`
- [Feature builder](https://github.com/ifm-ai/xllm/blob/main/xllm/data/dataset_streamer/feature_builder/feature_builder.py): `xllm/data/dataset_streamer/feature_builder/feature_builder.py`
- [Buffer packing](https://github.com/ifm-ai/xllm/blob/main/xllm/data/buffer_assembler/buffer_assembler.py): `xllm/data/buffer_assembler/buffer_assembler.py`
- [Training loader](https://github.com/ifm-ai/xllm/blob/main/xllm/data/dataloader.py): `xllm/data/dataloader.py`

### Model runtime

- [Model architectures](https://github.com/ifm-ai/xllm/blob/main/xllm/models/build.py): `xllm/models/build.py`
- [Base model](https://github.com/ifm-ai/xllm/blob/main/xllm/models/xllm.py): `xllm/models/xllm.py`
- [Transformer, MoE, MoVA](https://github.com/ifm-ai/xllm/blob/main/xllm/models/transformer.py): `xllm/models/transformer.py`
- [Attention modules](https://github.com/ifm-ai/xllm/blob/main/xllm/modules/multihead_attention.py): `xllm/modules/multihead_attention.py`
- [Expert routing](https://github.com/ifm-ai/xllm/blob/main/xllm/modules/moe/moe.py): `xllm/modules/moe/moe.py`
- [Moving-average layers](https://github.com/ifm-ai/xllm/blob/main/xllm/modules/complex_exponential_moving_average.py): `xllm/modules/complex_exponential_moving_average.py`
- [Fused operations](https://github.com/ifm-ai/xllm/blob/main/xllm/modules/fused_ops/__init__.py): `xllm/modules/fused_ops/__init__.py`
- [Native CUDA ops](https://github.com/ifm-ai/xllm/blob/main/xllm/csrc/xllm_extension.cc): `xllm/csrc/xllm_extension.cc`

### Training runtime

- [Training entry](https://github.com/ifm-ai/xllm/blob/main/train.py): `train.py`
- [Run configuration](https://github.com/ifm-ai/xllm/blob/main/xllm/config.py): `xllm/config.py`
- [Parallel training](https://github.com/ifm-ai/xllm/blob/main/xllm/distributed/initialize.py): `xllm/distributed/initialize.py`
- [Optimizer and schedule](https://github.com/ifm-ai/xllm/blob/main/xllm/optim/optimizer.py): `xllm/optim/optimizer.py`
- [Checkpoint and resume](https://github.com/ifm-ai/xllm/blob/main/xllm/checkpointing.py): `xllm/checkpointing.py`

### Evaluation

- [Evaluation entry](https://github.com/ifm-ai/xllm/blob/main/eval.py): `eval.py`
- [Evaluation runner](https://github.com/ifm-ai/xllm/blob/main/xllm/eval/launch_eval.py): `xllm/eval/launch_eval.py`
- [Benchmark tasks](https://github.com/ifm-ai/xllm/blob/main/xllm/eval/task/base.py): `xllm/eval/task/base.py`
- [Model inference](https://github.com/ifm-ai/xllm/blob/main/xllm/inference.py): `xllm/inference.py`

### Other components

- **Training operator**
- **Text corpus**
- **xBridges export and serving**

## Connections

- Training operator → Training entry: launches
- Training entry → Run configuration: loads settings
- Text corpus → Dataset streamer: provides text
- Dataset streamer → Tokenizer backends: tokenizes
- Dataset streamer → Feature builder: builds features
- Dataset streamer → Buffer packing: prepares buffers
- Buffer packing → Training loader: packs batches
- Training entry → Training loader: consumes batches
- Training entry → Model architectures: builds model
- Model architectures → Base model: constructs
- Base model → Transformer, MoE, MoVA: implements
- Transformer, MoE, MoVA → Attention modules: uses
- Transformer, MoE, MoVA → Expert routing: uses
- Transformer, MoE, MoVA → Moving-average layers: uses
- Attention modules → Fused operations: dispatches ops
- Moving-average layers → Fused operations: calls scan
- Fused operations → Native CUDA ops: dispatches kernels
- Training entry → Parallel training: initializes parallelism
- Training entry → Optimizer and schedule: trains with
- Training entry → Checkpoint and resume: saves and resumes
- Training operator → Evaluation entry: launches evaluation
- Evaluation entry → Evaluation runner: starts runner
- Evaluation runner → Benchmark tasks: selects tasks
- Evaluation runner → Model inference: scores model
- Model inference → Base model: runs
- Checkpoint and resume → xBridges export and serving: exports or serves