How fabienfrfr/kairos works
KairosFM is an experimental multimodal foundation model and low-compute training stack. Its documented workflow builds or loads multimodal examples, encodes modalities with a shared byte-level tokenizer, packs training data, trains a hybrid diffusion model, then generates continuations or publishes a trained model. The graph preserves the model’s attention, routing, and residual architecture alongside dataset construction, training, and generation. Evidence is limited to the README and excerpts/SOURCE INDEX references; unsampled pipeline internals and notebook stages are not expanded into speculative links.
Main entry points: pipeline.py Training pipeline · modeling.py Kairos model · kairos_pretraining.py Pretraining workflow · generation.py Diffusion generation
Components 14
Data preparation
- Multimodal corpus builder
scripts/pretrain/build_keep_it_simple_multimodal.py - Multimodal tokenizer
kairos/tokenizer.py - Dataset packing
kairos/dataset.py
Model runtime
- Kairos model
kairos/modeling.py - SWA and DeltaNet
kairos/attentions.py - Diffusion generation
kairos/generation.py
Training and evaluation
- Training pipeline
kairos/pipeline.py - Diffusion trainer
kairos/trainer.py - Compute diagnostics
kairos/utils.py - Pretraining workflow
notebook/kairos_pretraining.py
Inference and publishing
- Hugging Face Hub
- Generated tokens
- Trained model
Other components
- Researcher
Connections 21
- Researcher → Multimodal corpus builder: builds corpus
README.md - Multimodal corpus builder → Hugging Face Hub: pushes dataset
README.md - Researcher → Pretraining workflow: runs training
README.md - Pretraining workflow → Multimodal corpus builder: invokes builder
notebook/kairos_pretraining.py - Pretraining workflow → Hugging Face Hub: loads datasets
README.md - Pretraining workflow → Training pipeline: configures workflow
notebook/kairos_pretraining.py - Training pipeline → Multimodal tokenizer: initializes
kairos/pipeline.py - Training pipeline → Dataset packing: uses datasets
kairos/pipeline.py - Training pipeline → Kairos model: configures model
README.md - Training pipeline → Diffusion trainer: creates trainer
kairos/pipeline.py - Diffusion trainer → Kairos model: trains model
kairos/trainer.py - Kairos model → SWA and DeltaNet: uses attention
kairos/modeling.py - Kairos model → Diffusion generation: inherits generation
kairos/modeling.py - Training pipeline → Compute diagnostics: uses diagnostics
kairos/pipeline.py - Training pipeline → Trained model: saves checkpoints
README.md - Researcher → Training pipeline: requests generation
kairos/pipeline.py - Training pipeline → Diffusion generation: delegates generation
kairos/pipeline.py - Diffusion generation → Kairos model: runs forward pass
kairos/generation.py - Diffusion generation → Generated tokens: returns sequences
kairos/generation.py - Researcher → Training pipeline: publishes model
README.md - Training pipeline → Hugging Face Hub: pushes model
README.md