How tdrussell/qlora-pipe works
qlora-pipe is a configured LLM fine-tuning script. The main workflow is to launch training with a config, prepare and batch datasets, construct a model split into pipeline stages, run distributed training, and save models or resumable checkpoints. Documented capabilities also include multiple dataset formats and mixing, LoRA/QLoRA or full fine-tuning, evaluation, and TensorBoard metrics. The graph follows relationships visible in excerpts, the source index, or README; several documented features have no sampled implementation details.
Main entry points: train.py Training entrypoint · pipeline_model.py Pipeline model loader · engine.py Pipeline engine · utils.py Shared runtime utilities
Components 18
Training workflow
- Training config
examples - Training entrypoint
train.py
Data preparation
- Dataset preparation
utils/dataset_utils.py - Pipeline data loader
utils/dataloader.py - Axolotl datasets
axolotl
Model and kernels
- Model families
models/models.py - Pipeline model loader
models/pipeline_model.py - Pipeline layers
models/layers.py - Fused cross entropy
kernels/cross_entropy_loss.py - Kernel utilities
kernels/utils.py - HQQ quantization
utils/hqq_utils.py - Unsloth helpers
utils/unsloth_utils.py
Training runtime
- Pipeline engine
utils/engine.py - Evaluation and metrics
utils/engine.py - Model and state saver
utils/saver.py - Shared runtime utilities
utils/utils.py
Other components
- Training operator
- Model checkpoints
Connections 22
- Training operator → Training config: edits
README.md - Training operator → Training entrypoint: launches
README.md - Training config → Training entrypoint: configures
README.md - Training entrypoint → Dataset preparation: prepares data
train.py - Training entrypoint → Pipeline data loader: creates loader
train.py - Dataset preparation → Axolotl datasets: loads Axolotl data
README.md - Training entrypoint → Model families: builds model
train.py - Model families → Pipeline layers: specifies layers
models/models.py - Model families → Pipeline model loader: inherits loader
models/models.py - Pipeline model loader → Model checkpoints: loads weights
models/pipeline_model.py - Pipeline model loader → HQQ quantization: uses quantization
models/pipeline_model.py - Pipeline layers → Fused cross entropy: uses loss
models/layers.py - Fused cross entropy → Kernel utilities: imports helpers
kernels/cross_entropy_loss.py - Training entrypoint → Pipeline engine: initializes engine
train.py - Pipeline engine → Pipeline data loader: pulls batches
utils/engine.py - Pipeline engine → Evaluation and metrics: computes metrics
utils/engine.py - Training entrypoint → Model and state saver: saves training state
train.py - Training entrypoint → Unsloth helpers: uses helpers
train.py - Pipeline model loader → Shared runtime utilities: uses utilities
models/pipeline_model.py - Pipeline engine → Shared runtime utilities: uses utilities
utils/engine.py - Pipeline data loader → Shared runtime utilities: uses utilities
utils/dataloader.py - Model and state saver → Shared runtime utilities: uses utilities
utils/saver.py