# tdrussell/qlora-pipe architecture

> Architecture diagram of the GitHub repository tdrussell/qlora-pipe, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tdrussell/qlora-pipe
- Repository: https://github.com/tdrussell/qlora-pipe

## Overview

qlora-pipe is a configured LLM fine-tuning script. The main workflow is to launch training with a config, prepare and batch datasets, construct a model split into pipeline stages, run distributed training, and save models or resumable checkpoints. Documented capabilities also include multiple dataset formats and mixing, LoRA/QLoRA or full fine-tuning, evaluation, and TensorBoard metrics. The graph follows relationships visible in excerpts, the source index, or README; several documented features have no sampled implementation details.

Main entry points: [`train.py`](https://github.com/tdrussell/qlora-pipe/blob/main/train.py) (Training entrypoint), [`models/pipeline_model.py`](https://github.com/tdrussell/qlora-pipe/blob/main/models/pipeline_model.py) (Pipeline model loader), [`utils/engine.py`](https://github.com/tdrussell/qlora-pipe/blob/main/utils/engine.py) (Pipeline engine), [`utils/utils.py`](https://github.com/tdrussell/qlora-pipe/blob/main/utils/utils.py) (Shared runtime utilities)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_workflow["Training workflow"]
  node_config["Training config"]
  node_trainer["Training entrypoint<br/>[train.py]"]
end

subgraph group_data["Data preparation"]
  node_dataset["Dataset preparation<br/>[dataset_utils.py]"]
  node_loader["Pipeline data loader<br/>[dataloader.py]"]
  node_axolotl["Axolotl datasets"]
end

subgraph group_model["Model and kernels"]
  node_model_families["Model families<br/>[models.py]"]
  node_pipeline_model["Pipeline model loader<br/>[pipeline_model.py]"]
  node_pipe_layers["Pipeline layers<br/>[layers.py]"]
  node_ce_loss["Fused cross entropy"]
  node_kernel_utils["Kernel utilities<br/>[utils.py]"]
  node_hqq["HQQ quantization<br/>[hqq_utils.py]"]
  node_unsloth["Unsloth helpers<br/>[unsloth_utils.py]"]
end

subgraph group_runtime["Training runtime"]
  node_engine["Pipeline engine<br/>[engine.py]"]
  node_metrics["Evaluation and metrics<br/>[engine.py]"]
  node_saver["Model and state saver<br/>[saver.py]"]
  node_shared_utils["Shared runtime utilities<br/>[utils.py]"]
end

node_operator(("Training operator"))
node_model_files[("Model checkpoints")]

node_operator -->|"edits"| node_config
node_operator -->|"launches"| node_trainer
node_config -->|"configures"| node_trainer
node_trainer -->|"prepares data"| node_dataset
node_trainer -->|"creates loader"| node_loader
node_dataset -->|"loads Axolotl data"| node_axolotl
node_trainer -->|"builds model"| node_model_families
node_model_families -->|"specifies layers"| node_pipe_layers
node_model_families -->|"inherits loader"| node_pipeline_model
node_pipeline_model -->|"loads weights"| node_model_files
node_pipeline_model -->|"uses quantization"| node_hqq
node_pipe_layers -->|"uses loss"| node_ce_loss
node_ce_loss -->|"imports helpers"| node_kernel_utils
node_trainer -->|"initializes engine"| node_engine
node_engine -->|"pulls batches"| node_loader
node_engine -.->|"computes metrics"| node_metrics
node_trainer -->|"saves training state"| node_saver
node_trainer -->|"uses helpers"| node_unsloth
node_pipeline_model -->|"uses utilities"| node_shared_utils
node_engine -->|"uses utilities"| node_shared_utils
node_loader -->|"uses utilities"| node_shared_utils
node_saver -->|"uses utilities"| node_shared_utils

click node_config "https://github.com/tdrussell/qlora-pipe/tree/main/examples"
click node_trainer "https://github.com/tdrussell/qlora-pipe/blob/main/train.py"
click node_dataset "https://github.com/tdrussell/qlora-pipe/blob/main/utils/dataset_utils.py"
click node_loader "https://github.com/tdrussell/qlora-pipe/blob/main/utils/dataloader.py"
click node_axolotl "https://github.com/tdrussell/qlora-pipe/tree/main/axolotl"
click node_model_families "https://github.com/tdrussell/qlora-pipe/blob/main/models/models.py"
click node_pipeline_model "https://github.com/tdrussell/qlora-pipe/blob/main/models/pipeline_model.py"
click node_pipe_layers "https://github.com/tdrussell/qlora-pipe/blob/main/models/layers.py"
click node_ce_loss "https://github.com/tdrussell/qlora-pipe/blob/main/kernels/cross_entropy_loss.py"
click node_kernel_utils "https://github.com/tdrussell/qlora-pipe/blob/main/kernels/utils.py"
click node_hqq "https://github.com/tdrussell/qlora-pipe/blob/main/utils/hqq_utils.py"
click node_unsloth "https://github.com/tdrussell/qlora-pipe/blob/main/utils/unsloth_utils.py"
click node_engine "https://github.com/tdrussell/qlora-pipe/blob/main/utils/engine.py"
click node_metrics "https://github.com/tdrussell/qlora-pipe/blob/main/utils/engine.py"
click node_saver "https://github.com/tdrussell/qlora-pipe/blob/main/utils/saver.py"
click node_shared_utils "https://github.com/tdrussell/qlora-pipe/blob/main/utils/utils.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_config,node_trainer toneBlue
class node_dataset,node_loader,node_axolotl,node_model_files toneAmber
class node_model_families,node_pipeline_model,node_pipe_layers,node_ce_loss,node_kernel_utils,node_hqq,node_unsloth toneMint
class node_engine,node_metrics,node_saver,node_shared_utils toneRose
class node_operator toneIndigo
```

## Components

### Training workflow

- [Training config](https://github.com/tdrussell/qlora-pipe/tree/main/examples): `examples`
- [Training entrypoint](https://github.com/tdrussell/qlora-pipe/blob/main/train.py): `train.py`

### Data preparation

- [Dataset preparation](https://github.com/tdrussell/qlora-pipe/blob/main/utils/dataset_utils.py): `utils/dataset_utils.py`
- [Pipeline data loader](https://github.com/tdrussell/qlora-pipe/blob/main/utils/dataloader.py): `utils/dataloader.py`
- [Axolotl datasets](https://github.com/tdrussell/qlora-pipe/tree/main/axolotl): `axolotl`

### Model and kernels

- [Model families](https://github.com/tdrussell/qlora-pipe/blob/main/models/models.py): `models/models.py`
- [Pipeline model loader](https://github.com/tdrussell/qlora-pipe/blob/main/models/pipeline_model.py): `models/pipeline_model.py`
- [Pipeline layers](https://github.com/tdrussell/qlora-pipe/blob/main/models/layers.py): `models/layers.py`
- [Fused cross entropy](https://github.com/tdrussell/qlora-pipe/blob/main/kernels/cross_entropy_loss.py): `kernels/cross_entropy_loss.py`
- [Kernel utilities](https://github.com/tdrussell/qlora-pipe/blob/main/kernels/utils.py): `kernels/utils.py`
- [HQQ quantization](https://github.com/tdrussell/qlora-pipe/blob/main/utils/hqq_utils.py): `utils/hqq_utils.py`
- [Unsloth helpers](https://github.com/tdrussell/qlora-pipe/blob/main/utils/unsloth_utils.py): `utils/unsloth_utils.py`

### Training runtime

- [Pipeline engine](https://github.com/tdrussell/qlora-pipe/blob/main/utils/engine.py): `utils/engine.py`
- [Evaluation and metrics](https://github.com/tdrussell/qlora-pipe/blob/main/utils/engine.py): `utils/engine.py`
- [Model and state saver](https://github.com/tdrussell/qlora-pipe/blob/main/utils/saver.py): `utils/saver.py`
- [Shared runtime utilities](https://github.com/tdrussell/qlora-pipe/blob/main/utils/utils.py): `utils/utils.py`

### Other components

- **Training operator**
- **Model checkpoints**

## Connections

- Training operator → Training config: edits (evidence: `README.md`)
- Training operator → Training entrypoint: launches (evidence: `README.md`)
- Training config → Training entrypoint: configures (evidence: `README.md`)
- Training entrypoint → Dataset preparation: prepares data (evidence: `train.py`)
- Training entrypoint → Pipeline data loader: creates loader (evidence: `train.py`)
- Dataset preparation → Axolotl datasets: loads Axolotl data (evidence: `README.md`)
- Training entrypoint → Model families: builds model (evidence: `train.py`)
- Model families → Pipeline layers: specifies layers (evidence: `models/models.py`)
- Model families → Pipeline model loader: inherits loader (evidence: `models/models.py`)
- Pipeline model loader → Model checkpoints: loads weights (evidence: `models/pipeline_model.py`)
- Pipeline model loader → HQQ quantization: uses quantization (evidence: `models/pipeline_model.py`)
- Pipeline layers → Fused cross entropy: uses loss (evidence: `models/layers.py`)
- Fused cross entropy → Kernel utilities: imports helpers (evidence: `kernels/cross_entropy_loss.py`)
- Training entrypoint → Pipeline engine: initializes engine (evidence: `train.py`)
- Pipeline engine → Pipeline data loader: pulls batches (evidence: `utils/engine.py`)
- Pipeline engine → Evaluation and metrics: computes metrics (evidence: `utils/engine.py`)
- Training entrypoint → Model and state saver: saves training state (evidence: `train.py`)
- Training entrypoint → Unsloth helpers: uses helpers (evidence: `train.py`)
- Pipeline model loader → Shared runtime utilities: uses utilities (evidence: `models/pipeline_model.py`)
- Pipeline engine → Shared runtime utilities: uses utilities (evidence: `utils/engine.py`)
- Pipeline data loader → Shared runtime utilities: uses utilities (evidence: `utils/dataloader.py`)
- Model and state saver → Shared runtime utilities: uses utilities (evidence: `utils/saver.py`)
