# dataflowr/llm_efficiency architecture

> Architecture diagram of the GitHub repository dataflowr/llm_efficiency, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dataflowr/llm_efficiency
- Repository: https://github.com/dataflowr/llm_efficiency

## Overview

This repository is an LLM-efficiency homework codebase built around minGPT, with two product workflows: autoregressive generation accelerated by KV caching, and parameter-efficient fine-tuning with LoRA. The sorting-task demos orchestrate model setup, training, and evaluation; the shared trainer supplies the training loop. KV caching is implemented as a GPT variant, while LoRA replaces attention projections with adapter-aware linear layers. README capabilities include cache benchmarking and LoRA merge/de-merge; sampled excerpts do not establish all internal calls, so uncertain wiring is omitted.

Main entry points: [`mingpt/trainer.py`](https://github.com/dataflowr/llm_efficiency/blob/main/mingpt/trainer.py) (Training loop), [`kv_cache/kv_cache.py`](https://github.com/dataflowr/llm_efficiency/blob/main/kv_cache/kv_cache.py) (Cached GPT), [`kv_cache/demo_sort_kv.py`](https://github.com/dataflowr/llm_efficiency/blob/main/kv_cache/demo_sort_kv.py) (KV sort demo), [`lora/demo_sort_lora.py`](https://github.com/dataflowr/llm_efficiency/blob/main/lora/demo_sort_lora.py) (LoRA sort demo)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_experiments["Efficiency Experiments"]
  node_kv_demo["KV sort demo<br/>[demo_sort_kv.py]"]
  node_sort_data["Sorting data<br/>[demo_sort_kv.py]"]
  node_cache_benchmark["Cache benchmark<br/>[benchmark.py]"]
  node_lora_demo["LoRA sort demo<br/>[demo_sort_lora.py]"]
  node_lora_data["Sorting data<br/>[demo_sort_lora.py]"]
end

subgraph group_modeling["GPT Modeling"]
  node_cache_model["Cached GPT<br/>[kv_cache.py]"]
  node_cache_attention["Cached attention<br/>[kv_cache.py]"]
  node_base_gpt["minGPT model<br/>[model.py]"]
  node_lora_model["LoRA GPT<br/>[lora.py]"]
  node_lora_layers["LoRA layers<br/>[lora.py]"]
end

subgraph group_training["Training Core"]
  node_trainer["Training loop<br/>[trainer.py]"]
  node_utilities["Model utilities<br/>[utils.py]"]
end

node_learner(("Homework user"))
node_torch["PyTorch"]

node_learner -->|"runs"| node_kv_demo
node_learner -->|"runs"| node_lora_demo
node_kv_demo -->|"creates"| node_sort_data
node_kv_demo -->|"instantiates"| node_cache_model
node_kv_demo -->|"trains with"| node_trainer
node_trainer -->|"loads batches"| node_sort_data
node_trainer -->|"trains"| node_cache_model
node_cache_model -->|"uses"| node_cache_attention
node_cache_model -->|"extends"| node_base_gpt
node_cache_benchmark -->|"benchmarks"| node_cache_model
node_lora_demo -->|"creates"| node_lora_data
node_lora_demo -->|"instantiates"| node_lora_model
node_lora_demo -->|"trains with"| node_trainer
node_trainer -->|"trains"| node_lora_model
node_lora_model -->|"uses"| node_lora_layers
node_lora_model -->|"extends"| node_base_gpt
node_trainer -->|"imports config"| node_utilities
node_base_gpt -->|"uses"| node_torch

click node_kv_demo "https://github.com/dataflowr/llm_efficiency/blob/main/kv_cache/demo_sort_kv.py"
click node_sort_data "https://github.com/dataflowr/llm_efficiency/blob/main/kv_cache/demo_sort_kv.py"
click node_cache_model "https://github.com/dataflowr/llm_efficiency/blob/main/kv_cache/kv_cache.py"
click node_cache_attention "https://github.com/dataflowr/llm_efficiency/blob/main/kv_cache/kv_cache.py"
click node_base_gpt "https://github.com/dataflowr/llm_efficiency/blob/main/mingpt/model.py"
click node_trainer "https://github.com/dataflowr/llm_efficiency/blob/main/mingpt/trainer.py"
click node_cache_benchmark "https://github.com/dataflowr/llm_efficiency/blob/main/kv_cache/benchmark.py"
click node_lora_demo "https://github.com/dataflowr/llm_efficiency/blob/main/lora/demo_sort_lora.py"
click node_lora_data "https://github.com/dataflowr/llm_efficiency/blob/main/lora/demo_sort_lora.py"
click node_lora_model "https://github.com/dataflowr/llm_efficiency/blob/main/lora/lora.py"
click node_lora_layers "https://github.com/dataflowr/llm_efficiency/blob/main/lora/lora.py"
click node_utilities "https://github.com/dataflowr/llm_efficiency/blob/main/mingpt/utils.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_kv_demo,node_sort_data,node_cache_benchmark,node_lora_demo,node_lora_data,node_learner toneBlue
class node_cache_model,node_cache_attention,node_base_gpt,node_lora_model,node_lora_layers toneAmber
class node_trainer,node_utilities toneMint
class node_torch toneIndigo
```

## Components

### Efficiency Experiments

- [KV sort demo](https://github.com/dataflowr/llm_efficiency/blob/main/kv_cache/demo_sort_kv.py): `kv_cache/demo_sort_kv.py`
- [Sorting data](https://github.com/dataflowr/llm_efficiency/blob/main/kv_cache/demo_sort_kv.py): `kv_cache/demo_sort_kv.py`
- [Cache benchmark](https://github.com/dataflowr/llm_efficiency/blob/main/kv_cache/benchmark.py): `kv_cache/benchmark.py`
- [LoRA sort demo](https://github.com/dataflowr/llm_efficiency/blob/main/lora/demo_sort_lora.py): `lora/demo_sort_lora.py`
- [Sorting data](https://github.com/dataflowr/llm_efficiency/blob/main/lora/demo_sort_lora.py): `lora/demo_sort_lora.py`

### GPT Modeling

- [Cached GPT](https://github.com/dataflowr/llm_efficiency/blob/main/kv_cache/kv_cache.py): `kv_cache/kv_cache.py`
- [Cached attention](https://github.com/dataflowr/llm_efficiency/blob/main/kv_cache/kv_cache.py): `kv_cache/kv_cache.py`
- [minGPT model](https://github.com/dataflowr/llm_efficiency/blob/main/mingpt/model.py): `mingpt/model.py`
- [LoRA GPT](https://github.com/dataflowr/llm_efficiency/blob/main/lora/lora.py): `lora/lora.py`
- [LoRA layers](https://github.com/dataflowr/llm_efficiency/blob/main/lora/lora.py): `lora/lora.py`

### Training Core

- [Training loop](https://github.com/dataflowr/llm_efficiency/blob/main/mingpt/trainer.py): `mingpt/trainer.py`
- [Model utilities](https://github.com/dataflowr/llm_efficiency/blob/main/mingpt/utils.py): `mingpt/utils.py`

### Other components

- **Homework user**
- **PyTorch**

## Connections

- Homework user → KV sort demo: runs
- Homework user → LoRA sort demo: runs
- KV sort demo → Sorting data: creates
- KV sort demo → Cached GPT: instantiates
- KV sort demo → Training loop: trains with
- Training loop → Sorting data: loads batches
- Training loop → Cached GPT: trains
- Cached GPT → Cached attention: uses
- Cached GPT → minGPT model: extends
- Cache benchmark → Cached GPT: benchmarks
- LoRA sort demo → Sorting data: creates
- LoRA sort demo → LoRA GPT: instantiates
- LoRA sort demo → Training loop: trains with
- Training loop → LoRA GPT: trains
- LoRA GPT → LoRA layers: uses
- LoRA GPT → minGPT model: extends
- Training loop → Model utilities: imports config
- minGPT model → PyTorch: uses