How dataflowr/llm_efficiency works
This repository is an LLM-efficiency homework codebase built around minGPT, with two product workflows: autoregressive generation accelerated by KV caching, and parameter-efficient fine-tuning with LoRA. The sorting-task demos orchestrate model setup, training, and evaluation; the shared trainer supplies the training loop. KV caching is implemented as a GPT variant, while LoRA replaces attention projections with adapter-aware linear layers. README capabilities include cache benchmarking and LoRA merge/de-merge; sampled excerpts do not establish all internal calls, so uncertain wiring is omitted.
Main entry points: trainer.py Training loop · kv_cache.py Cached GPT · demo_sort_kv.py KV sort demo · demo_sort_lora.py LoRA sort demo
Components 14
Efficiency Experiments
- KV sort demo
kv_cache/demo_sort_kv.py - Sorting data
kv_cache/demo_sort_kv.py - Cache benchmark
kv_cache/benchmark.py - LoRA sort demo
lora/demo_sort_lora.py - Sorting data
lora/demo_sort_lora.py
GPT Modeling
- Cached GPT
kv_cache/kv_cache.py - Cached attention
kv_cache/kv_cache.py - minGPT model
mingpt/model.py - LoRA GPT
lora/lora.py - LoRA layers
lora/lora.py
Training Core
- Training loop
mingpt/trainer.py - Model utilities
mingpt/utils.py
Other components
- Homework user
- PyTorch
Connections 18
- Homework user → KV sort demo: runs
- Homework user → LoRA sort demo: runs
- KV sort demo → Sorting data: creates
- KV sort demo → Cached GPT: instantiates
- KV sort demo → Training loop: trains with
- Training loop → Sorting data: loads batches
- Training loop → Cached GPT: trains
- Cached GPT → Cached attention: uses
- Cached GPT → minGPT model: extends
- Cache benchmark → Cached GPT: benchmarks
- LoRA sort demo → Sorting data: creates
- LoRA sort demo → LoRA GPT: instantiates
- LoRA sort demo → Training loop: trains with
- Training loop → LoRA GPT: trains
- LoRA GPT → LoRA layers: uses
- LoRA GPT → minGPT model: extends
- Training loop → Model utilities: imports config
- minGPT model → PyTorch: uses