How hans0809/minimind-in-depth works
MiniMind-in-Depth is a Chinese-language tutorial series for understanding how large language models are built and trained, based on the MiniMind project. The reader’s workflow is to study topics spanning tokenizer construction and model architecture, then pretraining, instruction tuning and DPO, followed by LoRA and distillation. This repository contains explanatory documents rather than an executable training pipeline; the graph therefore shows documented learning topics, not runtime calls. The README establishes the curriculum, but no source excerpts were available to verify finer implementation relationships.
Main entry points: 1-如何从头训练tokenizer.md Tokenizer training · 2-一行代码之差，模型性能提升背后的RMSNorm玄机.md RMSNorm · 3-原始Transformer的位置编码及其缺陷.md Position encoding · 4-旋转位置编码原理与应用全解析.md RoPE
Components 13
Model foundations
- Tokenizer training
src/1-如何从头训练tokenizer.md - RMSNorm
src/2-一行代码之差，模型性能提升背后的RMSNorm玄机.md - Position encoding
src/3-原始Transformer的位置编码及其缺陷.md - RoPE
src/4-旋转位置编码原理与应用全解析.md
Model architecture
- Efficient attention
src/5-魔改的注意力机制，细数当代LLM的效率优化手段.md - Mixture of experts
src/6-从稠密到稀疏，详解专家混合模型MOE.md - Model assembly
src/7-像搭积木一样构建一个大模型.md
Training and alignment
- Pretraining
src/8-LLM预训练流程全解.md - Instruction tuning
src/9-指令微调详解-让大模型从“能说”变得“会听”.md - DPO alignment
src/10-DPO-大模型对齐训练的新范式.md
Optimization and compression
- LoRA fine-tuning
src/11-LoRA-LLM轻量化微调的利器.md - Model distillation
src/12-从白盒到黑盒，全面掌握大模型蒸馏技术.md
Other components
- Learner
Connections 12
- Learner → Tokenizer training: studies
- Learner → RMSNorm: studies
- Learner → Position encoding: studies
- Learner → RoPE: studies
- Learner → Efficient attention: studies
- Learner → Mixture of experts: studies
- Learner → Model assembly: studies
- Learner → Pretraining: studies
- Learner → Instruction tuning: studies
- Learner → DPO alignment: studies
- Learner → LoRA fine-tuning: studies
- Learner → Model distillation: studies