How drxikunzhao/minimind works
MiniMind is a from-scratch, lightweight language-model implementation and learning project: users prepare or select data, train a tokenizer and model through supervised and preference/agent training stages, then evaluate or serve generated conversations. Its README also documents distillation, tool use, adaptive thinking, and multiple rollout backends. The graph includes documented capabilities even where source was not sampled; those nodes use supported paths, and uncertain runtime links are omitted. The sampled code confirms the model, dataset formats, LoRA, agent tool execution, and rollout-engine behavior.
Main entry points: model_minimind.py MiniMind Dense and MoE · lm_dataset.py Dataset loaders · model Trained weights · rollout_engine.py Rollout engines
Components 25
Data and tokens
- Training corpora
dataset/dataset.md - Dataset loaders
dataset/lm_dataset.py - Tokenizer assets
model/tokenizer.json - Tokenizer training
trainer/train_tokenizer.py
Model core
- MiniMind Dense and MoE
model/model_minimind.py - LoRA weights
model/model_lora.py - Trained weights
model
Training workflows
- Pretraining
trainer/train_pretrain.py - Supervised tuning
trainer/train_full_sft.py - LoRA tuning
trainer/train_lora.py - DPO preference tuning
trainer/train_dpo.py - PPO tuning
trainer/train_ppo.py - GRPO and CISPO
trainer/train_grpo.py - Agentic RL
trainer/train_agent.py - Model distillation
trainer/train_distillation.py - Rollout engines
trainer/rollout_engine.py
Inference and evaluation
- Interactive inference
eval_llm.py - OpenAI-compatible API
scripts/serve_openai_api.py - Chat Web UI
scripts/web_demo.py - Chat API example
scripts/chat_api.py - Tool-call evaluation
scripts/eval_toolcall.py - Benchmark evaluation
eval_llm.py
Other components
- Engineer
- Chat user
- SGLang server
Connections 43
- Engineer → Training corpora: selects data
- Training corpora → Dataset loaders: loads records
- Dataset loaders → Pretraining: feeds batches
- Dataset loaders → Supervised tuning: feeds batches
- Dataset loaders → LoRA tuning: feeds batches
- Dataset loaders → DPO preference tuning: feeds preferences
- Dataset loaders → PPO tuning: feeds prompts
- Dataset loaders → GRPO and CISPO: feeds prompts
- Dataset loaders → Agentic RL: feeds conversations
- Tokenizer training → Training corpora: reads text
- Tokenizer training → Tokenizer assets: writes assets
- Tokenizer assets → Dataset loaders: encodes data
- MiniMind Dense and MoE → Pretraining: trains model
- MiniMind Dense and MoE → Supervised tuning: trains model
- MiniMind Dense and MoE → LoRA tuning: trains model
- MiniMind Dense and MoE → DPO preference tuning: trains model
- MiniMind Dense and MoE → PPO tuning: trains actor
- MiniMind Dense and MoE → GRPO and CISPO: trains policy
- MiniMind Dense and MoE → Agentic RL: trains policy
- MiniMind Dense and MoE → Model distillation: student model
- LoRA weights → LoRA tuning: provides adapters
- Model distillation → Trained weights: saves weights
- Pretraining → Trained weights: saves weights
- Supervised tuning → Trained weights: saves weights
- LoRA tuning → Trained weights: saves adapters
- DPO preference tuning → Trained weights: saves weights
- PPO tuning → Rollout engines: requests generations
- GRPO and CISPO → Rollout engines: requests generations
- Agentic RL → Rollout engines: requests generations
- Rollout engines → MiniMind Dense and MoE: generates tokens
- Rollout engines → SGLang server: sends HTTP requests
- Chat user → Interactive inference: enters prompts
- Trained weights → Interactive inference: loads weights
- Tokenizer assets → Interactive inference: encodes prompts
- Interactive inference → MiniMind Dense and MoE: generates response
- Chat user → OpenAI-compatible API: sends chat request
- Chat user → Chat Web UI: enters conversation
- Chat user → Chat API example: sends prompt
- MiniMind Dense and MoE → OpenAI-compatible API: serves generation
- MiniMind Dense and MoE → Chat Web UI: serves generation
- MiniMind Dense and MoE → Chat API example: serves generation
- Tool-call evaluation → MiniMind Dense and MoE: evaluates tool calls
- Benchmark evaluation → MiniMind Dense and MoE: runs inference