# jiwubu/minimind architecture

> Architecture diagram of the GitHub repository jiwubu/minimind, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jiwubu/minimind
- Repository: https://github.com/jiwubu/minimind

## Overview

MiniMind is a from-scratch, lightweight language-model implementation and learning project. Its main workflow is to prepare or generate training data, train or adapt a MiniMind model, then serve or interact with it and evaluate its outputs. The repository also has a substantial specialized math-data and math-evaluation branch, plus distillation, preference optimization, RL and tool-using agent workflows. The graph reflects documented capabilities and SOURCE INDEX references; unsampled modules are represented only at the level supported by the tree and README.

Main entry points: [`model/model_minimind.py`](https://github.com/jiwubu/minimind/blob/master/model/model_minimind.py) (MiniMind architecture), [`dataset/lm_dataset.py`](https://github.com/jiwubu/minimind/blob/master/dataset/lm_dataset.py) (Language datasets), [`trainer/trainer_utils.py`](https://github.com/jiwubu/minimind/blob/master/trainer/trainer_utils.py) (Training utilities), [`trainer/train_ppo.py`](https://github.com/jiwubu/minimind/blob/master/trainer/train_ppo.py) (PPO and GRPO)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_model["Model and tokenizer"]
  node_model_arch["MiniMind architecture<br/>[model_minimind.py]"]
  node_lora["LoRA adaptation<br/>[model_lora.py]"]
  node_tokenizer_train["Tokenizer training<br/>[train_tokenizer.py]"]
end

subgraph group_data["Data and math"]
  node_lm_data["Language datasets<br/>[lm_dataset.py]"]
  node_math_format["Math trace format<br/>[math_format.py]"]
  node_addsub_data["Add/sub data"]
  node_muldiv_data["Mul/div data"]
end

subgraph group_training["Training workflows"]
  node_pretrain["Pretraining and SFT<br/>[train_pretrain.py]"]
  node_full_sft["Full SFT<br/>[train_full_sft.py]"]
  node_lora_train["LoRA training<br/>[train_lora.py]"]
  node_preference["DPO training<br/>[train_dpo.py]"]
  node_rl_train["PPO and GRPO<br/>[train_ppo.py]"]
  node_agent_train["Agentic RL<br/>[train_agent.py]"]
  node_distillation["Model distillation"]
  node_rollout["Rollout engines<br/>[rollout_engine.py]"]
  node_training_utils["Training utilities<br/>[trainer_utils.py]"]
end

subgraph group_inference["Inference and tools"]
  node_chat_api["Chat examples<br/>[chat_api.py]"]
  node_openai_api["OpenAI-compatible API"]
  node_web_demo["Chat web UI<br/>[web_demo.py]"]
  node_convert["Merge and export<br/>[convert_model.py]"]
end

subgraph group_evaluation["Evaluation"]
  node_math_eval["Math evaluation<br/>[eval_math.py]"]
  node_tool_eval["Tool-call evaluation<br/>[eval_toolcall.py]"]
  node_general_eval["Model evaluation<br/>[eval_llm.py]"]
end

node_learner(("Model learner"))
node_chat_user(("Chat user"))
node_sglang["SGLang service"]
node_model_user(("Model user"))

node_learner -->|"trains tokenizer"| node_tokenizer_train
node_learner -->|"generates math data"| node_addsub_data
node_learner -->|"generates math data"| node_muldiv_data
node_learner -->|"trains model"| node_pretrain
node_learner -->|"fine-tunes model"| node_full_sft
node_learner -->|"optimizes preferences"| node_preference
node_learner -->|"trains with RL"| node_rl_train
node_learner -->|"trains tool agent"| node_agent_train
node_learner -->|"distills model"| node_distillation
node_addsub_data -->|"uses format"| node_math_format
node_muldiv_data -->|"uses format"| node_math_format
node_pretrain -->|"loads datasets"| node_lm_data
node_pretrain -->|"trains model"| node_model_arch
node_pretrain -->|"uses utilities"| node_training_utils
node_full_sft -->|"loads datasets"| node_lm_data
node_full_sft -->|"trains model"| node_model_arch
node_full_sft -->|"uses utilities"| node_training_utils
node_lora_train -->|"loads datasets"| node_lm_data
node_lora_train -->|"trains model"| node_model_arch
node_lora_train -->|"applies adaptation"| node_lora
node_preference -->|"loads datasets"| node_lm_data
node_preference -->|"trains model"| node_model_arch
node_preference -->|"uses utilities"| node_training_utils
node_rl_train -->|"loads datasets"| node_lm_data
node_rl_train -->|"builds policy"| node_model_arch
node_rl_train -->|"generates rollouts"| node_rollout
node_rl_train -->|"uses utilities"| node_training_utils
node_agent_train -->|"loads datasets"| node_lm_data
node_agent_train -->|"trains model"| node_model_arch
node_agent_train -->|"generates rollouts"| node_rollout
node_agent_train -->|"uses utilities"| node_training_utils
node_distillation -->|"loads datasets"| node_lm_data
node_distillation -->|"trains model"| node_model_arch
node_distillation -->|"uses utilities"| node_training_utils
node_rollout -->|"generates with"| node_model_arch
node_rollout -->|"sends HTTP requests"| node_sglang
node_chat_user -->|"asks questions"| node_chat_api
node_chat_user -->|"sends chat requests"| node_openai_api
node_chat_user -->|"uses chat UI"| node_web_demo
node_chat_api -.->|"runs inference"| node_model_arch
node_openai_api -->|"serves inference"| node_model_arch
node_web_demo -->|"uses utilities"| node_training_utils
node_web_demo -.->|"serves inference"| node_model_arch
node_model_user -->|"evaluates model"| node_general_eval
node_general_eval -->|"loads model"| node_model_arch
node_model_user -->|"checks math ability"| node_math_eval
node_math_eval -->|"generates answers"| node_model_arch
node_math_eval -->|"uses parser"| node_math_format

click node_model_arch "https://github.com/jiwubu/minimind/blob/master/model/model_minimind.py"
click node_lora "https://github.com/jiwubu/minimind/blob/master/model/model_lora.py"
click node_tokenizer_train "https://github.com/jiwubu/minimind/blob/master/trainer/train_tokenizer.py"
click node_lm_data "https://github.com/jiwubu/minimind/blob/master/dataset/lm_dataset.py"
click node_math_format "https://github.com/jiwubu/minimind/blob/master/math/math_format.py"
click node_addsub_data "https://github.com/jiwubu/minimind/blob/master/math/gen_math_data_addsub.py"
click node_muldiv_data "https://github.com/jiwubu/minimind/blob/master/math/gen_math_data_muldiv.py"
click node_pretrain "https://github.com/jiwubu/minimind/blob/master/trainer/train_pretrain.py"
click node_full_sft "https://github.com/jiwubu/minimind/blob/master/trainer/train_full_sft.py"
click node_lora_train "https://github.com/jiwubu/minimind/blob/master/trainer/train_lora.py"
click node_preference "https://github.com/jiwubu/minimind/blob/master/trainer/train_dpo.py"
click node_rl_train "https://github.com/jiwubu/minimind/blob/master/trainer/train_ppo.py"
click node_agent_train "https://github.com/jiwubu/minimind/blob/master/trainer/train_agent.py"
click node_distillation "https://github.com/jiwubu/minimind/blob/master/trainer/train_distillation.py"
click node_rollout "https://github.com/jiwubu/minimind/blob/master/trainer/rollout_engine.py"
click node_training_utils "https://github.com/jiwubu/minimind/blob/master/trainer/trainer_utils.py"
click node_chat_api "https://github.com/jiwubu/minimind/blob/master/scripts/chat_api.py"
click node_openai_api "https://github.com/jiwubu/minimind/blob/master/scripts/serve_openai_api.py"
click node_web_demo "https://github.com/jiwubu/minimind/blob/master/scripts/web_demo.py"
click node_math_eval "https://github.com/jiwubu/minimind/blob/master/math/eval_math.py"
click node_tool_eval "https://github.com/jiwubu/minimind/blob/master/scripts/eval_toolcall.py"
click node_general_eval "https://github.com/jiwubu/minimind/blob/master/eval_llm.py"
click node_convert "https://github.com/jiwubu/minimind/blob/master/scripts/convert_model.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_model_arch,node_lora,node_tokenizer_train,node_chat_user,node_model_user toneBlue
class node_lm_data,node_math_format,node_addsub_data,node_muldiv_data toneAmber
class node_pretrain,node_full_sft,node_lora_train,node_preference,node_rl_train,node_agent_train,node_distillation,node_rollout,node_training_utils toneMint
class node_chat_api,node_openai_api,node_web_demo,node_convert toneRose
class node_math_eval,node_tool_eval,node_general_eval,node_learner,node_sglang toneIndigo
```

## Components

### Model and tokenizer

- [MiniMind architecture](https://github.com/jiwubu/minimind/blob/master/model/model_minimind.py): `model/model_minimind.py`
- [LoRA adaptation](https://github.com/jiwubu/minimind/blob/master/model/model_lora.py): `model/model_lora.py`
- [Tokenizer training](https://github.com/jiwubu/minimind/blob/master/trainer/train_tokenizer.py): `trainer/train_tokenizer.py`

### Data and math

- [Language datasets](https://github.com/jiwubu/minimind/blob/master/dataset/lm_dataset.py): `dataset/lm_dataset.py`
- [Math trace format](https://github.com/jiwubu/minimind/blob/master/math/math_format.py): `math/math_format.py`
- [Add/sub data](https://github.com/jiwubu/minimind/blob/master/math/gen_math_data_addsub.py): `math/gen_math_data_addsub.py`
- [Mul/div data](https://github.com/jiwubu/minimind/blob/master/math/gen_math_data_muldiv.py): `math/gen_math_data_muldiv.py`

### Training workflows

- [Pretraining and SFT](https://github.com/jiwubu/minimind/blob/master/trainer/train_pretrain.py): `trainer/train_pretrain.py`
- [Full SFT](https://github.com/jiwubu/minimind/blob/master/trainer/train_full_sft.py): `trainer/train_full_sft.py`
- [LoRA training](https://github.com/jiwubu/minimind/blob/master/trainer/train_lora.py): `trainer/train_lora.py`
- [DPO training](https://github.com/jiwubu/minimind/blob/master/trainer/train_dpo.py): `trainer/train_dpo.py`
- [PPO and GRPO](https://github.com/jiwubu/minimind/blob/master/trainer/train_ppo.py): `trainer/train_ppo.py`
- [Agentic RL](https://github.com/jiwubu/minimind/blob/master/trainer/train_agent.py): `trainer/train_agent.py`
- [Model distillation](https://github.com/jiwubu/minimind/blob/master/trainer/train_distillation.py): `trainer/train_distillation.py`
- [Rollout engines](https://github.com/jiwubu/minimind/blob/master/trainer/rollout_engine.py): `trainer/rollout_engine.py`
- [Training utilities](https://github.com/jiwubu/minimind/blob/master/trainer/trainer_utils.py): `trainer/trainer_utils.py`

### Inference and tools

- [Chat examples](https://github.com/jiwubu/minimind/blob/master/scripts/chat_api.py): `scripts/chat_api.py`
- [OpenAI-compatible API](https://github.com/jiwubu/minimind/blob/master/scripts/serve_openai_api.py): `scripts/serve_openai_api.py`
- [Chat web UI](https://github.com/jiwubu/minimind/blob/master/scripts/web_demo.py): `scripts/web_demo.py`
- [Merge and export](https://github.com/jiwubu/minimind/blob/master/scripts/convert_model.py): `scripts/convert_model.py`

### Evaluation

- [Math evaluation](https://github.com/jiwubu/minimind/blob/master/math/eval_math.py): `math/eval_math.py`
- [Tool-call evaluation](https://github.com/jiwubu/minimind/blob/master/scripts/eval_toolcall.py): `scripts/eval_toolcall.py`
- [Model evaluation](https://github.com/jiwubu/minimind/blob/master/eval_llm.py): `eval_llm.py`

### Other components

- **Model learner**
- **Chat user**
- **SGLang service**
- **Model user**

## Connections

- Model learner → Tokenizer training: trains tokenizer (evidence: `README.md`)
- Model learner → Add/sub data: generates math data (evidence: `README.md`)
- Model learner → Mul/div data: generates math data (evidence: `README.md`)
- Model learner → Pretraining and SFT: trains model (evidence: `README.md`)
- Model learner → Full SFT: fine-tunes model (evidence: `README.md`)
- Model learner → DPO training: optimizes preferences (evidence: `README.md`)
- Model learner → PPO and GRPO: trains with RL (evidence: `README.md`)
- Model learner → Agentic RL: trains tool agent (evidence: `README.md`)
- Model learner → Model distillation: distills model (evidence: `README.md`)
- Add/sub data → Math trace format: uses format (evidence: `math/gen_math_data_addsub.py`)
- Mul/div data → Math trace format: uses format (evidence: `math/gen_math_data_muldiv.py`)
- Pretraining and SFT → Language datasets: loads datasets (evidence: `trainer/train_pretrain.py`)
- Pretraining and SFT → MiniMind architecture: trains model (evidence: `trainer/train_pretrain.py`)
- Pretraining and SFT → Training utilities: uses utilities (evidence: `trainer/train_pretrain.py`)
- Full SFT → Language datasets: loads datasets (evidence: `trainer/train_full_sft.py`)
- Full SFT → MiniMind architecture: trains model (evidence: `trainer/train_full_sft.py`)
- Full SFT → Training utilities: uses utilities (evidence: `trainer/train_full_sft.py`)
- LoRA training → Language datasets: loads datasets (evidence: `trainer/train_lora.py`)
- LoRA training → MiniMind architecture: trains model (evidence: `trainer/train_lora.py`)
- LoRA training → LoRA adaptation: applies adaptation (evidence: `trainer/train_lora.py`)
- DPO training → Language datasets: loads datasets (evidence: `trainer/train_dpo.py`)
- DPO training → MiniMind architecture: trains model (evidence: `trainer/train_dpo.py`)
- DPO training → Training utilities: uses utilities (evidence: `trainer/train_dpo.py`)
- PPO and GRPO → Language datasets: loads datasets (evidence: `trainer/train_ppo.py`)
- PPO and GRPO → MiniMind architecture: builds policy (evidence: `trainer/train_ppo.py`)
- PPO and GRPO → Rollout engines: generates rollouts (evidence: `trainer/train_ppo.py`)
- PPO and GRPO → Training utilities: uses utilities (evidence: `trainer/train_ppo.py`)
- Agentic RL → Language datasets: loads datasets (evidence: `trainer/train_agent.py`)
- Agentic RL → MiniMind architecture: trains model (evidence: `trainer/train_agent.py`)
- Agentic RL → Rollout engines: generates rollouts (evidence: `trainer/train_agent.py`)
- Agentic RL → Training utilities: uses utilities (evidence: `trainer/train_agent.py`)
- Model distillation → Language datasets: loads datasets (evidence: `trainer/train_distillation.py`)
- Model distillation → MiniMind architecture: trains model (evidence: `trainer/train_distillation.py`)
- Model distillation → Training utilities: uses utilities (evidence: `trainer/train_distillation.py`)
- Rollout engines → MiniMind architecture: generates with (evidence: `trainer/rollout_engine.py`)
- Rollout engines → SGLang service: sends HTTP requests (evidence: `trainer/rollout_engine.py`)
- Chat user → Chat examples: asks questions (evidence: `README.md`)
- Chat user → OpenAI-compatible API: sends chat requests (evidence: `README.md`)
- Chat user → Chat web UI: uses chat UI (evidence: `README.md`)
- Chat examples → MiniMind architecture: runs inference
- OpenAI-compatible API → MiniMind architecture: serves inference (evidence: `scripts/serve_openai_api.py`)
- Chat web UI → Training utilities: uses utilities (evidence: `scripts/web_demo.py`)
- Chat web UI → MiniMind architecture: serves inference
- Model user → Model evaluation: evaluates model (evidence: `README.md`)
- Model evaluation → MiniMind architecture: loads model (evidence: `eval_llm.py`)
- Model user → Math evaluation: checks math ability (evidence: `README.md`)
- Math evaluation → MiniMind architecture: generates answers (evidence: `math/eval_math.py`)
- Math evaluation → Math trace format: uses parser (evidence: `math/eval_math.py`)
