# jingyaogong/minimind architecture

> Architecture diagram of the GitHub repository jingyaogong/minimind, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jingyaogong/minimind
- Repository: https://github.com/jingyaogong/minimind

## Overview

MiniMind is a from-scratch, compact language-model project: its main user journey is preparing training data, training or adapting a model, then evaluating or serving generated conversations. The repository also provides distinct preference/RL, distillation, tool-use, tokenizer, and chat-interface capabilities. The graph preserves these documented branches while limiting connections to relationships visible in excerpts, the source index, or README; unsampled implementation details are not inferred.

Main entry points: [`model/model_minimind.py`](https://github.com/jingyaogong/minimind/blob/master/model/model_minimind.py) (Dense and MoE model), [`dataset/lm_dataset.py`](https://github.com/jingyaogong/minimind/blob/master/dataset/lm_dataset.py) (Dataset loaders), [`trainer/rollout_engine.py`](https://github.com/jingyaogong/minimind/blob/master/trainer/rollout_engine.py) (Rollout engines), [`model/model_lora.py`](https://github.com/jingyaogong/minimind/blob/master/model/model_lora.py) (LoRA adapters)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Data and tokenization"]
  node_data_formats["Training corpora<br/>[dataset.md]"]
  node_tokenizer_training["Tokenizer training<br/>[train_tokenizer.py]"]
  node_datasets["Dataset loaders<br/>[lm_dataset.py]"]
end

subgraph group_training["Model training"]
  node_pretrain["Pretraining<br/>[train_pretrain.py]"]
  node_sft["SFT and LoRA<br/>[train_full_sft.py]"]
  node_lora_training["LoRA training<br/>[train_lora.py]"]
  node_preference_training["DPO training<br/>[train_dpo.py]"]
  node_policy_training["PPO and GRPO<br/>[train_ppo.py]"]
  node_grpo_training["GRPO training<br/>[train_grpo.py]"]
  node_agent_training["Agent RL<br/>[train_agent.py]"]
  node_distillation["Distillation"]
  node_rollout["Rollout engines<br/>[rollout_engine.py]"]
  node_training_utils["Training utilities<br/>[trainer_utils.py]"]
end

subgraph group_runtime["Model runtime"]
  node_model["Dense and MoE model<br/>[model_minimind.py]"]
  node_lora["LoRA adapters<br/>[model_lora.py]"]
  node_convert["Weight conversion<br/>[convert_model.py]"]
end

subgraph group_interfaces["Evaluation and chat"]
  node_evaluation["Model evaluation<br/>[eval_llm.py]"]
  node_tool_evaluation["Tool-call evaluation<br/>[eval_toolcall.py]"]
  node_api["OpenAI API server"]
  node_web["Chat Web UI<br/>[web_demo.py]"]
  node_chat_api["Chat API example<br/>[chat_api.py]"]
end

node_user(("Engineer or user"))
node_sglang{{"SGLang service"}}

node_user -->|"trains tokenizer"| node_tokenizer_training
node_user -->|"starts training"| node_pretrain
node_user -->|"enters prompt"| node_evaluation
node_user -.->|"sends requests"| node_api
node_user -.->|"chats"| node_web
node_data_formats -->|"provides examples"| node_datasets
node_pretrain -->|"loads data"| node_datasets
node_sft -->|"loads data"| node_datasets
node_lora_training -->|"loads data"| node_datasets
node_preference_training -->|"loads preferences"| node_datasets
node_policy_training -->|"loads data"| node_datasets
node_grpo_training -->|"loads data"| node_datasets
node_agent_training -->|"loads data"| node_datasets
node_distillation -->|"loads data"| node_datasets
node_pretrain -->|"trains"| node_model
node_sft -->|"trains"| node_model
node_lora_training -->|"applies adapters"| node_lora
node_preference_training -->|"trains policy"| node_model
node_policy_training -->|"trains actor"| node_model
node_grpo_training -->|"trains policy"| node_model
node_agent_training -->|"trains agent"| node_model
node_distillation -->|"trains student"| node_model
node_agent_training -->|"uses engine"| node_rollout
node_policy_training -->|"uses engine"| node_rollout
node_grpo_training -->|"uses engine"| node_rollout
node_rollout -->|"sends HTTP requests"| node_sglang
node_training_utils -->|"initializes model"| node_model
node_evaluation -->|"loads and generates"| node_model
node_evaluation -->|"loads adapters"| node_lora
node_tool_evaluation -->|"evaluates calls"| node_model
node_api -->|"serves inference"| node_model
node_api -->|"supports adapters"| node_lora
node_web -->|"uses utilities"| node_training_utils
node_convert -->|"loads base weights"| node_model
node_convert -->|"merges adapters"| node_lora
node_tokenizer_training -.->|"produces tokenizer"| node_user

click node_data_formats "https://github.com/jingyaogong/minimind/blob/master/dataset/dataset.md"
click node_tokenizer_training "https://github.com/jingyaogong/minimind/blob/master/trainer/train_tokenizer.py"
click node_datasets "https://github.com/jingyaogong/minimind/blob/master/dataset/lm_dataset.py"
click node_pretrain "https://github.com/jingyaogong/minimind/blob/master/trainer/train_pretrain.py"
click node_sft "https://github.com/jingyaogong/minimind/blob/master/trainer/train_full_sft.py"
click node_lora_training "https://github.com/jingyaogong/minimind/blob/master/trainer/train_lora.py"
click node_preference_training "https://github.com/jingyaogong/minimind/blob/master/trainer/train_dpo.py"
click node_policy_training "https://github.com/jingyaogong/minimind/blob/master/trainer/train_ppo.py"
click node_grpo_training "https://github.com/jingyaogong/minimind/blob/master/trainer/train_grpo.py"
click node_agent_training "https://github.com/jingyaogong/minimind/blob/master/trainer/train_agent.py"
click node_distillation "https://github.com/jingyaogong/minimind/blob/master/trainer/train_distillation.py"
click node_rollout "https://github.com/jingyaogong/minimind/blob/master/trainer/rollout_engine.py"
click node_training_utils "https://github.com/jingyaogong/minimind/blob/master/trainer/trainer_utils.py"
click node_model "https://github.com/jingyaogong/minimind/blob/master/model/model_minimind.py"
click node_lora "https://github.com/jingyaogong/minimind/blob/master/model/model_lora.py"
click node_convert "https://github.com/jingyaogong/minimind/blob/master/scripts/convert_model.py"
click node_evaluation "https://github.com/jingyaogong/minimind/blob/master/eval_llm.py"
click node_tool_evaluation "https://github.com/jingyaogong/minimind/blob/master/scripts/eval_toolcall.py"
click node_api "https://github.com/jingyaogong/minimind/blob/master/scripts/serve_openai_api.py"
click node_web "https://github.com/jingyaogong/minimind/blob/master/scripts/web_demo.py"
click node_chat_api "https://github.com/jingyaogong/minimind/blob/master/scripts/chat_api.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_data_formats,node_tokenizer_training,node_datasets,node_user toneBlue
class node_pretrain,node_sft,node_lora_training,node_preference_training,node_policy_training,node_grpo_training,node_agent_training,node_distillation,node_rollout,node_training_utils toneAmber
class node_model,node_lora,node_convert toneMint
class node_evaluation,node_tool_evaluation,node_api,node_web,node_chat_api toneRose
class node_sglang toneIndigo
```

## Components

### Data and tokenization

- [Training corpora](https://github.com/jingyaogong/minimind/blob/master/dataset/dataset.md): `dataset/dataset.md`
- [Tokenizer training](https://github.com/jingyaogong/minimind/blob/master/trainer/train_tokenizer.py): `trainer/train_tokenizer.py`
- [Dataset loaders](https://github.com/jingyaogong/minimind/blob/master/dataset/lm_dataset.py): `dataset/lm_dataset.py`

### Model training

- [Pretraining](https://github.com/jingyaogong/minimind/blob/master/trainer/train_pretrain.py): `trainer/train_pretrain.py`
- [SFT and LoRA](https://github.com/jingyaogong/minimind/blob/master/trainer/train_full_sft.py): `trainer/train_full_sft.py`
- [LoRA training](https://github.com/jingyaogong/minimind/blob/master/trainer/train_lora.py): `trainer/train_lora.py`
- [DPO training](https://github.com/jingyaogong/minimind/blob/master/trainer/train_dpo.py): `trainer/train_dpo.py`
- [PPO and GRPO](https://github.com/jingyaogong/minimind/blob/master/trainer/train_ppo.py): `trainer/train_ppo.py`
- [GRPO training](https://github.com/jingyaogong/minimind/blob/master/trainer/train_grpo.py): `trainer/train_grpo.py`
- [Agent RL](https://github.com/jingyaogong/minimind/blob/master/trainer/train_agent.py): `trainer/train_agent.py`
- [Distillation](https://github.com/jingyaogong/minimind/blob/master/trainer/train_distillation.py): `trainer/train_distillation.py`
- [Rollout engines](https://github.com/jingyaogong/minimind/blob/master/trainer/rollout_engine.py): `trainer/rollout_engine.py`
- [Training utilities](https://github.com/jingyaogong/minimind/blob/master/trainer/trainer_utils.py): `trainer/trainer_utils.py`

### Model runtime

- [Dense and MoE model](https://github.com/jingyaogong/minimind/blob/master/model/model_minimind.py): `model/model_minimind.py`
- [LoRA adapters](https://github.com/jingyaogong/minimind/blob/master/model/model_lora.py): `model/model_lora.py`
- [Weight conversion](https://github.com/jingyaogong/minimind/blob/master/scripts/convert_model.py): `scripts/convert_model.py`

### Evaluation and chat

- [Model evaluation](https://github.com/jingyaogong/minimind/blob/master/eval_llm.py): `eval_llm.py`
- [Tool-call evaluation](https://github.com/jingyaogong/minimind/blob/master/scripts/eval_toolcall.py): `scripts/eval_toolcall.py`
- [OpenAI API server](https://github.com/jingyaogong/minimind/blob/master/scripts/serve_openai_api.py): `scripts/serve_openai_api.py`
- [Chat Web UI](https://github.com/jingyaogong/minimind/blob/master/scripts/web_demo.py): `scripts/web_demo.py`
- [Chat API example](https://github.com/jingyaogong/minimind/blob/master/scripts/chat_api.py): `scripts/chat_api.py`

### Other components

- **Engineer or user**
- **SGLang service**

## Connections

- Engineer or user → Tokenizer training: trains tokenizer (evidence: `README.md`)
- Engineer or user → Pretraining: starts training (evidence: `README.md`)
- Engineer or user → Model evaluation: enters prompt (evidence: `eval_llm.py`)
- Engineer or user → OpenAI API server: sends requests (evidence: `README.md`)
- Engineer or user → Chat Web UI: chats (evidence: `README.md`)
- Training corpora → Dataset loaders: provides examples (evidence: `dataset/lm_dataset.py`)
- Pretraining → Dataset loaders: loads data (evidence: `trainer/train_pretrain.py`)
- SFT and LoRA → Dataset loaders: loads data (evidence: `trainer/train_full_sft.py`)
- LoRA training → Dataset loaders: loads data (evidence: `trainer/train_lora.py`)
- DPO training → Dataset loaders: loads preferences (evidence: `trainer/train_dpo.py`)
- PPO and GRPO → Dataset loaders: loads data (evidence: `trainer/train_ppo.py`)
- GRPO training → Dataset loaders: loads data (evidence: `trainer/train_grpo.py`)
- Agent RL → Dataset loaders: loads data (evidence: `trainer/train_agent.py`)
- Distillation → Dataset loaders: loads data (evidence: `trainer/train_distillation.py`)
- Pretraining → Dense and MoE model: trains (evidence: `trainer/train_pretrain.py`)
- SFT and LoRA → Dense and MoE model: trains (evidence: `trainer/train_full_sft.py`)
- LoRA training → LoRA adapters: applies adapters (evidence: `trainer/train_lora.py`)
- DPO training → Dense and MoE model: trains policy (evidence: `trainer/train_dpo.py`)
- PPO and GRPO → Dense and MoE model: trains actor (evidence: `trainer/train_ppo.py`)
- GRPO training → Dense and MoE model: trains policy (evidence: `trainer/train_grpo.py`)
- Agent RL → Dense and MoE model: trains agent (evidence: `trainer/train_agent.py`)
- Distillation → Dense and MoE model: trains student (evidence: `trainer/train_distillation.py`)
- Agent RL → Rollout engines: uses engine (evidence: `trainer/train_agent.py`)
- PPO and GRPO → Rollout engines: uses engine (evidence: `trainer/train_ppo.py`)
- GRPO training → Rollout engines: uses engine (evidence: `trainer/train_grpo.py`)
- Rollout engines → SGLang service: sends HTTP requests (evidence: `trainer/rollout_engine.py`)
- Training utilities → Dense and MoE model: initializes model (evidence: `trainer/trainer_utils.py`)
- Model evaluation → Dense and MoE model: loads and generates (evidence: `eval_llm.py`)
- Model evaluation → LoRA adapters: loads adapters (evidence: `eval_llm.py`)
- Tool-call evaluation → Dense and MoE model: evaluates calls (evidence: `scripts/eval_toolcall.py`)
- OpenAI API server → Dense and MoE model: serves inference (evidence: `scripts/serve_openai_api.py`)
- OpenAI API server → LoRA adapters: supports adapters (evidence: `scripts/serve_openai_api.py`)
- Chat Web UI → Training utilities: uses utilities (evidence: `scripts/web_demo.py`)
- Weight conversion → Dense and MoE model: loads base weights (evidence: `scripts/convert_model.py`)
- Weight conversion → LoRA adapters: merges adapters (evidence: `scripts/convert_model.py`)
- Tokenizer training → Engineer or user: produces tokenizer (evidence: `README.md`)
