# drxikunzhao/minimind architecture

> Architecture diagram of the GitHub repository drxikunzhao/minimind, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/drxikunzhao/minimind
- Repository: https://github.com/drxikunzhao/minimind

## Overview

MiniMind is a from-scratch, lightweight language-model implementation and learning project: users prepare or select data, train a tokenizer and model through supervised and preference/agent training stages, then evaluate or serve generated conversations. Its README also documents distillation, tool use, adaptive thinking, and multiple rollout backends. The graph includes documented capabilities even where source was not sampled; those nodes use supported paths, and uncertain runtime links are omitted. The sampled code confirms the model, dataset formats, LoRA, agent tool execution, and rollout-engine behavior.

Main entry points: [`model/model_minimind.py`](https://github.com/drxikunzhao/minimind/blob/master/model/model_minimind.py) (MiniMind Dense and MoE), [`dataset/lm_dataset.py`](https://github.com/drxikunzhao/minimind/blob/master/dataset/lm_dataset.py) (Dataset loaders), [`model`](https://github.com/drxikunzhao/minimind/tree/master/model) (Trained weights), [`trainer/rollout_engine.py`](https://github.com/drxikunzhao/minimind/blob/master/trainer/rollout_engine.py) (Rollout engines)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Data and tokens"]
  node_corpora["Training corpora<br/>[dataset.md]"]
  node_dataset["Dataset loaders<br/>[lm_dataset.py]"]
  node_tokenizer["Tokenizer assets<br/>[tokenizer.json]"]
  node_tokenizer_train["Tokenizer training<br/>[train_tokenizer.py]"]
end

subgraph group_model["Model core"]
  node_architecture["MiniMind Dense and MoE<br/>[model_minimind.py]"]
  node_lora["LoRA weights<br/>[model_lora.py]"]
  node_model_files["Trained weights"]
end

subgraph group_training["Training workflows"]
  node_pretrain["Pretraining<br/>[train_pretrain.py]"]
  node_sft["Supervised tuning<br/>[train_full_sft.py]"]
  node_lora_train["LoRA tuning<br/>[train_lora.py]"]
  node_dpo["DPO preference tuning<br/>[train_dpo.py]"]
  node_ppo["PPO tuning<br/>[train_ppo.py]"]
  node_grpo["GRPO and CISPO<br/>[train_grpo.py]"]
  node_agent["Agentic RL<br/>[train_agent.py]"]
  node_distill["Model distillation"]
  node_rollout["Rollout engines<br/>[rollout_engine.py]"]
end

subgraph group_inference["Inference and evaluation"]
  node_cli["Interactive inference<br/>[eval_llm.py]"]
  node_openai["OpenAI-compatible API"]
  node_web["Chat Web UI<br/>[web_demo.py]"]
  node_chat_api["Chat API example<br/>[chat_api.py]"]
  node_tool_eval["Tool-call evaluation<br/>[eval_toolcall.py]"]
  node_evaluation["Benchmark evaluation<br/>[eval_llm.py]"]
end

node_user(("Engineer"))
node_user_chat(("Chat user"))
node_sglang["SGLang server"]

node_user -->|"selects data"| node_corpora
node_corpora -->|"loads records"| node_dataset
node_dataset -->|"feeds batches"| node_pretrain
node_dataset -->|"feeds batches"| node_sft
node_dataset -->|"feeds batches"| node_lora_train
node_dataset -->|"feeds preferences"| node_dpo
node_dataset -->|"feeds prompts"| node_ppo
node_dataset -->|"feeds prompts"| node_grpo
node_dataset -->|"feeds conversations"| node_agent
node_tokenizer_train -->|"reads text"| node_corpora
node_tokenizer_train -->|"writes assets"| node_tokenizer
node_tokenizer -->|"encodes data"| node_dataset
node_architecture -->|"trains model"| node_pretrain
node_architecture -->|"trains model"| node_sft
node_architecture -->|"trains model"| node_lora_train
node_architecture -->|"trains model"| node_dpo
node_architecture -->|"trains actor"| node_ppo
node_architecture -->|"trains policy"| node_grpo
node_architecture -->|"trains policy"| node_agent
node_architecture -->|"student model"| node_distill
node_lora -->|"provides adapters"| node_lora_train
node_distill -->|"saves weights"| node_model_files
node_pretrain -->|"saves weights"| node_model_files
node_sft -->|"saves weights"| node_model_files
node_lora_train -->|"saves adapters"| node_model_files
node_dpo -->|"saves weights"| node_model_files
node_ppo -->|"requests generations"| node_rollout
node_grpo -->|"requests generations"| node_rollout
node_agent -->|"requests generations"| node_rollout
node_rollout -->|"generates tokens"| node_architecture
node_rollout -.->|"sends HTTP requests"| node_sglang
node_user_chat -->|"enters prompts"| node_cli
node_model_files -->|"loads weights"| node_cli
node_tokenizer -->|"encodes prompts"| node_cli
node_cli -->|"generates response"| node_architecture
node_user_chat -->|"sends chat request"| node_openai
node_user_chat -->|"enters conversation"| node_web
node_user_chat -->|"sends prompt"| node_chat_api
node_architecture -->|"serves generation"| node_openai
node_architecture -->|"serves generation"| node_web
node_architecture -->|"serves generation"| node_chat_api
node_tool_eval -->|"evaluates tool calls"| node_architecture
node_evaluation -->|"runs inference"| node_architecture

click node_corpora "https://github.com/drxikunzhao/minimind/blob/master/dataset/dataset.md"
click node_dataset "https://github.com/drxikunzhao/minimind/blob/master/dataset/lm_dataset.py"
click node_tokenizer "https://github.com/drxikunzhao/minimind/blob/master/model/tokenizer.json"
click node_tokenizer_train "https://github.com/drxikunzhao/minimind/blob/master/trainer/train_tokenizer.py"
click node_architecture "https://github.com/drxikunzhao/minimind/blob/master/model/model_minimind.py"
click node_lora "https://github.com/drxikunzhao/minimind/blob/master/model/model_lora.py"
click node_pretrain "https://github.com/drxikunzhao/minimind/blob/master/trainer/train_pretrain.py"
click node_sft "https://github.com/drxikunzhao/minimind/blob/master/trainer/train_full_sft.py"
click node_lora_train "https://github.com/drxikunzhao/minimind/blob/master/trainer/train_lora.py"
click node_dpo "https://github.com/drxikunzhao/minimind/blob/master/trainer/train_dpo.py"
click node_ppo "https://github.com/drxikunzhao/minimind/blob/master/trainer/train_ppo.py"
click node_grpo "https://github.com/drxikunzhao/minimind/blob/master/trainer/train_grpo.py"
click node_agent "https://github.com/drxikunzhao/minimind/blob/master/trainer/train_agent.py"
click node_distill "https://github.com/drxikunzhao/minimind/blob/master/trainer/train_distillation.py"
click node_rollout "https://github.com/drxikunzhao/minimind/blob/master/trainer/rollout_engine.py"
click node_model_files "https://github.com/drxikunzhao/minimind/tree/master/model"
click node_cli "https://github.com/drxikunzhao/minimind/blob/master/eval_llm.py"
click node_openai "https://github.com/drxikunzhao/minimind/blob/master/scripts/serve_openai_api.py"
click node_web "https://github.com/drxikunzhao/minimind/blob/master/scripts/web_demo.py"
click node_chat_api "https://github.com/drxikunzhao/minimind/blob/master/scripts/chat_api.py"
click node_tool_eval "https://github.com/drxikunzhao/minimind/blob/master/scripts/eval_toolcall.py"
click node_evaluation "https://github.com/drxikunzhao/minimind/blob/master/eval_llm.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_corpora,node_dataset,node_tokenizer,node_tokenizer_train,node_user_chat toneBlue
class node_architecture,node_lora,node_model_files toneAmber
class node_pretrain,node_sft,node_lora_train,node_dpo,node_ppo,node_grpo,node_agent,node_distill,node_rollout,node_sglang toneMint
class node_cli,node_openai,node_web,node_chat_api,node_tool_eval,node_evaluation toneRose
class node_user toneIndigo
```

## Components

### Data and tokens

- [Training corpora](https://github.com/drxikunzhao/minimind/blob/master/dataset/dataset.md): `dataset/dataset.md`
- [Dataset loaders](https://github.com/drxikunzhao/minimind/blob/master/dataset/lm_dataset.py): `dataset/lm_dataset.py`
- [Tokenizer assets](https://github.com/drxikunzhao/minimind/blob/master/model/tokenizer.json): `model/tokenizer.json`
- [Tokenizer training](https://github.com/drxikunzhao/minimind/blob/master/trainer/train_tokenizer.py): `trainer/train_tokenizer.py`

### Model core

- [MiniMind Dense and MoE](https://github.com/drxikunzhao/minimind/blob/master/model/model_minimind.py): `model/model_minimind.py`
- [LoRA weights](https://github.com/drxikunzhao/minimind/blob/master/model/model_lora.py): `model/model_lora.py`
- [Trained weights](https://github.com/drxikunzhao/minimind/tree/master/model): `model`

### Training workflows

- [Pretraining](https://github.com/drxikunzhao/minimind/blob/master/trainer/train_pretrain.py): `trainer/train_pretrain.py`
- [Supervised tuning](https://github.com/drxikunzhao/minimind/blob/master/trainer/train_full_sft.py): `trainer/train_full_sft.py`
- [LoRA tuning](https://github.com/drxikunzhao/minimind/blob/master/trainer/train_lora.py): `trainer/train_lora.py`
- [DPO preference tuning](https://github.com/drxikunzhao/minimind/blob/master/trainer/train_dpo.py): `trainer/train_dpo.py`
- [PPO tuning](https://github.com/drxikunzhao/minimind/blob/master/trainer/train_ppo.py): `trainer/train_ppo.py`
- [GRPO and CISPO](https://github.com/drxikunzhao/minimind/blob/master/trainer/train_grpo.py): `trainer/train_grpo.py`
- [Agentic RL](https://github.com/drxikunzhao/minimind/blob/master/trainer/train_agent.py): `trainer/train_agent.py`
- [Model distillation](https://github.com/drxikunzhao/minimind/blob/master/trainer/train_distillation.py): `trainer/train_distillation.py`
- [Rollout engines](https://github.com/drxikunzhao/minimind/blob/master/trainer/rollout_engine.py): `trainer/rollout_engine.py`

### Inference and evaluation

- [Interactive inference](https://github.com/drxikunzhao/minimind/blob/master/eval_llm.py): `eval_llm.py`
- [OpenAI-compatible API](https://github.com/drxikunzhao/minimind/blob/master/scripts/serve_openai_api.py): `scripts/serve_openai_api.py`
- [Chat Web UI](https://github.com/drxikunzhao/minimind/blob/master/scripts/web_demo.py): `scripts/web_demo.py`
- [Chat API example](https://github.com/drxikunzhao/minimind/blob/master/scripts/chat_api.py): `scripts/chat_api.py`
- [Tool-call evaluation](https://github.com/drxikunzhao/minimind/blob/master/scripts/eval_toolcall.py): `scripts/eval_toolcall.py`
- [Benchmark evaluation](https://github.com/drxikunzhao/minimind/blob/master/eval_llm.py): `eval_llm.py`

### Other components

- **Engineer**
- **Chat user**
- **SGLang server**

## Connections

- Engineer → Training corpora: selects data
- Training corpora → Dataset loaders: loads records
- Dataset loaders → Pretraining: feeds batches
- Dataset loaders → Supervised tuning: feeds batches
- Dataset loaders → LoRA tuning: feeds batches
- Dataset loaders → DPO preference tuning: feeds preferences
- Dataset loaders → PPO tuning: feeds prompts
- Dataset loaders → GRPO and CISPO: feeds prompts
- Dataset loaders → Agentic RL: feeds conversations
- Tokenizer training → Training corpora: reads text
- Tokenizer training → Tokenizer assets: writes assets
- Tokenizer assets → Dataset loaders: encodes data
- MiniMind Dense and MoE → Pretraining: trains model
- MiniMind Dense and MoE → Supervised tuning: trains model
- MiniMind Dense and MoE → LoRA tuning: trains model
- MiniMind Dense and MoE → DPO preference tuning: trains model
- MiniMind Dense and MoE → PPO tuning: trains actor
- MiniMind Dense and MoE → GRPO and CISPO: trains policy
- MiniMind Dense and MoE → Agentic RL: trains policy
- MiniMind Dense and MoE → Model distillation: student model
- LoRA weights → LoRA tuning: provides adapters
- Model distillation → Trained weights: saves weights
- Pretraining → Trained weights: saves weights
- Supervised tuning → Trained weights: saves weights
- LoRA tuning → Trained weights: saves adapters
- DPO preference tuning → Trained weights: saves weights
- PPO tuning → Rollout engines: requests generations
- GRPO and CISPO → Rollout engines: requests generations
- Agentic RL → Rollout engines: requests generations
- Rollout engines → MiniMind Dense and MoE: generates tokens
- Rollout engines → SGLang server: sends HTTP requests
- Chat user → Interactive inference: enters prompts
- Trained weights → Interactive inference: loads weights
- Tokenizer assets → Interactive inference: encodes prompts
- Interactive inference → MiniMind Dense and MoE: generates response
- Chat user → OpenAI-compatible API: sends chat request
- Chat user → Chat Web UI: enters conversation
- Chat user → Chat API example: sends prompt
- MiniMind Dense and MoE → OpenAI-compatible API: serves generation
- MiniMind Dense and MoE → Chat Web UI: serves generation
- MiniMind Dense and MoE → Chat API example: serves generation
- Tool-call evaluation → MiniMind Dense and MoE: evaluates tool calls
- Benchmark evaluation → MiniMind Dense and MoE: runs inference