# baidu-baige/loongforge architecture

> Architecture diagram of the GitHub repository baidu-baige/loongforge, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/baidu-baige/loongforge
- Repository: https://github.com/baidu-baige/loongforge

## Overview

LoongForge is a multi-backend framework for training LLMs, VLMs, diffusion, and embodied models. Its principal workflow prepares training data, constructs a model, and executes pre-training or SFT; the embodied stack adds dataset transforms, distributed training support, and policy evaluation. The README documents both Megatron and torch-native backends and broad model coverage. Sampled source confirms the LLM dataset/model paths and selected embodied transform and evaluation behavior; other subsystem links reflect documented capabilities rather than inspected call sites.

Main entry points: [`loongforge/embodied/eval/orchestrator/runners`](https://github.com/baidu-baige/loongforge/tree/master/loongforge/embodied/eval/orchestrator/runners) (Benchmark Runners), [`loongforge/train/pretrain/pretrain_llm.py`](https://github.com/baidu-baige/loongforge/blob/master/loongforge/train/pretrain/pretrain_llm.py) (LLM Pretraining), [`loongforge/train/sft/sft_llm.py`](https://github.com/baidu-baige/loongforge/blob/master/loongforge/train/sft/sft_llm.py) (LLM SFT), [`loongforge/embodied/eval/orchestrator/run.py`](https://github.com/baidu-baige/loongforge/blob/master/loongforge/embodied/eval/orchestrator/run.py) (Evaluation Orchestrator)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_llm["Megatron Training"]
  node_pretrain["LLM Pretraining<br/>[pretrain_llm.py]"]
  node_sft["LLM SFT<br/>[sft_llm.py]"]
  node_llmdata["GPT Datasets<br/>[pretrain_llm.py]"]
  node_sftdata["SFT Datasets<br/>[sft_llm.py]"]
  node_llmmodel["LLM Provider"]
  node_moerouter["MoE Routing<br/>[ernie_router.py]"]
end

subgraph group_embodied["Embodied Stack"]
  node_embodieddata["Robot Datasets<br/>[dataset_builder.py]"]
  node_transforms["Sample Transforms<br/>[pipeline.py]"]
  node_embodiedmodels["Embodied Models<br/>[registry.py]"]
end

subgraph group_evaluation["Policy Evaluation"]
  node_evalrun["Evaluation Orchestrator<br/>[run.py]"]
  node_runner["Benchmark Runners"]
  node_servermanager["Server Manager<br/>[server_manager.py]"]
  node_policyserver["Policy Server"]
  node_policyclient["Policy Client<br/>[client.py]"]
  node_evalresults["Evaluation Results<br/>[results.py]"]
end

subgraph group_runtime["Training Runtime"]
  node_distributed["Distributed Runtime<br/>[parallel.py]"]
  node_checkpoint["Checkpointing<br/>[checkpoint.py]"]
  node_optimizer["Optimizers<br/>[optimizer.py]"]
end

node_engineer(("Training User"))
node_evaluser(("Evaluation User"))
node_benchmark(("Robot Benchmark"))

node_engineer -->|"starts training"| node_pretrain
node_engineer -->|"starts fine-tuning"| node_sft
node_pretrain -->|"builds datasets"| node_llmdata
node_sft -->|"builds datasets"| node_sftdata
node_pretrain -.->|"uses model"| node_llmmodel
node_sft -.->|"uses model"| node_llmmodel
node_llmmodel -.->|"routes experts"| node_moerouter
node_engineer -.->|"configures training"| node_embodieddata
node_embodieddata -.->|"builds transforms"| node_transforms
node_embodiedmodels -.->|"runs across ranks"| node_distributed
node_distributed -.->|"saves state"| node_checkpoint
node_engineer -->|"starts evaluation"| node_evalrun
node_evalrun -.->|"dispatches benchmark"| node_runner
node_evalrun -->|"manages server"| node_servermanager
node_servermanager -->|"starts process"| node_policyserver
node_runner -.->|"requests actions"| node_policyclient
node_policyclient -->|"sends inference"| node_policyserver
node_benchmark -.->|"provides episodes"| node_runner
node_runner -.->|"records metrics"| node_evalresults

click node_pretrain "https://github.com/baidu-baige/loongforge/blob/master/loongforge/train/pretrain/pretrain_llm.py"
click node_sft "https://github.com/baidu-baige/loongforge/blob/master/loongforge/train/sft/sft_llm.py"
click node_llmdata "https://github.com/baidu-baige/loongforge/blob/master/loongforge/train/pretrain/pretrain_llm.py"
click node_sftdata "https://github.com/baidu-baige/loongforge/blob/master/loongforge/train/sft/sft_llm.py"
click node_llmmodel "https://github.com/baidu-baige/loongforge/blob/master/loongforge/models/foundation/llm_model_provider.py"
click node_moerouter "https://github.com/baidu-baige/loongforge/blob/master/loongforge/models/foundation/ernie4_5_vl/ernie_router.py"
click node_embodieddata "https://github.com/baidu-baige/loongforge/blob/master/loongforge/embodied/data/datasets/dataset_builder.py"
click node_transforms "https://github.com/baidu-baige/loongforge/blob/master/loongforge/embodied/data/datasets/transforms/pipeline.py"
click node_embodiedmodels "https://github.com/baidu-baige/loongforge/blob/master/loongforge/embodied/model/registry.py"
click node_distributed "https://github.com/baidu-baige/loongforge/blob/master/loongforge/embodied/distributed/parallel.py"
click node_checkpoint "https://github.com/baidu-baige/loongforge/blob/master/loongforge/embodied/distributed/checkpoint.py"
click node_optimizer "https://github.com/baidu-baige/loongforge/blob/master/loongforge/embodied/optimizer/optimizer.py"
click node_evalrun "https://github.com/baidu-baige/loongforge/blob/master/loongforge/embodied/eval/orchestrator/run.py"
click node_runner "https://github.com/baidu-baige/loongforge/tree/master/loongforge/embodied/eval/orchestrator/runners"
click node_servermanager "https://github.com/baidu-baige/loongforge/blob/master/loongforge/embodied/eval/orchestrator/server_manager.py"
click node_policyserver "https://github.com/baidu-baige/loongforge/blob/master/loongforge/embodied/eval/servers/loongforge_server.py"
click node_policyclient "https://github.com/baidu-baige/loongforge/blob/master/loongforge/embodied/eval/transport/client.py"
click node_evalresults "https://github.com/baidu-baige/loongforge/blob/master/loongforge/embodied/eval/metrics/results.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_pretrain,node_sft,node_llmdata,node_sftdata,node_llmmodel,node_moerouter,node_engineer,node_evaluser toneBlue
class node_embodieddata,node_transforms,node_embodiedmodels toneAmber
class node_evalrun,node_runner,node_servermanager,node_policyserver,node_policyclient,node_evalresults toneMint
class node_distributed,node_checkpoint,node_optimizer toneRose
class node_benchmark toneIndigo
```

## Components

### Megatron Training

- [LLM Pretraining](https://github.com/baidu-baige/loongforge/blob/master/loongforge/train/pretrain/pretrain_llm.py): `loongforge/train/pretrain/pretrain_llm.py`
- [LLM SFT](https://github.com/baidu-baige/loongforge/blob/master/loongforge/train/sft/sft_llm.py): `loongforge/train/sft/sft_llm.py`
- [GPT Datasets](https://github.com/baidu-baige/loongforge/blob/master/loongforge/train/pretrain/pretrain_llm.py): `loongforge/train/pretrain/pretrain_llm.py`
- [SFT Datasets](https://github.com/baidu-baige/loongforge/blob/master/loongforge/train/sft/sft_llm.py): `loongforge/train/sft/sft_llm.py`
- [LLM Provider](https://github.com/baidu-baige/loongforge/blob/master/loongforge/models/foundation/llm_model_provider.py): `loongforge/models/foundation/llm_model_provider.py`
- [MoE Routing](https://github.com/baidu-baige/loongforge/blob/master/loongforge/models/foundation/ernie4_5_vl/ernie_router.py): `loongforge/models/foundation/ernie4_5_vl/ernie_router.py`

### Embodied Stack

- [Robot Datasets](https://github.com/baidu-baige/loongforge/blob/master/loongforge/embodied/data/datasets/dataset_builder.py): `loongforge/embodied/data/datasets/dataset_builder.py`
- [Sample Transforms](https://github.com/baidu-baige/loongforge/blob/master/loongforge/embodied/data/datasets/transforms/pipeline.py): `loongforge/embodied/data/datasets/transforms/pipeline.py`
- [Embodied Models](https://github.com/baidu-baige/loongforge/blob/master/loongforge/embodied/model/registry.py): `loongforge/embodied/model/registry.py`

### Policy Evaluation

- [Evaluation Orchestrator](https://github.com/baidu-baige/loongforge/blob/master/loongforge/embodied/eval/orchestrator/run.py): `loongforge/embodied/eval/orchestrator/run.py`
- [Benchmark Runners](https://github.com/baidu-baige/loongforge/tree/master/loongforge/embodied/eval/orchestrator/runners): `loongforge/embodied/eval/orchestrator/runners`
- [Server Manager](https://github.com/baidu-baige/loongforge/blob/master/loongforge/embodied/eval/orchestrator/server_manager.py): `loongforge/embodied/eval/orchestrator/server_manager.py`
- [Policy Server](https://github.com/baidu-baige/loongforge/blob/master/loongforge/embodied/eval/servers/loongforge_server.py): `loongforge/embodied/eval/servers/loongforge_server.py`
- [Policy Client](https://github.com/baidu-baige/loongforge/blob/master/loongforge/embodied/eval/transport/client.py): `loongforge/embodied/eval/transport/client.py`
- [Evaluation Results](https://github.com/baidu-baige/loongforge/blob/master/loongforge/embodied/eval/metrics/results.py): `loongforge/embodied/eval/metrics/results.py`

### Training Runtime

- [Distributed Runtime](https://github.com/baidu-baige/loongforge/blob/master/loongforge/embodied/distributed/parallel.py): `loongforge/embodied/distributed/parallel.py`
- [Checkpointing](https://github.com/baidu-baige/loongforge/blob/master/loongforge/embodied/distributed/checkpoint.py): `loongforge/embodied/distributed/checkpoint.py`
- [Optimizers](https://github.com/baidu-baige/loongforge/blob/master/loongforge/embodied/optimizer/optimizer.py): `loongforge/embodied/optimizer/optimizer.py`

### Other components

- **Training User**
- **Evaluation User**
- **Robot Benchmark**

## Connections

- Training User → LLM Pretraining: starts training
- Training User → LLM SFT: starts fine-tuning
- LLM Pretraining → GPT Datasets: builds datasets
- LLM SFT → SFT Datasets: builds datasets
- LLM Pretraining → LLM Provider: uses model
- LLM SFT → LLM Provider: uses model
- LLM Provider → MoE Routing: routes experts
- Training User → Robot Datasets: configures training
- Robot Datasets → Sample Transforms: builds transforms
- Embodied Models → Distributed Runtime: runs across ranks
- Distributed Runtime → Checkpointing: saves state
- Training User → Evaluation Orchestrator: starts evaluation
- Evaluation Orchestrator → Benchmark Runners: dispatches benchmark
- Evaluation Orchestrator → Server Manager: manages server
- Server Manager → Policy Server: starts process
- Benchmark Runners → Policy Client: requests actions
- Policy Client → Policy Server: sends inference
- Robot Benchmark → Benchmark Runners: provides episodes
- Benchmark Runners → Evaluation Results: records metrics