# paddlepaddle/paddlenlp architecture

> Architecture diagram of the GitHub repository paddlepaddle/paddlenlp, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/paddlepaddle/paddlenlp
- Repository: https://github.com/paddlepaddle/paddlenlp

## Overview

PaddleNLP is a Paddle-based NLP and large-language-model development suite spanning data preparation, model training and alignment, compression, inference, and application serving. A representative workflow is to prepare data, train or fine-tune a supported model, optionally compress it, then run inference or deploy a service. The README documents distributed training, hardware diversity, PEFT, and high-performance inference; sampled sources confirm selected pipeline, engine, and serving relationships. Other subsystem links are documented capability-level connections because their implementation was not sampled.

Main entry points: [`llm/run_pretrain.py`](https://github.com/paddlepaddle/paddlenlp/blob/develop/llm/run_pretrain.py) (LLM training entry), [`llm/server/server/server/engine/engine.py`](https://github.com/paddlepaddle/paddlenlp/blob/develop/llm/server/server/server/engine/engine.py) (LLM runtime), [`paddlenlp/transformers`](https://github.com/paddlepaddle/paddlenlp/tree/develop/paddlenlp/transformers) (Model families), [`paddlenlp/trainer/trainer.py`](https://github.com/paddlepaddle/paddlenlp/blob/develop/paddlenlp/trainer/trainer.py) (Trainer)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Data and models"]
  node_datasets["Dataset loaders<br/>[dataset.py]"]
  node_data_utils["Data utilities"]
  node_model_families["Model families"]
end

subgraph group_training["Training and alignment"]
  node_training_entry["LLM training entry<br/>[run_pretrain.py]"]
  node_trainer["Trainer<br/>[trainer.py]"]
  node_parallel_engine["Parallel engine<br/>[auto_engine.py]"]
  node_eager_engine["Eager engine<br/>[eager_engine.py]"]
  node_checkpoint["Unified checkpoint<br/>[async_handler.py]"]
  node_peft["PEFT methods"]
  node_alignment["Alignment workflows"]
end

subgraph group_optimization["Compression and kernels"]
  node_quantization["Quantization"]
  node_custom_ops["Optimized operators"]
end

subgraph group_inference["Inference and serving"]
  node_generation["Generation APIs<br/>[utils.py]"]
  node_llm_runtime["LLM runtime<br/>[engine.py]"]
  node_resource_manager["Resource manager"]
  node_task_queue["Task queue"]
  node_token_processor["Token processor<br/>[token_processor.py]"]
  node_http_app["HTTP server<br/>[app.py]"]
end

subgraph group_applications["Applications and tooling"]
  node_pipelines["NLP pipelines"]
  node_simple_server["SimpleServer<br/>[server.py]"]
  node_cli["PaddleNLP CLI<br/>[main.py]"]
end

node_developer(("Developer"))

node_developer -.->|"starts training"| node_training_entry
node_datasets -.->|"provides data"| node_data_utils
node_data_utils -.->|"feeds training"| node_training_entry
node_model_families -.->|"trains models"| node_training_entry
node_training_entry -.->|"runs training"| node_trainer
node_training_entry -.->|"supports parallelism"| node_parallel_engine
node_parallel_engine -.->|"offers engine"| node_eager_engine
node_trainer -.->|"saves checkpoints"| node_checkpoint
node_trainer -.->|"supports fine-tuning"| node_peft
node_training_entry -.->|"supports alignment"| node_alignment
node_model_families -.->|"compresses models"| node_quantization
node_quantization -.->|"enables inference"| node_llm_runtime
node_custom_ops -.->|"accelerates inference"| node_llm_runtime
node_model_families -.->|"generates outputs"| node_generation
node_http_app -->|"routes requests"| node_llm_runtime
node_llm_runtime -->|"uses resources"| node_resource_manager
node_llm_runtime -->|"uses queue"| node_task_queue
node_llm_runtime -->|"uses processor"| node_token_processor
node_developer -.->|"builds applications"| node_pipelines
node_pipelines -.->|"retrieves answers"| node_model_families
node_developer -.->|"deploys services"| node_simple_server
node_developer -->|"uses commands"| node_cli

click node_datasets "https://github.com/paddlepaddle/paddlenlp/blob/develop/paddlenlp/datasets/dataset.py"
click node_data_utils "https://github.com/paddlepaddle/paddlenlp/tree/develop/paddlenlp/data"
click node_model_families "https://github.com/paddlepaddle/paddlenlp/tree/develop/paddlenlp/transformers"
click node_training_entry "https://github.com/paddlepaddle/paddlenlp/blob/develop/llm/run_pretrain.py"
click node_trainer "https://github.com/paddlepaddle/paddlenlp/blob/develop/paddlenlp/trainer/trainer.py"
click node_parallel_engine "https://github.com/paddlepaddle/paddlenlp/blob/develop/slm/model_zoo/gpt-3/ppfleetx/core/engine/auto_engine.py"
click node_eager_engine "https://github.com/paddlepaddle/paddlenlp/blob/develop/slm/model_zoo/gpt-3/ppfleetx/core/engine/eager_engine.py"
click node_checkpoint "https://github.com/paddlepaddle/paddlenlp/blob/develop/paddlenlp/trainer/unified_checkpoint/async_handler.py"
click node_peft "https://github.com/paddlepaddle/paddlenlp/tree/develop/paddlenlp/peft"
click node_alignment "https://github.com/paddlepaddle/paddlenlp/tree/develop/llm/alignment"
click node_quantization "https://github.com/paddlepaddle/paddlenlp/blob/develop/llm/run_quantization.py"
click node_custom_ops "https://github.com/paddlepaddle/paddlenlp/tree/develop/ops/src/paddlenlp_kernel"
click node_generation "https://github.com/paddlepaddle/paddlenlp/blob/develop/paddlenlp/generation/utils.py"
click node_llm_runtime "https://github.com/paddlepaddle/paddlenlp/blob/develop/llm/server/server/server/engine/engine.py"
click node_resource_manager "https://github.com/paddlepaddle/paddlenlp/blob/develop/llm/server/server/server/engine/resource_manager.py"
click node_task_queue "https://github.com/paddlepaddle/paddlenlp/blob/develop/llm/server/server/server/engine/task_queue_manager.py"
click node_token_processor "https://github.com/paddlepaddle/paddlenlp/blob/develop/llm/server/server/server/engine/token_processor.py"
click node_http_app "https://github.com/paddlepaddle/paddlenlp/blob/develop/llm/server/server/server/http_server/app.py"
click node_pipelines "https://github.com/paddlepaddle/paddlenlp/blob/develop/slm/pipelines/pipelines/pipelines/standard_pipelines.py"
click node_simple_server "https://github.com/paddlepaddle/paddlenlp/blob/develop/paddlenlp/server/server.py"
click node_cli "https://github.com/paddlepaddle/paddlenlp/blob/develop/paddlenlp/cli/main.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_datasets,node_data_utils,node_model_families toneBlue
class node_training_entry,node_trainer,node_parallel_engine,node_eager_engine,node_checkpoint,node_peft,node_alignment toneAmber
class node_quantization,node_custom_ops toneMint
class node_generation,node_llm_runtime,node_resource_manager,node_task_queue,node_token_processor,node_http_app toneRose
class node_pipelines,node_simple_server,node_cli,node_developer toneIndigo
```

## Components

### Data and models

- [Dataset loaders](https://github.com/paddlepaddle/paddlenlp/blob/develop/paddlenlp/datasets/dataset.py): `paddlenlp/datasets/dataset.py`
- [Data utilities](https://github.com/paddlepaddle/paddlenlp/tree/develop/paddlenlp/data): `paddlenlp/data`
- [Model families](https://github.com/paddlepaddle/paddlenlp/tree/develop/paddlenlp/transformers): `paddlenlp/transformers`

### Training and alignment

- [LLM training entry](https://github.com/paddlepaddle/paddlenlp/blob/develop/llm/run_pretrain.py): `llm/run_pretrain.py`
- [Trainer](https://github.com/paddlepaddle/paddlenlp/blob/develop/paddlenlp/trainer/trainer.py): `paddlenlp/trainer/trainer.py`
- [Parallel engine](https://github.com/paddlepaddle/paddlenlp/blob/develop/slm/model_zoo/gpt-3/ppfleetx/core/engine/auto_engine.py): `slm/model_zoo/gpt-3/ppfleetx/core/engine/auto_engine.py`
- [Eager engine](https://github.com/paddlepaddle/paddlenlp/blob/develop/slm/model_zoo/gpt-3/ppfleetx/core/engine/eager_engine.py): `slm/model_zoo/gpt-3/ppfleetx/core/engine/eager_engine.py`
- [Unified checkpoint](https://github.com/paddlepaddle/paddlenlp/blob/develop/paddlenlp/trainer/unified_checkpoint/async_handler.py): `paddlenlp/trainer/unified_checkpoint/async_handler.py`
- [PEFT methods](https://github.com/paddlepaddle/paddlenlp/tree/develop/paddlenlp/peft): `paddlenlp/peft`
- [Alignment workflows](https://github.com/paddlepaddle/paddlenlp/tree/develop/llm/alignment): `llm/alignment`

### Compression and kernels

- [Quantization](https://github.com/paddlepaddle/paddlenlp/blob/develop/llm/run_quantization.py): `llm/run_quantization.py`
- [Optimized operators](https://github.com/paddlepaddle/paddlenlp/tree/develop/ops/src/paddlenlp_kernel): `ops/src/paddlenlp_kernel`

### Inference and serving

- [Generation APIs](https://github.com/paddlepaddle/paddlenlp/blob/develop/paddlenlp/generation/utils.py): `paddlenlp/generation/utils.py`
- [LLM runtime](https://github.com/paddlepaddle/paddlenlp/blob/develop/llm/server/server/server/engine/engine.py): `llm/server/server/server/engine/engine.py`
- [Resource manager](https://github.com/paddlepaddle/paddlenlp/blob/develop/llm/server/server/server/engine/resource_manager.py): `llm/server/server/server/engine/resource_manager.py`
- [Task queue](https://github.com/paddlepaddle/paddlenlp/blob/develop/llm/server/server/server/engine/task_queue_manager.py): `llm/server/server/server/engine/task_queue_manager.py`
- [Token processor](https://github.com/paddlepaddle/paddlenlp/blob/develop/llm/server/server/server/engine/token_processor.py): `llm/server/server/server/engine/token_processor.py`
- [HTTP server](https://github.com/paddlepaddle/paddlenlp/blob/develop/llm/server/server/server/http_server/app.py): `llm/server/server/server/http_server/app.py`

### Applications and tooling

- [NLP pipelines](https://github.com/paddlepaddle/paddlenlp/blob/develop/slm/pipelines/pipelines/pipelines/standard_pipelines.py): `slm/pipelines/pipelines/pipelines/standard_pipelines.py`
- [SimpleServer](https://github.com/paddlepaddle/paddlenlp/blob/develop/paddlenlp/server/server.py): `paddlenlp/server/server.py`
- [PaddleNLP CLI](https://github.com/paddlepaddle/paddlenlp/blob/develop/paddlenlp/cli/main.py): `paddlenlp/cli/main.py`

### Other components

- **Developer**

## Connections

- Developer → LLM training entry: starts training (evidence: `README.md`)
- Dataset loaders → Data utilities: provides data (evidence: `README.md`)
- Data utilities → LLM training entry: feeds training (evidence: `README.md`)
- Model families → LLM training entry: trains models (evidence: `README.md`)
- LLM training entry → Trainer: runs training (evidence: `README.md`)
- LLM training entry → Parallel engine: supports parallelism (evidence: `README.md`)
- Parallel engine → Eager engine: offers engine (evidence: `README.md`)
- Trainer → Unified checkpoint: saves checkpoints (evidence: `README.md`)
- Trainer → PEFT methods: supports fine-tuning (evidence: `README.md`)
- LLM training entry → Alignment workflows: supports alignment (evidence: `README.md`)
- Model families → Quantization: compresses models (evidence: `README.md`)
- Quantization → LLM runtime: enables inference (evidence: `README.md`)
- Optimized operators → LLM runtime: accelerates inference (evidence: `README.md`)
- Model families → Generation APIs: generates outputs (evidence: `README.md`)
- HTTP server → LLM runtime: routes requests (evidence: `llm/server/server/server/http_server/app.py`)
- LLM runtime → Resource manager: uses resources (evidence: `llm/server/server/server/engine/engine.py`)
- LLM runtime → Task queue: uses queue (evidence: `llm/server/server/server/engine/engine.py`)
- LLM runtime → Token processor: uses processor (evidence: `llm/server/server/server/engine/engine.py`)
- Developer → NLP pipelines: builds applications (evidence: `README.md`)
- NLP pipelines → Model families: retrieves answers (evidence: `README.md`)
- Developer → SimpleServer: deploys services (evidence: `README.md`)
- Developer → PaddleNLP CLI: uses commands (evidence: `paddlenlp/cli/main.py`)