How paddlepaddle/ernie works
ERNIEKit is PaddlePaddle’s toolkit for configuring, fine-tuning, evaluating, serving, and exporting ERNIE text and vision-language models. A typical user starts with the CLI and training arguments, prepares text or multimodal examples, and trains a model using ERNIE’s model and distributed-computing layers. The toolkit also provides evaluation, export, chat/server, and WebUI routes. The graph combines README capability statements with indexed source references; several links to unsampled implementations are documented or left dashed rather than treated as inspected call paths.
Main entry points: modeling.py ERNIE models · tuner.py Training workflows · infer_vl.py VL inference · cli.py CLI
Components 21
Toolkit entrypoints
Data preparation
- Data processing
data_processor/steps/input_ids_messaging/example_to_feature.py - Image processing
data_processor/image_preprocessor/image_preprocessor_adaptive.py - Training datasets
ernie/dataset/finetuning.py
Model runtime
- Text and VL tokenizers
ernie/tokenizer_vl.py - Model configuration
ernie/configuration.py - ERNIE models
ernie/modeling.py - MoE models
ernie/modeling_moe.py - Vision encoder
ernie/siglip/modeling.py - Distributed layers
ernie/distributed/common_dist_utils.py
Training and evaluation
- Training workflows
erniekit/train/tuner.py - SFT trainer
erniekit/train/sft/trainer.py - Evaluation
erniekit/eval/eval.py
Serving and delivery
- VL inference
tools/inference/infer_vl.py - Chat interface
erniekit/chat/chat.py - Model server
erniekit/chat/server.py - Model export
erniekit/export/export.py
Other components
- Toolkit user
- FastDeploy runtime
Connections 26
- Toolkit user → CLI: runs commands
README.md - CLI → Arguments: parses options
erniekit/cli.py - CLI → Training workflows: dispatches training
erniekit/cli.py - Toolkit user → WebUI: uses interface
README.md - Arguments → Training workflows: configures training
- Training workflows → SFT trainer: runs fine-tuning
README.md - Training workflows → Training datasets: loads examples
README.md - Training datasets → Data processing: processes examples
- Data processing → Text and VL tokenizers: uses tokenizer
data_processor/steps/input_ids_messaging/example_to_feature.py - Data processing → Image processing: adapts images
- Training workflows → ERNIE models: trains models
README.md - ERNIE models → Model configuration: uses configuration
ernie/modeling_moe.py - MoE models → Distributed layers: uses parallel layers
ernie/modeling_moe.py - MoE models → ERNIE models: extends model layers
ernie/modeling_moe.py - Vision encoder → ERNIE models: provides vision features
- Toolkit user → Evaluation: requests evaluation
README.md - Evaluation → ERNIE models: loads model
erniekit/eval/eval.py - Toolkit user → VL inference: runs prediction
tools/inference/infer_vl.py - VL inference → Text and VL tokenizers: tokenizes input
tools/inference/infer_vl.py - VL inference → Image processing: preprocesses images
tools/inference/infer_vl.py - VL inference → MoE models: loads VL model
tools/inference/infer_vl.py - Toolkit user → Chat interface: starts conversation
README.md - Chat interface → Model server: uses server
- Model server → FastDeploy runtime: launches runtime
erniekit/chat/server.py - Toolkit user → Model export: requests export
README.md - Model export → ERNIE models: exports weights
erniekit/export/export.py