How paddlepaddle/paddle works
PaddlePaddle is a deep-learning framework spanning model training and inference, automatic/distributed parallelism, differentiation, and compiler/runtime optimization. A principal workflow is to define a model, loss, and optimizer; use the static auto-parallel engine to build and execute programs; and obtain training or inference outputs. Separate branches represent pipeline parallelism, IR/compiler facilities, and optimized inference integrations. The README documents broader capabilities than the sampled source demonstrates; unsampled subsystem links are therefore omitted rather than inferred.
Main entry points: engine.py Auto-parallel engine · storage_manager.h IR storage manager · tensorrt_engine_op.h TensorRT operator · pipeline_parallel.py Pipeline parallel
Components 19
Python training
- Auto-parallel engine
python/paddle/distributed/auto_parallel/static/engine.py - Model and loss
- Optimizer
- Training outputs
Parallel execution
- Pipeline parallel
python/paddle/distributed/fleet/meta_parallel/pipeline_parallel.py - Pipeline optimizer
python/paddle/incubate/optimizer/pipeline.py
IR and compiler
- IR storage manager
paddle/pir/include/core/storage_manager.h - Storage implementation
paddle/pir/src/core/storage_manager.cc - AP expression interpreter
paddle/ap/include/axpr/interpreter.h - DRR rewriting
paddle/ap/include/drr/drr_interpreter.h - Kernel code generation
paddle/ap/include/code_gen/op_code_gen_ctx.h
Execution runtime
- Program executor
- Partial gradients
paddle/fluid/imperative/partial_grad_engine.cc - Kernel dispatch
paddle/ap/include/paddle/phi/kernel_dispatch_helper.h - Device context
paddle/ap/include/kernel_dispatch/device_ctx.h
Inference integrations
- TensorRT operator
paddle/fluid/operators/tensorrt/tensorrt_engine_op.h - TensorRT engine
paddle/fluid/platform/tensorrt/engine.cc - Custom engine manager
paddle/fluid/custom_engine/custom_engine_manager.cc
Other components
- Model developer
Connections 10
- Model developer → Auto-parallel engine: configures
- Auto-parallel engine → Model and loss: accepts
- Auto-parallel engine → Optimizer: accepts
- Auto-parallel engine → Program executor: runs programs
- Program executor → Training outputs: returns outputs
- Model developer → Pipeline parallel: configures
- Model developer → Pipeline optimizer: configures
- IR storage manager → Storage implementation: implemented by
- TensorRT operator → TensorRT engine: prepares and executes
- Model developer → Custom engine manager: registers integration