How lyuwenyu/rt-detr works
Project type and purpose
• This is an open-source object-detection library and toolkit (“RT-DETR” and “RT-DETRv2”) that provides both PaddlePaddle and PyTorch implementations of high-performance, real-time DETR models.
• It supplies training pipelines, inference scripts, benchmarking utilities, model export/deployment tools (ONNX, TensorRT, OpenVINO), and pre-built configurations & weights.
File structure
• Top-level directories:
• rtdetr_paddle / rtdetr_pytorch: v1 implementations in Paddle and PyTorch
• rtdetrv2_paddle / rtdetrv2_pytorch: v2 implementations
• benchmark: standalone scripts for speed/accuracy tests
• tools under each implement directory: train.py, infer.py, export_model.py, run_profile.py, etc.
• Common internal patterns:
• configs/: YAML configuration files (dataset, model, runtime)
• src/ or ppdet/: source code split into data, modeling, engine, metrics, optimizer, utils
• tools/: entry points for training, evaluation, inference, export
• references/deploy/: sample deployment code for runtimes (TensorRT, ONNXRuntime, OpenVINO)
• Build and deployment files: Dockerfile, docker-compose.yml, hubconf.py, onnx2trt.sh
README for additional insights
• Confirms two versions (v1 and v2), two backends (Paddle, PyTorch).
• Points to supported datasets (COCO, VOC, Objects365), external integrations (Torch Hub, Hugging Face).
• Highlights performance benchmarking and deployment discussions.
a. Main components
• Configuration Manager (YAML files + workspace loader)
• Data Pipeline: dataset loaders → transforms → batch operators
• Model Factory: backbones → DETR transformer modules → heads → post-processing
• Training Engine/Solver: optimizer, scheduler, callbacks, EMA, metrics
• Inference Engine: model loading, pre-/post-processing, slicing support
• Export/Deployment Interfaces: ONNX, TensorRT, OpenVINO, Torch Hub
• Benchmark Suite: scripts that wrap inference engine + metrics
b. Relationships & interactions
• Config Manager feeds parameters to Data Pipeline, Model Factory, and Training/Inference Engines.
• Data Pipeline outputs batches to Training Engine; Training Engine updates Model Factory via Optimizer.
• Inference Engine reuses Model Factory + Data Pipeline for real-time inference.
• Export Interfaces take the trained model from Model Factory and produce deployable artifacts.
• Benchmark Suite invokes Inference Engine and metrics modules to report performance.
c. Architectural patterns & design principles
• Modular separation: data, model, engine, utils all decoupled
• Config-driven: behavior controlled entirely by YAML + CLI flags
• Backend-agnostic abstractions: same pattern used in Paddle and PyTorch implementations
• Extensibility points: custom datasets, custom backbones, transformer operators (ext_op)
• Layered architecture: low-level tensor ops → modeling primitives → training/inference orchestration → deployment
d. Key technologies/frameworks
• PaddlePaddle & PyTorch deep-learning frameworks
• YAML for configuration (yaml_helpers, yaml_utils)
• CUDA/C++ extension for deformable attention (ms_deformable_attn_op)
• ONNX, TensorRT, OpenVINO for deployment
• Docker for reproducible environments
Essential details to capture
• Config management at the center (common entry point).
• Two parallel stacks (Paddle vs. PyTorch) abstracted behind identical module interfaces.
• Key scripts/tools at the edge: train.py, infer.py, export_model.py, run_profile.py.
• Deployment reference folder showing how onnx/trt/openvino scripts consume exported models.