How ishan1440/rt-detr works
Project Type & Purpose
• RT-DETR is an open-source real-time object-detection framework implementing two generations of Detection Transformers (RT-DETR and RT-DETRv2).
• It provides dual backends (PaddlePaddle and PyTorch), complete training/evaluation pipelines, model export to ONNX/TensorRT/OpenVINO, and benchmarking scripts.
A. Version Packages
• RT-DETR (rtdetr_paddle / rtdetr_pytorch)
• RT-DETRv2 (rtdetrv2_paddle / rtdetrv2_pytorch)
B. Core Subsystems (common across versions/backends)
1. Configuration Loader
– YAML files under configs/ + core/yaml_utils
2. Data Pipeline
– Dataset readers (COCO, VOC), transforms, dataloader
3. Model Definition
– Backbones, Transformer encoder/decoder, detection head, post-processing
4. Engine / Trainer
– Training loop, callbacks, optimizer, metrics, checkpointing
5. Tools & CLI
– train.py, eval.py, infer.py, export_model.py, run_profile.py
6. Export & Deployment
– export_utils → ONNX / TorchScript / TensorRT / OpenVINO
7. Benchmark
– trtexec wrapper, dataset sampler, timing scripts
• User invokes Tools/CLI →
→ Configuration Loader reads YAML →
→ Data Pipeline produces batches →
→ Engine instantiates Model →
→ Forward pass + loss calculation →
→ Backprop → optimizer updates Model →
→ Metrics & Callback logs → checkpoint save
• For Inference / Export:
Tools call export_utils → produce serialized model →
External runtime (ONNXRuntime / TensorRT / OpenVINO) → inference results → benchmark utilities measure FPS/accuracy
Architectural Patterns & Design Principles
• Layered modular design: clear separation of config, data, model, engine, tools, and deployment.
• Plugin-style backbones & heads: new architectures can be registered via factory pattern in modeling/architectures.
• Abstract base classes for data sources, solvers, and engines, enabling backend-agnostic code.
• YAML-driven configuration: decouples hyperparameters & dataset definitions from code.
Technologies & Frameworks
• PaddlePaddle & PyTorch implementations in parallel directory structures.
• Configuration: Python + PyYAML
• Deployment: ONNX, TensorRT (CUDA/C++ extensions), OpenVINO
• Benchmark: Python wrappers, trtexec CLI
• Utilities: logging, profiler, CLI argument parsing
Detail Level
• Sub-components for one backend (e.g., rtdetr_pytorch/src/core, /data, /nn, /solver) and note that an analogous structure exists in the Paddle folder.
• Call out external modules (ext_op for custom CUDA kernels).