# lyuwenyu/rt-detr architecture

> Architecture diagram of the GitHub repository lyuwenyu/rt-detr, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/lyuwenyu/rt-detr
- Repository: https://github.com/lyuwenyu/rt-detr

## Overview

##### Project type and purpose
- This is an open-source object-detection library and toolkit (“RT-DETR” and “RT-DETRv2”) that provides both PaddlePaddle and PyTorch implementations of high-performance, real-time DETR models.
- It supplies training pipelines, inference scripts, benchmarking utilities, model export/deployment tools (ONNX, TensorRT, OpenVINO), and pre-built configurations & weights.

##### File structure
- Top-level directories:
- rtdetr_paddle / rtdetr_pytorch: v1 implementations in Paddle and PyTorch
- rtdetrv2_paddle / rtdetrv2_pytorch: v2 implementations
- benchmark: standalone scripts for speed/accuracy tests
- tools under each implement directory: train.py, infer.py, export_model.py, run_profile.py, etc.
- Common internal patterns:
- configs/: YAML configuration files (dataset, model, runtime)
- src/ or ppdet/: source code split into data, modeling, engine, metrics, optimizer, utils
- tools/: entry points for training, evaluation, inference, export
- references/deploy/: sample deployment code for runtimes (TensorRT, ONNXRuntime, OpenVINO)
- Build and deployment files: Dockerfile, docker-compose.yml, hubconf.py, onnx2trt.sh

##### README for additional insights
- Confirms two versions (v1 and v2), two backends (Paddle, PyTorch).
- Points to supported datasets (COCO, VOC, Objects365), external integrations (Torch Hub, Hugging Face).
- Highlights performance benchmarking and deployment discussions.

a. Main components
- Configuration Manager (YAML files + workspace loader)
- Data Pipeline: dataset loaders → transforms → batch operators
- Model Factory: backbones → DETR transformer modules → heads → post-processing
- Training Engine/Solver: optimizer, scheduler, callbacks, EMA, metrics
- Inference Engine: model loading, pre-/post-processing, slicing support
- Export/Deployment Interfaces: ONNX, TensorRT, OpenVINO, Torch Hub
- Benchmark Suite: scripts that wrap inference engine + metrics
b. Relationships & interactions
- Config Manager feeds parameters to Data Pipeline, Model Factory, and Training/Inference Engines.
- Data Pipeline outputs batches to Training Engine; Training Engine updates Model Factory via Optimizer.
- Inference Engine reuses Model Factory + Data Pipeline for real-time inference.
- Export Interfaces take the trained model from Model Factory and produce deployable artifacts.
- Benchmark Suite invokes Inference Engine and metrics modules to report performance.
c. Architectural patterns & design principles
- Modular separation: data, model, engine, utils all decoupled
- Config-driven: behavior controlled entirely by YAML + CLI flags
- Backend-agnostic abstractions: same pattern used in Paddle and PyTorch implementations
- Extensibility points: custom datasets, custom backbones, transformer operators (ext_op)
- Layered architecture: low-level tensor ops → modeling primitives → training/inference orchestration → deployment
d. Key technologies/frameworks
- PaddlePaddle & PyTorch deep-learning frameworks
- YAML for configuration (yaml_helpers, yaml_utils)
- CUDA/C++ extension for deformable attention (ms_deformable_attn_op)
- ONNX, TensorRT, OpenVINO for deployment
- Docker for reproducible environments

##### Essential details to capture
- Config management at the center (common entry point).
- Two parallel stacks (Paddle vs. PyTorch) abstracted behind identical module interfaces.
- Key scripts/tools at the edge: train.py, infer.py, export_model.py, run_profile.py.
- Deployment reference folder showing how onnx/trt/openvino scripts consume exported models.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Configuration Manager
    subgraph "Configuration Manager" 
        direction TB
        CM1["Paddle v1 Configs"]:::config
        CM2["schema.py"]:::config
        CM3["yaml_helpers.py"]:::config
        CM4["workspace.py"]:::config
        CM5["PyTorch v1 Configs"]:::config
        CM6["config.py"]:::config
        CM7["yaml_config.py"]:::config
        CM8["yaml_utils.py"]:::config
        CM9["PyTorch v2 Configs"]:::config
        CM10["_config.py"]:::config
        CM11["yaml_config.py"]:::config
        CM12["yaml_utils.py"]:::config
    end

    %% Core Library Modules
    subgraph "Core Library" 
        direction TB
        DP["Data Pipeline"]:::data
        MF["Model Factory"]:::modeling
        TE["Training Engine"]:::training
        IE["Inference Engine"]:::inference
    end

    %% Deployment & Export
    subgraph "Export & Deployment Interfaces"
        direction TB
        EX["Export Interfaces"]:::deployment
    end

    %% Benchmark
    BS["Benchmark Suite"]:::utils

    %% Arrows
    CM1 -->|"provides configs to"| DP
    CM1 -->|"provides configs to"| MF
    CM1 -->|"provides configs to"| TE
    CM1 -->|"provides configs to"| IE
    DP -->|"batches for"| TE
    TE -->|"updates weights in"| MF
    IE -->|"uses model from"| MF
    MF -->|"exports model to"| EX
    EX -->|"consumed by"| BS
    IE -->|"evaluated by"| BS

    %% Click events for Configuration Manager components
    click CM1 "https://github.com/lyuwenyu/rt-detr/tree/main/rtdetr_paddle/configs/"
    click CM2 "https://github.com/lyuwenyu/rt-detr/blob/main/rtdetr_paddle/ppdet/core/config/schema.py"
    click CM3 "https://github.com/lyuwenyu/rt-detr/blob/main/rtdetr_paddle/ppdet/core/config/yaml_helpers.py"
    click CM4 "https://github.com/lyuwenyu/rt-detr/blob/main/rtdetr_paddle/ppdet/core/workspace.py"
    click CM5 "https://github.com/lyuwenyu/rt-detr/tree/main/rtdetr_pytorch/configs/"
    click CM6 "https://github.com/lyuwenyu/rt-detr/blob/main/rtdetr_pytorch/src/core/config.py"
    click CM7 "https://github.com/lyuwenyu/rt-detr/blob/main/rtdetr_pytorch/src/core/yaml_config.py"
    click CM8 "https://github.com/lyuwenyu/rt-detr/blob/main/rtdetr_pytorch/src/core/yaml_utils.py"
    click CM9 "https://github.com/lyuwenyu/rt-detr/tree/main/rtdetrv2_pytorch/configs/"
    click CM10 "https://github.com/lyuwenyu/rt-detr/blob/main/rtdetrv2_pytorch/src/core/_config.py"
    click CM11 "https://github.com/lyuwenyu/rt-detr/blob/main/rtdetrv2_pytorch/src/core/yaml_config.py"
    click CM12 "https://github.com/lyuwenyu/rt-detr/blob/main/rtdetrv2_pytorch/src/core/yaml_utils.py"

    %% Styles
    classDef config fill:#AED6F1,stroke:#1F618D,color:#1F618D
    classDef data fill:#AED6F1,stroke:#1F618D,color:#1F618D
    classDef modeling fill:#A9DFBF,stroke:#196F3D,color:#196F3D
    classDef training fill:#A9DFBF,stroke:#196F3D,color:#196F3D
    classDef inference fill:#F9E79F,stroke:#B9770E,color:#B9770E
    classDef deployment fill:#F5CBA7,stroke:#B03A2E,color:#B03A2E
    classDef utils fill:#D5DBDB,stroke:#6E2C00,color:#6E2C00
```
