# ishan1440/rt-detr architecture

> Architecture diagram of the GitHub repository ishan1440/rt-detr, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ishan1440/rt-detr
- Repository: https://github.com/ishan1440/rt-detr

## Overview

##### Project Type & Purpose
- RT-DETR is an open-source real-time object-detection framework implementing two generations of Detection Transformers (RT-DETR and RT-DETRv2).
- It provides dual backends (PaddlePaddle and PyTorch), complete training/evaluation pipelines, model export to ONNX/TensorRT/OpenVINO, and benchmarking scripts.

A. Version Packages
- RT-DETR (rtdetr_paddle / rtdetr_pytorch)
- RT-DETRv2 (rtdetrv2_paddle / rtdetrv2_pytorch)
B. Core Subsystems (common across versions/backends)
1. Configuration Loader
– YAML files under configs/ + core/yaml_utils
2. Data Pipeline
– Dataset readers (COCO, VOC), transforms, dataloader
3. Model Definition
– Backbones, Transformer encoder/decoder, detection head, post-processing
4. Engine / Trainer
– Training loop, callbacks, optimizer, metrics, checkpointing
5. Tools & CLI
– train.py, eval.py, infer.py, export_model.py, run_profile.py
6. Export & Deployment
– export_utils → ONNX / TorchScript / TensorRT / OpenVINO
7. Benchmark
– trtexec wrapper, dataset sampler, timing scripts

- User invokes Tools/CLI →
→ Configuration Loader reads YAML →
→ Data Pipeline produces batches →
→ Engine instantiates Model →
→ Forward pass + loss calculation →
→ Backprop → optimizer updates Model →
→ Metrics & Callback logs → checkpoint save
- For Inference / Export:
Tools call export_utils → produce serialized model →
External runtime (ONNXRuntime / TensorRT / OpenVINO) → inference results → benchmark utilities measure FPS/accuracy

##### Architectural Patterns & Design Principles
- Layered modular design: clear separation of config, data, model, engine, tools, and deployment.
- Plugin-style backbones & heads: new architectures can be registered via factory pattern in modeling/architectures.
- Abstract base classes for data sources, solvers, and engines, enabling backend-agnostic code.
- YAML-driven configuration: decouples hyperparameters & dataset definitions from code.

##### Technologies & Frameworks
- PaddlePaddle & PyTorch implementations in parallel directory structures.
- Configuration: Python + PyYAML
- Deployment: ONNX, TensorRT (CUDA/C++ extensions), OpenVINO
- Benchmark: Python wrappers, trtexec CLI
- Utilities: logging, profiler, CLI argument parsing

##### Detail Level
- Sub-components for one backend (e.g., rtdetr_pytorch/src/core, /data, /nn, /solver) and note that an analogous structure exists in the Paddle folder.
- Call out external modules (ext_op for custom CUDA kernels).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Version Packages
    subgraph "Version Packages"
        direction TB
        RP["RT-DETR Paddle"]:::version
        RPT["RT-DETRv2 Paddle"]:::version
        RT["RT-DETR PyTorch"]:::version
        RT2["RT-DETRv2 PyTorch"]:::version
    end

    %% Paddle Workflow
    subgraph "Paddle Backend Workflow"
        direction TB
        CP["Configuration Loader (Paddle)"]:::configdata
        DP["Data Pipeline (Paddle)"]:::configdata
        MP["Model Definition (Paddle)"]:::modelengine
        EP["Engine / Trainer (Paddle)"]:::modelengine
        TP["Tools & CLI (Paddle)"]:::tools
        XP["Export & Deployment (Paddle)"]:::export
    end

    %% PyTorch Workflow
    subgraph "PyTorch Backend Workflow"
        direction TB
        CT["Configuration Loader (PyTorch)"]:::configdata
        DT["Data Pipeline (PyTorch)"]:::configdata
        MT["Model Definition (PyTorch)"]:::modelengine
        ET["Engine / Trainer (PyTorch)"]:::modelengine
        TT["Tools & CLI (PyTorch)"]:::tools
        XT["Export & Deployment (PyTorch)"]:::export
    end

    %% External & Benchmark
    subgraph "Deployment & Benchmark"
        direction TB
        OR["External Runtimes (ONNX/TensorRT/OpenVINO)"]:::benchmark
        BM["Benchmark Utilities"]:::benchmark
    end

    %% Connections from Version Packages to Workflows
    RP -->|uses| CP
    RP -->|uses| DP
    RP -->|uses| MP
    RP -->|uses| EP
    RP -->|uses| TP
    RP -->|uses| XP

    RT -->|uses| CT
    RT -->|uses| DT
    RT -->|uses| MT
    RT -->|uses| ET
    RT -->|uses| TT
    RT -->|uses| XT

    RPT -->|uses| CP
    RPT -->|uses| DP
    RPT -->|uses| MP
    RPT -->|uses| EP
    RPT -->|uses| TP
    RPT -->|uses| XP

    RT2 -->|uses| CT
    RT2 -->|uses| DT
    RT2 -->|uses| MT
    RT2 -->|uses| ET
    RT2 -->|uses| TT
    RT2 -->|uses| XT

    %% Paddle Workflow Flow
    CP -->|"reads YAML"| DP
    DP -->|"batch images"| EP
    EP -->|"instantiates model"| MP
    MP -->|"forward & loss"| EP
    EP -->|"metrics & checkpoint"| TP
    TP -->|"export_utils"| XP
    XP -->|"serialized model"| OR
    OR -->|"inference results"| BM

    %% PyTorch Workflow Flow (same)
    CT -->|"reads YAML"| DT
    DT -->|"batch images"| ET
    ET -->|"instantiates model"| MT
    MT -->|"forward & loss"| ET
    ET -->|"metrics & checkpoint"| TT
    TT -->|"export_onnx"| XT
    XT -->|"serialized model"| OR
    OR -->|"inference results"| BM

    %% Click Events
    click RP "https://github.com/ishan1440/rt-detr/tree/main/rtdetr_paddle/"
    click RPT "https://github.com/ishan1440/rt-detr/tree/main/rtdetrv2_paddle/"
    click RT "https://github.com/ishan1440/rt-detr/tree/main/rtdetr_pytorch/"
    click RT2 "https://github.com/ishan1440/rt-detr/tree/main/rtdetrv2_pytorch/"
    click CP "https://github.com/ishan1440/rt-detr/tree/main/rtdetr_paddle/configs/"
    click CP "https://github.com/ishan1440/rt-detr/tree/main/rtdetr_paddle/ppdet/core/config/"
    click CP "https://github.com/ishan1440/rt-detr/blob/main/rtdetr_paddle/ppdet/core/config/yaml_helpers.py"
    click CT "https://github.com/ishan1440/rt-detr/tree/main/rtdetr_pytorch/configs/"
    click CT "https://github.com/ishan1440/rt-detr/blob/main/rtdetr_pytorch/src/core/config.py"
    click CT "https://github.com/ishan1440/rt-detr/blob/main/rtdetr_pytorch/src/core/yaml_utils.py"
    click DP "https://github.com/ishan1440/rt-detr/tree/main/rtdetr_paddle/dataset/coco/"
    click DP "https://github.com/ishan1440/rt-detr/tree/main/rtdetr_paddle/dataset/voc/"
    click DP "https://github.com/ishan1440/rt-detr/tree/main/rtdetr_paddle/ppdet/data/source/"
    click DP "https://github.com/ishan1440/rt-detr/tree/main/rtdetr_paddle/ppdet/data/transform/"
    click DT "https://github.com/ishan1440/rt-detr/tree/main/rtdetr_pytorch/src/data/coco/"
    click DT "https://github.com/ishan1440/rt-detr/blob/main/rtdetr_pytorch/src/data/transforms.py"
    click DT "https://github.com/ishan1440/rt-detr/blob/main/rtdetr_pytorch/src/data/dataloader.py"
    click MP "https://github.com/ishan1440/rt-detr/tree/main/rtdetr_paddle/ppdet/modeling/architectures/"
    click MP "https://github.com/ishan1440/rt-detr/tree/main/rtdetr_paddle/ppdet/modeling/backbones/"
    click MP "https://github.com/ishan1440/rt-detr/blob/main/rtdetr_paddle/ppdet/modeling/heads/detr_head.py"
    click MP "https://github.com/ishan1440/rt-detr/blob/main/rtdetr_paddle/ppdet/modeling/losses/detr_loss.py"
    click MP "https://github.com/ishan1440/rt-detr/tree/main/rtdetr_paddle/ppdet/modeling/transformers/"
    click MT "https://github.com/ishan1440/rt-detr/tree/main/rtdetr_pytorch/src/nn/arch/"
    click MT "https://github.com/ishan1440/rt-detr/tree/main/rtdetr_pytorch/src/zoo/rtdetr/"
    click MT "https://github.com/ishan1440/rt-detr/tree/main/rtdetr_pytorch/src/nn/backbone/"
    click MT "https://github.com/ishan1440/rt-detr/tree/main/rtdetr_pytorch/src/nn/criterion/"
    click MT "https://github.com/ishan1440/rt-detr/blob/main/rtdetr_pytorch/src/zoo/rtdetr/rtdetr_postprocessor.py"
    click EP "https://github.com/ishan1440/rt-detr/blob/main/rtdetr_paddle/ppdet/engine/trainer.py"
    click EP "https://github.com/ishan1440/rt-detr/blob/main/rtdetr_paddle/ppdet/engine/callbacks.py"
    click EP "https://github.com/ishan1440/rt-detr/tree/main/rtdetr_paddle/ppdet/metrics/"
    click EP "https://github.com/ishan1440/rt-detr/blob/main/rtdetr_paddle/ppdet/utils/checkpoint.py"
    click ET "https://github.com/ishan1440/rt-detr/blob/main/rtdetr_pytorch/src/solver/det_engine.py"
    click ET "https://github.com/ishan1440/rt-detr/blob/main/rtdetr_pytorch/src/solver/solver.py"
    click TP "https://github.com/ishan1440/rt-detr/blob/main/rtdetr_paddle/tools/train.py"
    click TP "https://github.com/ishan1440/rt-detr/blob/main/rtdetr_paddle/tools/eval.py"
    click TP "https://github.com/ishan1440/rt-detr/blob/main/rtdetr_paddle/tools/infer.py"
    click TP "https://github.com/ishan1440/rt-detr/blob/main/rtdetr_paddle/tools/export_model.py"
    click TT "https://github.com/ishan1440/rt-detr/blob/main/rtdetr_pytorch/tools/train.py"
    click TT "https://github.com/ishan1440/rt-detr/blob/main/rtdetr_pytorch/tools/infer.py"
    click TT "https://github.com/ishan1440/rt-detr/blob/main/rtdetr_pytorch/tools/export_onnx.py"
    click XP "https://github.com/ishan1440/rt-detr/blob/main/rtdetr_paddle/ppdet/engine/export_utils.py"
    click XT "https://github.com/ishan1440/rt-detr/blob/main/rtdetr_pytorch/tools/export_onnx.py"
    click BM "https://github.com/ishan1440/rt-detr/tree/main/benchmark/"
    click BM "https://github.com/ishan1440/rt-detr/blob/main/benchmark/utils.py"
    click BM "https://github.com/ishan1440/rt-detr/blob/main/benchmark/trtinfer.py"

    %% Styles
    classDef version fill:#f2f2f2,stroke:#333,stroke-width:1px
    classDef configdata fill:#a2cfff,stroke:#333,stroke-width:1px
    classDef modelengine fill:#8fce00,stroke:#333,stroke-width:1px
    classDef tools fill:#ffcc66,stroke:#333,stroke-width:1px
    classDef export fill:#d49aff,stroke:#333,stroke-width:1px
    classDef benchmark fill:#cccccc,stroke:#333,stroke-width:1px
```
