# triple-mu/yolov8-tensorrt architecture

> Architecture diagram of the GitHub repository triple-mu/yolov8-tensorrt, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/triple-mu/yolov8-tensorrt
- Repository: https://github.com/triple-mu/yolov8-tensorrt

## Overview

YOLOv8-TensorRT converts trained YOLO models into TensorRT engines and runs image inference through Python or C++ for detection, segmentation, pose, oriented boxes, and classification. The main workflow is model export, engine construction, image preprocessing, inference, task-specific decoding, then visualized or saved results. Python provides interchangeable Torch, CUDA Runtime, and PyCUDA backends; C++ shares a core engine runtime with task-specific executables. README documents an optional DeepStream parser. Source excerpts confirm parts of the inference and C++ runtime paths; unsampled task and backend wiring is represented only where README establishes the capability.

Main entry points: [`infer.py`](https://github.com/triple-mu/yolov8-tensorrt/blob/main/infer.py) (Inference entry), [`csrc/core/src/runner.cpp`](https://github.com/triple-mu/yolov8-tensorrt/blob/main/csrc/core/src/runner.cpp) (Image runner), [`models`](https://github.com/triple-mu/yolov8-tensorrt/tree/main/models) (Inference backends), [`csrc/core/src/engine.cpp`](https://github.com/triple-mu/yolov8-tensorrt/blob/main/csrc/core/src/engine.cpp) (Engine runtime)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_model["Model preparation"]
  node_export_det["Detection export<br/>[export-det.py]"]
  node_export_seg["Segmentation export<br/>[export-seg.py]"]
  node_engine_builder["Engine builder<br/>[build.py]"]
end

subgraph group_python["Python inference"]
  node_python_entry["Inference entry<br/>[infer.py]"]
  node_task_select["Task handlers"]
  node_py_preprocess["Preprocessing<br/>[utils.py]"]
  node_py_backend["Inference backends"]
  node_torch_backend["Torch backend<br/>[engine.py]"]
  node_cudart_backend["CUDA Runtime backend<br/>[cudart_api.py]"]
  node_pycuda_backend["PyCUDA backend<br/>[pycuda_api.py]"]
  node_py_decode["Task decoding"]
  node_py_results["Rendered results<br/>[infer.py]"]
end

subgraph group_cpp["C++ inference"]
  node_cpp_apps["Task executables"]
  node_cpp_runner["Image runner<br/>[runner.cpp]"]
  node_cpp_engine["Engine runtime<br/>[engine.cpp]"]
  node_cpp_preprocess["Image preprocessing<br/>[preprocess.cpp]"]
  node_cpp_task["Task decoding and drawing"]
  node_labels["Class labels<br/>[labels.cpp]"]
  node_cpp_results["Rendered results<br/>[runner.cpp]"]
end

subgraph group_integrations["Optional integration"]
  node_deepstream["DeepStream parser"]
end

node_developer(("Model user"))
node_onnx["ONNX model"]
node_trt["TensorRT"]
node_engine_file["TensorRT engine"]
node_images["Input images"]
node_deepstream_sdk["DeepStream"]

node_developer -->|"exports weights"| node_export_det
node_developer -->|"exports weights"| node_export_seg
node_export_det -->|"produces ONNX"| node_onnx
node_export_seg -->|"produces ONNX"| node_onnx
node_onnx -->|"builds from"| node_engine_builder
node_engine_builder -->|"builds engine"| node_trt
node_engine_builder -->|"writes engine"| node_engine_file
node_developer -->|"runs inference"| node_python_entry
node_images -->|"supplies images"| node_python_entry
node_python_entry -->|"selects task"| node_task_select
node_python_entry -->|"selects backend"| node_py_backend
node_py_backend -->|"dispatches"| node_torch_backend
node_py_backend -->|"dispatches"| node_cudart_backend
node_py_backend -->|"dispatches"| node_pycuda_backend
node_engine_file -->|"deserializes"| node_torch_backend
node_engine_file -->|"loads engine"| node_cudart_backend
node_engine_file -->|"loads engine"| node_pycuda_backend
node_python_entry -->|"prepares images"| node_py_preprocess
node_python_entry -->|"decodes outputs"| node_py_decode
node_task_select -->|"task preprocessing"| node_py_preprocess
node_task_select -->|"task postprocessing"| node_py_decode
node_python_entry -->|"saves or displays"| node_py_results
node_developer -->|"runs executable"| node_cpp_apps
node_images -->|"supplies images"| node_cpp_runner
node_cpp_apps -->|"calls runner"| node_cpp_runner
node_cpp_apps -->|"creates engine"| node_cpp_engine
node_engine_file -->|"loads engine"| node_cpp_engine
node_cpp_runner -->|"runs inference"| node_cpp_engine
node_cpp_engine -->|"preprocesses via"| node_cpp_preprocess
node_cpp_runner -->|"calls task methods"| node_cpp_task
node_cpp_task -->|"loads class names"| node_labels
node_cpp_runner -->|"writes or displays"| node_cpp_results
node_deepstream_sdk -.->|"dispatches parser"| node_deepstream
node_deepstream -.->|"decodes detections"| node_cpp_results

click node_export_det "https://github.com/triple-mu/yolov8-tensorrt/blob/main/export-det.py"
click node_export_seg "https://github.com/triple-mu/yolov8-tensorrt/blob/main/export-seg.py"
click node_engine_builder "https://github.com/triple-mu/yolov8-tensorrt/blob/main/build.py"
click node_python_entry "https://github.com/triple-mu/yolov8-tensorrt/blob/main/infer.py"
click node_task_select "https://github.com/triple-mu/yolov8-tensorrt/tree/main/models/tasks"
click node_py_preprocess "https://github.com/triple-mu/yolov8-tensorrt/blob/main/models/utils.py"
click node_py_backend "https://github.com/triple-mu/yolov8-tensorrt/tree/main/models"
click node_torch_backend "https://github.com/triple-mu/yolov8-tensorrt/blob/main/models/engine.py"
click node_cudart_backend "https://github.com/triple-mu/yolov8-tensorrt/blob/main/models/cudart_api.py"
click node_pycuda_backend "https://github.com/triple-mu/yolov8-tensorrt/blob/main/models/pycuda_api.py"
click node_py_decode "https://github.com/triple-mu/yolov8-tensorrt/tree/main/models/tasks"
click node_py_results "https://github.com/triple-mu/yolov8-tensorrt/blob/main/infer.py"
click node_cpp_apps "https://github.com/triple-mu/yolov8-tensorrt/tree/main/csrc/apps"
click node_cpp_runner "https://github.com/triple-mu/yolov8-tensorrt/blob/main/csrc/core/src/runner.cpp"
click node_cpp_engine "https://github.com/triple-mu/yolov8-tensorrt/blob/main/csrc/core/src/engine.cpp"
click node_cpp_preprocess "https://github.com/triple-mu/yolov8-tensorrt/blob/main/csrc/core/src/preprocess.cpp"
click node_cpp_task "https://github.com/triple-mu/yolov8-tensorrt/tree/main/csrc/apps"
click node_labels "https://github.com/triple-mu/yolov8-tensorrt/blob/main/csrc/core/src/labels.cpp"
click node_cpp_results "https://github.com/triple-mu/yolov8-tensorrt/blob/main/csrc/core/src/runner.cpp"
click node_deepstream "https://github.com/triple-mu/yolov8-tensorrt/blob/main/csrc/deepstream/custom_bbox_parser/nvdsparsebbox_yoloV8.cpp"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_export_det,node_export_seg,node_engine_builder,node_developer toneBlue
class node_python_entry,node_task_select,node_py_preprocess,node_py_backend,node_torch_backend,node_cudart_backend,node_pycuda_backend,node_py_decode,node_py_results toneAmber
class node_cpp_apps,node_cpp_runner,node_cpp_engine,node_cpp_preprocess,node_cpp_task,node_labels,node_cpp_results toneMint
class node_deepstream toneRose
class node_onnx,node_trt,node_engine_file,node_images,node_deepstream_sdk toneIndigo
```

## Components

### Model preparation

- [Detection export](https://github.com/triple-mu/yolov8-tensorrt/blob/main/export-det.py): `export-det.py`
- [Segmentation export](https://github.com/triple-mu/yolov8-tensorrt/blob/main/export-seg.py): `export-seg.py`
- [Engine builder](https://github.com/triple-mu/yolov8-tensorrt/blob/main/build.py): `build.py`

### Python inference

- [Inference entry](https://github.com/triple-mu/yolov8-tensorrt/blob/main/infer.py): `infer.py`
- [Task handlers](https://github.com/triple-mu/yolov8-tensorrt/tree/main/models/tasks): `models/tasks`
- [Preprocessing](https://github.com/triple-mu/yolov8-tensorrt/blob/main/models/utils.py): `models/utils.py`
- [Inference backends](https://github.com/triple-mu/yolov8-tensorrt/tree/main/models): `models`
- [Torch backend](https://github.com/triple-mu/yolov8-tensorrt/blob/main/models/engine.py): `models/engine.py`
- [CUDA Runtime backend](https://github.com/triple-mu/yolov8-tensorrt/blob/main/models/cudart_api.py): `models/cudart_api.py`
- [PyCUDA backend](https://github.com/triple-mu/yolov8-tensorrt/blob/main/models/pycuda_api.py): `models/pycuda_api.py`
- [Task decoding](https://github.com/triple-mu/yolov8-tensorrt/tree/main/models/tasks): `models/tasks`
- [Rendered results](https://github.com/triple-mu/yolov8-tensorrt/blob/main/infer.py): `infer.py`

### C++ inference

- [Task executables](https://github.com/triple-mu/yolov8-tensorrt/tree/main/csrc/apps): `csrc/apps`
- [Image runner](https://github.com/triple-mu/yolov8-tensorrt/blob/main/csrc/core/src/runner.cpp): `csrc/core/src/runner.cpp`
- [Engine runtime](https://github.com/triple-mu/yolov8-tensorrt/blob/main/csrc/core/src/engine.cpp): `csrc/core/src/engine.cpp`
- [Image preprocessing](https://github.com/triple-mu/yolov8-tensorrt/blob/main/csrc/core/src/preprocess.cpp): `csrc/core/src/preprocess.cpp`
- [Task decoding and drawing](https://github.com/triple-mu/yolov8-tensorrt/tree/main/csrc/apps): `csrc/apps`
- [Class labels](https://github.com/triple-mu/yolov8-tensorrt/blob/main/csrc/core/src/labels.cpp): `csrc/core/src/labels.cpp`
- [Rendered results](https://github.com/triple-mu/yolov8-tensorrt/blob/main/csrc/core/src/runner.cpp): `csrc/core/src/runner.cpp`

### Optional integration

- [DeepStream parser](https://github.com/triple-mu/yolov8-tensorrt/blob/main/csrc/deepstream/custom_bbox_parser/nvdsparsebbox_yoloV8.cpp): `csrc/deepstream/custom_bbox_parser/nvdsparsebbox_yoloV8.cpp`

### Other components

- **Model user**
- **ONNX model**
- **TensorRT**
- **TensorRT engine**
- **Input images**
- **DeepStream**

## Connections

- Model user → Detection export: exports weights
- Model user → Segmentation export: exports weights
- Detection export → ONNX model: produces ONNX
- Segmentation export → ONNX model: produces ONNX
- ONNX model → Engine builder: builds from
- Engine builder → TensorRT: builds engine
- Engine builder → TensorRT engine: writes engine
- Model user → Inference entry: runs inference
- Input images → Inference entry: supplies images
- Inference entry → Task handlers: selects task
- Inference entry → Inference backends: selects backend
- Inference backends → Torch backend: dispatches
- Inference backends → CUDA Runtime backend: dispatches
- Inference backends → PyCUDA backend: dispatches
- TensorRT engine → Torch backend: deserializes
- TensorRT engine → CUDA Runtime backend: loads engine
- TensorRT engine → PyCUDA backend: loads engine
- Inference entry → Preprocessing: prepares images
- Inference entry → Task decoding: decodes outputs
- Task handlers → Preprocessing: task preprocessing
- Task handlers → Task decoding: task postprocessing
- Inference entry → Rendered results: saves or displays
- Model user → Task executables: runs executable
- Input images → Image runner: supplies images
- Task executables → Image runner: calls runner
- Task executables → Engine runtime: creates engine
- TensorRT engine → Engine runtime: loads engine
- Image runner → Engine runtime: runs inference
- Engine runtime → Image preprocessing: preprocesses via
- Image runner → Task decoding and drawing: calls task methods
- Task decoding and drawing → Class labels: loads class names
- Image runner → Rendered results: writes or displays
- DeepStream → DeepStream parser: dispatches parser
- DeepStream parser → Rendered results: decodes detections