How triple-mu/yolov8-tensorrt works
YOLOv8-TensorRT converts trained YOLO models into TensorRT engines and runs image inference through Python or C++ for detection, segmentation, pose, oriented boxes, and classification. The main workflow is model export, engine construction, image preprocessing, inference, task-specific decoding, then visualized or saved results. Python provides interchangeable Torch, CUDA Runtime, and PyCUDA backends; C++ shares a core engine runtime with task-specific executables. README documents an optional DeepStream parser. Source excerpts confirm parts of the inference and C++ runtime paths; unsampled task and backend wiring is represented only where README establishes the capability.
Main entry points: infer.py Inference entry · runner.cpp Image runner · models Inference backends · engine.cpp Engine runtime
Components 26
Model preparation
- Detection export
export-det.py - Segmentation export
export-seg.py - Engine builder
build.py
Python inference
- Inference entry
infer.py - Task handlers
models/tasks - Preprocessing
models/utils.py - Inference backends
models - Torch backend
models/engine.py - CUDA Runtime backend
models/cudart_api.py - PyCUDA backend
models/pycuda_api.py - Task decoding
models/tasks - Rendered results
infer.py
C++ inference
- Task executables
csrc/apps - Image runner
csrc/core/src/runner.cpp - Engine runtime
csrc/core/src/engine.cpp - Image preprocessing
csrc/core/src/preprocess.cpp - Task decoding and drawing
csrc/apps - Class labels
csrc/core/src/labels.cpp - Rendered results
csrc/core/src/runner.cpp
Optional integration
- DeepStream parser
csrc/deepstream/custom_bbox_parser/nvdsparsebbox_yoloV8.cpp
Other components
- Model user
- ONNX model
- TensorRT
- TensorRT engine
- Input images
- DeepStream
Connections 34
- Model user → Detection export: exports weights
- Model user → Segmentation export: exports weights
- Detection export → ONNX model: produces ONNX
- Segmentation export → ONNX model: produces ONNX
- ONNX model → Engine builder: builds from
- Engine builder → TensorRT: builds engine
- Engine builder → TensorRT engine: writes engine
- Model user → Inference entry: runs inference
- Input images → Inference entry: supplies images
- Inference entry → Task handlers: selects task
- Inference entry → Inference backends: selects backend
- Inference backends → Torch backend: dispatches
- Inference backends → CUDA Runtime backend: dispatches
- Inference backends → PyCUDA backend: dispatches
- TensorRT engine → Torch backend: deserializes
- TensorRT engine → CUDA Runtime backend: loads engine
- TensorRT engine → PyCUDA backend: loads engine
- Inference entry → Preprocessing: prepares images
- Inference entry → Task decoding: decodes outputs
- Task handlers → Preprocessing: task preprocessing
- Task handlers → Task decoding: task postprocessing
- Inference entry → Rendered results: saves or displays
- Model user → Task executables: runs executable
- Input images → Image runner: supplies images
- Task executables → Image runner: calls runner
- Task executables → Engine runtime: creates engine
- TensorRT engine → Engine runtime: loads engine
- Image runner → Engine runtime: runs inference
- Engine runtime → Image preprocessing: preprocesses via
- Image runner → Task decoding and drawing: calls task methods
- Task decoding and drawing → Class labels: loads class names
- Image runner → Rendered results: writes or displays
- DeepStream → DeepStream parser: dispatches parser
- DeepStream parser → Rendered results: decodes detections