How marcoslucianops/deepstream-yolo works
Project type and purpose
• This is an SDK-based inference pipeline integration, not a web app. It leverages NVIDIA DeepStream (a GStreamer-based video analytics framework) plus custom plugins for running various YOLO (and YOLO-like) models via TensorRT.
File structure
Top-level:
• config_infer_primary_*.txt — one config per supported model
• deepstream_app_config.txt — main GStreamer/DeepStream pipeline config
• docs/ — per-model usage and notes
• nvdsinfer_custom_impl_Yolo/ — C/C++/CUDA plugin source for YOLO parsing
• utils/ — Python scripts to export model ONNX files for each supported model
• labels.txt — common class labels
Key observations:
• The project follows a plugin approach (DeepStream’s custom-infer extension).
• It separates model conversion (utils → ONNX) from runtime inference (custom plugin + DeepStream).
• Build scripts (Makefile) compile the custom plugin into a shared library.
README for additional insights
• The architecture is a GStreamer pipeline anchored by DeepStream SDK elements (nvstreammux, primary-gie, tracker, sink).
• Custom YOLO parser library is loaded by the primary-gie element.
• Utilities generate ONNX models; TensorRT engine creation is triggered by DeepStream at startup.
• Support for INT8 calibration implies an optional calibration stage before engine building.
Main components & relationships (
a. Input sources
• File/RTSP/Camera —> GStreamer source elements
b. DeepStream core pipeline
1. nvstreammux — batches frames
2. nvinfer (primary-gie) — configured by deepstream_app_config.txt and model-specific config_infer_primary_*.txt
– loads TensorRT engine (built from ONNX or Darknet via utils + plugin)
– links to custom parser library libnvdsinfer_custom_impl_Yolo.so
3. nvinfer parsing stage — invokes custom layers (activation, conv, NMS, etc.) implemented under nvdsinfer_custom_impl_Yolo/layers
4. Metadata probe — NVDS metadata (NvDsObjectMeta, NvDsFrameMeta) exposed for post-processing
5. (Optional) tracker, secondary GIEs, tiler
6. Sink — render/display or file/network output
c. Model conversion & calibration
• utils/*.py scripts — export model from upstream repos to ONNX
• INT8 Calibration — uses calibrator.cpp to generate calibrator table, applied by TensorRT builder
d. Environment & dependencies
• DeepStream SDK (GStreamer + TensorRT + CUDA)
• NVIDIA Drivers
• Docker container or host OS
• External repos: Darknet, ultralytics/yolov5, etc.
Architectural patterns & principles
• GStreamer pipeline pattern: each functional block is a distinct plugin element.
• Plugin-based extensibility: custom parsers for each YOLO variant, isolated in nvdsinfer_custom_impl_Yolo.
• Separation of concerns: model-export (Python scripts) vs. runtime inference (C++ plugin).
• Configuration-driven: new YOLO variants supported by adding a new config_infer_primary_*.txt and export script.