# marcoslucianops/deepstream-yolo architecture

> Architecture diagram of the GitHub repository marcoslucianops/deepstream-yolo, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/marcoslucianops/deepstream-yolo
- Repository: https://github.com/marcoslucianops/deepstream-yolo

## Overview

##### Project type and purpose
- This is an SDK-based inference pipeline integration, not a web app. It leverages NVIDIA DeepStream (a GStreamer-based video analytics framework) plus custom plugins for running various YOLO (and YOLO-like) models via TensorRT.

##### File structure
Top-level:
- config_infer_primary_*.txt — one config per supported model
- deepstream_app_config.txt — main GStreamer/DeepStream pipeline config
- docs/ — per-model usage and notes
- nvdsinfer_custom_impl_Yolo/ — C/C++/CUDA plugin source for YOLO parsing
- utils/ — Python scripts to export model ONNX files for each supported model
- labels.txt — common class labels
Key observations:
- The project follows a plugin approach (DeepStream’s custom-infer extension).
- It separates model conversion (utils → ONNX) from runtime inference (custom plugin + DeepStream).
- Build scripts (Makefile) compile the custom plugin into a shared library.

##### README for additional insights
- The architecture is a GStreamer pipeline anchored by DeepStream SDK elements (nvstreammux, primary-gie, tracker, sink).
- Custom YOLO parser library is loaded by the `primary-gie` element.
- Utilities generate ONNX models; TensorRT engine creation is triggered by DeepStream at startup.
- Support for INT8 calibration implies an optional calibration stage before engine building.

##### Main components & relationships (
a. Input sources
- File/RTSP/Camera —> GStreamer source elements
b. DeepStream core pipeline
1. nvstreammux — batches frames
2. nvinfer (primary-gie) — configured by deepstream_app_config.txt and model-specific config_infer_primary_*.txt
– loads TensorRT engine (built from ONNX or Darknet via utils + plugin)
– links to custom parser library `libnvdsinfer_custom_impl_Yolo.so`
3. nvinfer parsing stage — invokes custom layers (activation, conv, NMS, etc.) implemented under nvdsinfer_custom_impl_Yolo/layers
4. Metadata probe — NVDS metadata (`NvDsObjectMeta`, `NvDsFrameMeta`) exposed for post-processing
5. (Optional) tracker, secondary GIEs, tiler
6. Sink — render/display or file/network output
c. Model conversion & calibration
- utils/*.py scripts — export model from upstream repos to ONNX
- INT8 Calibration — uses calibrator.cpp to generate calibrator table, applied by TensorRT builder
d. Environment & dependencies
- DeepStream SDK (GStreamer + TensorRT + CUDA)
- NVIDIA Drivers
- Docker container or host OS
- External repos: Darknet, ultralytics/yolov5, etc.

##### Architectural patterns & principles
- GStreamer pipeline pattern: each functional block is a distinct plugin element.
- Plugin-based extensibility: custom parsers for each YOLO variant, isolated in `nvdsinfer_custom_impl_Yolo`.
- Separation of concerns: model-export (Python scripts) vs. runtime inference (C++ plugin).
- Configuration-driven: new YOLO variants supported by adding a new config_infer_primary_*.txt and export script.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Dependencies
    subgraph "External Dependencies"
        CUDA["CUDA"]:::external
        TRT["TensorRT"]:::external
        DS["DeepStream SDK"]:::external
        OS["OS/Driver"]:::external
    end

    %% Build-Time Components
    subgraph "Build-Time Components"
        MAIN_UTIL["ONNX Export / Model Conversion Utilities"]:::build
        CAL_MOD["Calibration Module"]:::build
        DOCS["Documentation (Usage & Notes)"]:::config
        README["Project Overview & Build Instructions"]:::config
        CFG_MODEL["Model-Specific Inference Configurations"]:::config
        LBL["Common Class Labels"]:::config
        subgraph "Custom YOLO Parser Plugin"
            PLUGIN_ROOT["nvdsinfer_custom_impl_Yolo/"]:::build
            MAKEFILE["Makefile"]:::build
            CAL_CPP["calibrator.cpp/.h"]:::build
            CORE["nvdsinfer_yolo_engine.cpp / yolo.cpp/.h / utils.cpp/.h"]:::build
            KERNELS["yoloForward CUDA Kernels"]:::build
            INTERFACES["yoloPlugins.cpp/.h"]:::build
            SPARSE["nvdsparsebbox_Yolo.cpp / .cu"]:::build
            LAYERS["layers/*.cpp/.h"]:::build
        end
        TRT_BUILD["TensorRT Builder"]:::build
    end

    %% Run-Time Pipeline
    subgraph "Run-Time Pipeline"
        SRC["GStreamer Source"]:::runtime
        MUX["nvstreammux"]:::runtime
        INFER["nvinfer (primary-gie)"]:::runtime
        PARSER["Custom YOLO Parser Library"]:::runtime
        METADATA["Metadata Probe"]:::runtime
        SINK["Sink (Render/Output)"]:::runtime
    end

    %% Data Flow
    SRC --> MUX --> INFER --> PARSER --> METADATA --> SINK

    %% Configuration & Control Paths
    MAIN_UTIL -.->|export ONNX| TRT_BUILD
    CAL_MOD -.->|calibration data| TRT_BUILD
    TRT_BUILD -.->|build engine| INFER
    CFG_MODEL -.-> INFER
    LBL -.-> METADATA
    DOCS -.-> MAIN_UTIL
    README -.-> MAIN_UTIL

    %% Runtime Dependencies
    CUDA --> DS
    TRT --> DS
    DS --> INFER
    OS --> DS

    %% Plugin Loading
    INFER -->|loads| PARSER

    %% Click Events
    click MAIN_CFG "https://github.com/marcoslucianops/deepstream-yolo/blob/master/deepstream_app_config.txt"
    click CFG_MODEL "https://github.com/marcoslucianops/deepstream-yolo/blob/master/config_infer_primary_yoloV5.txt"
    click LBL "https://github.com/marcoslucianops/deepstream-yolo/blob/master/labels.txt"
    click DOCS "https://github.com/marcoslucianops/deepstream-yolo/tree/master/docs/"
    click MAIN_UTIL "https://github.com/marcoslucianops/deepstream-yolo/blob/master/utils/export_yoloV5.py"
    click PLUGIN_ROOT "https://github.com/marcoslucianops/deepstream-yolo/tree/master/nvdsinfer_custom_impl_Yolo/"
    click MAKEFILE "https://github.com/marcoslucianops/deepstream-yolo/tree/master/nvdsinfer_custom_impl_Yolo/Makefile"
    click CAL_CPP "https://github.com/marcoslucianops/deepstream-yolo/blob/master/nvdsinfer_custom_impl_Yolo/calibrator.cpp"
    click CORE "https://github.com/marcoslucianops/deepstream-yolo/blob/master/nvdsinfer_custom_impl_Yolo/nvdsinfer_yolo_engine.cpp"
    click KERNELS "https://github.com/marcoslucianops/deepstream-yolo/blob/master/nvdsinfer_custom_impl_Yolo/yoloForward.cu"
    click INTERFACES "https://github.com/marcoslucianops/deepstream-yolo/blob/master/nvdsinfer_custom_impl_Yolo/yoloPlugins.cpp"
    click SPARSE "https://github.com/marcoslucianops/deepstream-yolo/blob/master/nvdsinfer_custom_impl_Yolo/nvdsparsebbox_Yolo.cpp"
    click LAYERS "https://github.com/marcoslucianops/deepstream-yolo/blob/master/nvdsinfer_custom_impl_Yolo/layers/activation_layer.cpp"

    %% Styles
    classDef runtime fill:#cce5ff,stroke:#005cbf,color:#005cbf
    classDef build fill:#d4edda,stroke:#155724,color:#155724
    classDef config fill:#fff3cd,stroke:#856404,color:#856404
    classDef external fill:#e2e3e5,stroke:#6c757d,color:#6c757d
```
