How quangdungluong/deepstream-yolov11 works
Project type and purpose
• This is an open-source toolkit that supplies custom TensorRT parsers (C++ plugins) for AI models in NVIDIA’s DeepStream SDK.
• It streamlines the end-to-end flow of exporting backbone networks (YOLOv11, SCRFD, D-FINE) to ONNX, converting them to TRT engines, and loading them into a DeepStream pipeline.
File structure
• Top level
– configs/: DeepStream and parser configuration files for multiple model modes (det, seg, pose).
– libs/: Custom parser code (C++/CUDA) organized by model type (yolo, yolo_pose, yolo_seg) plus build/install scripts.
– models/: Python export scripts and model-specific assets for ONNX conversion.
– scripts/: Shell helpers to compile parsers.
– docs/: Usage guides for each model and feature.
– CI/config files (.github, .pre-commit, etc.).
• Architectural cues
– libs/ → plugin architecture for DeepStream.
– models/ → code-based model export pipeline.
– configs/ → declarative wiring between parsers, engines, and deepstream-app.
README for additional insights
• Workflow stages documented:
1. Clone repo
2. Build custom parser with compile_nvdsinfer.sh
3. Export model to ONNX via Python scripts
4. Convert ONNX to TRT engine with trtexec
5. Edit parser & DeepStream config (.txt)
6. Run deepstream-app
• Target platforms: Jetson & dGPU, DeepStream SDK 6.2–7.1.
a. Main components
– Developer workstation (or CI) with code repository & build environment
– Build system (compile_nvdsinfer.sh, Makefiles) producing .so parser libraries
– Model export subsystem (Python scripts under models/) outputting ONNX
– TensorRT conversion (trtexec) producing .engine files
– DeepStream runtime: deepstream-app + SDK plugins + custom parser libraries + TRT engines
– Video I/O: camera/file input → inference → overlay/tracker → output
– External dependencies: NVIDIA GPU/Jetson hardware, CUDA, ONNX, DeepStream SDK, TensorRT
b. Relationships & data flow
– Code repo → Build system → Parser shared libraries
– Code repo → Model export scripts → ONNX files → TensorRT conversion → engine files
– DeepStream-app loads configs, custom parser .so, TRT engine → ingests video → GIE plugin → metadata → display or downstream bus
c. Architectural patterns & principles
– Plugin architecture: libs/nvdsinfer_customparser_* are dynamically loaded by DeepStream’s GIE plugin.
– Dataflow/streaming pipeline: DeepStream’s mux → GIE → tracker → OSD → sink.
– Separation of concerns: export, conversion, configuration, runtime all live in distinct directories.
d. Key technologies/frameworks
– C++, CUDA, cuDNN (through TRT), Python (model export), ONNX, TensorRT, DeepStream SDK