How nanmi/yolov7-pose works
1. A custom TensorRT plugin for YOLO operations (found in the “YoloLayer_TRT_v7.0” directory) that provides a CUDA/C++ implementation of the custom layer.
2. A pose detection inference module (located in the “yolov7-pose” directory) that builds an executable to run inference using a generated TensorRT engine.
Project Components
• Custom TensorRT Plugin:
• Source files: “yololayer.cu”, “yololayer.h”, and additional support (macros.h, add_custom_yolo_op.py script).
• Build process: Uses CMakeLists.txt to compile a shared library (libyolo.so) containing the custom YOLO layer for TensorRT.
• YOLOv7 Pose Inference Module:
• Source and build files: “yolov7_pose.cpp”, associated headers (logging.h, macros.h, utils.h), CMakeLists.txt.
• Inference process: Consumes a pre-generated TensorRT engine (the engine is generated by trtexec in combination with the plugin), processes input images (e.g., person.jpg), and outputs visual results.
• Pre-processing / Model Export Pipeline (described in README):
• A Python script is described to convert a PyTorch YOLOv7 pose detection model into an ONNX model.
• This involves the modification of specific operators (reshapings, permutation) so that they are export-friendly.
• External Dependencies:
• CUDA, TensorRT, OpenCV, and PyTorch. These form the backbone for GPU support, high-performance inference, and image handling.
a. Model Preparation and Export:
• Document the Python code that converts the PyTorch model into an ONNX model.
• Necessity for modifying operations (reshaping, activation substitution) for export.
b. Plugin Build and Integration:
• “YoloLayer_TRT_v7.0” component where the custom ops are compiled into libyolo.so.
• A note that the “add_custom_yolo_op.py” script is used to add the custom operation into the TensorRT runtime.
c. Engine Creation:
• Use of the trtexec command to combine the ONNX model and the YOLO plugin library to create the TensorRT engine.
d. Inference Execution:
• Detail the “yolov7-pose” component:
• The build process, compiling “yolov7_pose.cpp” and linked against necessary libraries.
• The runtime flow where an image (for example, person.jpg) is processed by the inference engine to detect pose.
• From the Python model export stage to the ONNX model.
• From ONNX + Plugin (libyolo.so) via trtexec to creating the TensorRT engine.
• From the TensorRT engine being used in the inference executable that processes input images and produces output.
Architectural Principles and Technologies
• Clearly distinguish between build-time components (C++/CUDA source for plugins and inference; CMake-based builds) and runtime components (TensorRT engine, inference executable).
• Layered approach:
• A “Data Preparation” layer (model conversion),
• a “Plugin and Engine Integration” layer (custom op and engine build),
• and an “Inference Application” layer (pose inference using built engine).
• Use of industry-standard high-performance libraries: CUDA for GPU support, TensorRT for optimized inference, and OpenCV for image input/output.