How opencv/opencv_zoo works
1. This repository is an OpenCV model zoo, not a single application. The core architectural pattern is “one model family per directory,” each packaged with its own weights, model wrapper, demos, and example outputs.
2. The main runtime boundary is the model package. Under models/…, each subdirectory is a self-contained inference module for a specific task such as face detection, object detection, text recognition, segmentation, tracking, or pose estimation. Most packages expose:
- a Python demo,
- a C++ demo for OpenCV builds,
- a model-specific Python wrapper,
- pretrained artifacts (.onnx, sometimes .caffemodel/.prototxt),
- and sample output images or videos.
3. The common inference technology across the zoo is OpenCV DNN. The README emphasizes models “tuned for OpenCV DNN,” and the per-model demo scripts are the primary execution entrypoints. Several models also include int8 / int8bq variants, showing that deployment targets include quantized execution on edge accelerators.
4. The repo has a strong split between model-specific logic and shared tooling. The model-specific code lives inside each models/<name> directory, while reusable support code is centralized in benchmark/utils and tools/*. This keeps inference wrappers isolated from benchmarking, evaluation, and quantization concerns.
5. The benchmark subsystem is a separate pipeline for performance measurement. It reads model-specific config files from benchmark/config, downloads benchmark data via benchmark/download_data.py, runs timed inference through benchmark/benchmark.py, and aggregates results into tables using benchmark/generate_table.py. The benchmark README and table_config.yaml drive which models and devices are reported.
6. Benchmark execution is built around a reusable internal framework:
- dataloaders provide model/task-specific input iteration,
- metrics define task-appropriate scoring,
- factory.py selects implementations by config,
- timer.py measures elapsed inference time.
This suggests a generic benchmark harness that can support detection, recognition, and tracking with different data and scoring rules.
7. The tools/eval subtree is a separate evaluation layer from benchmarking. It contains dataset adapters for common vision datasets such as ImageNet, LFW, WIDER Face, ICDAR, IIIT5K, and MiniSupervisely. This is the repo’s “offline evaluation” boundary: benchmarks focus on speed and some metrics, while eval tools are for accuracy testing against standard datasets.
8. The tools/quantize subtree is the deployment optimization pipeline. It contains:
- quantize-ort.py for ONNX Runtime–style quantization workflows,
- quantize-inc.py and inc_configs for Intel Neural Compressor–based quantization configurations,
- block_quantize.py and transform.py for model graph transformations.
This indicates that quantization is treated as an external build/deployment step that produces the int8/int8bq artifacts used by many models.
9. The model directories show the supported task graph very clearly. The zoo covers:
- detection and recognition chains for faces, license plates, QR codes, and text,
- human-centric tasks such as pose, hand pose, and segmentation,
- general vision tasks like classification, tracking, optical flow, edge detection, inpainting, and deblurring.
These are mostly standalone inference endpoints, but some are composable into pipelines, such as face detection plus face recognition.
10. Some model packages have multiple backend or format inputs, which affects architecture boundaries. Most use ONNX, but WeChatQRCode uses Caffe models, and several packages include multiple precision variants. That means the repo is not tied to a single model exchange format; instead, each package encapsulates the format required by that model family.
11. The demos are dual-language where needed. Many model directories provide both demo.py and demo.cpp, and the C++ demos are accompanied by CMakeLists.txt. This implies two supported execution paths: Python for quick usage and C++ for native/OpenCV build integration.
12. The top-level reports directory contains generated benchmark/report artifacts rather than active runtime code. It functions as documentation/output storage for published benchmark results, while the authoritative executable paths are under models, benchmark, and tools.
13. Operationally, model assets are large and are intended to be fetched via Git LFS. The README also notes that the project is now hosted on Hugging Face for models and online demos, so the repository is best understood as an open-source source tree plus a distribution layer for model weights and examples.
14. If you were drawing an architecture diagram, the key boxes would be: per-model inference packages, shared benchmark framework, shared evaluation datasets/adapters, shared quantization tooling, and generated reports. The main data flow is: datasets or input media → model wrapper/demo → OpenCV DNN inference → post-processing/output, with benchmark/eval/quantize as auxiliary pipelines around that core.
Main entry points: eval Eval Tools · quantize Quantize Tools · benchmark.py Benchmark Runner · face_detection_yunet Face Detect
Components 21
Model Zoo
Self-contained model packages, each bundling wrappers, demos, and weights.
- Face Detect · YuNet face detection package with Python/C++ demos and quantized variants.
models/face_detection_yunet - Face Rec · SFace face recognition package with matching demos and int8 variants.
models/face_recognition_sface - Face Quality · Face image quality assessment package.
models/face_image_quality_assessment_ediffiqa - Detection · General object and license-plate detection families.
models/object_detection_nanodet - Text Detect · PPOCR-based text detection package.
models/text_detection_ppocr - Text Rec · CRNN text recognition package with multiple language/precision variants.
models/text_recognition_crnn - Pose & Hand · MediaPipe-based human pose and hand pose packages.
models/pose_estimation_mediapipe - Segmentation · Human and generic image segmentation packages.
models/human_segmentation_pphumanseg - Tracking · ViTTrack object tracking package.
models/object_tracking_vittrack - Restoration · Deblurring and inpainting packages.
models/deblurring_nafnet - Classification · Image classification families such as MobileNet and PPResNet.
models/image_classification_mobilenet - QR Code · WeChatQRCode detection and super-resolution package.
models/qrcode_wechatqrcode - Optical Flow · RAFT optical flow package.
models/optical_flow_estimation_raft
Benchmarking
Performance measurement and report generation pipeline.
- Benchmark Runner · Timed inference entrypoint for benchmark execution.
benchmark/benchmark.py - Data Loaders · Task-specific benchmark input iteration and factory selection.
benchmark/utils/dataloaders - Metrics · Benchmark scoring for detection, recognition, and tracking tasks.
benchmark/utils/metrics - Table Gen · Aggregates benchmark runs into report tables.
benchmark/generate_table.py
Tools
Shared evaluation and quantization subsystems.
- Eval Tools · Dataset adapters and evaluation scripts for standard vision benchmarks.
tools/eval - Quantize Tools · ONNX Runtime and Intel Neural Compressor quantization workflows and graph transforms.
tools/quantize
Generated Output
Published reports and sample artifacts.
- Reports · Published benchmark reports and generated artifacts.
reports
Other components
- OpenCV DNN · Common inference backend used across the zoo.
Connections 27
- Face Detect → OpenCV DNN: inference · Face detection demos execute through the shared OpenCV DNN backend.
- Face Rec → OpenCV DNN: inference · Face recognition demos execute through the shared OpenCV DNN backend.
- Face Quality → OpenCV DNN: inference · Face quality assessment runs on the shared backend.
- Detection → OpenCV DNN: inference · Detection families are tuned for OpenCV DNN execution.
- Text Detect → OpenCV DNN: inference · Text detection uses the common inference runtime.
- Text Rec → OpenCV DNN: inference · Text recognition uses the common inference runtime.
- Pose & Hand → OpenCV DNN: inference · Pose and hand models use the common inference runtime.
- Segmentation → OpenCV DNN: inference · Segmentation packages use the common inference runtime.
- Tracking → OpenCV DNN: inference · Tracking models run through OpenCV DNN.
- Restoration → OpenCV DNN: inference · Restoration models run through OpenCV DNN.
- Classification → OpenCV DNN: inference · Classification models run through OpenCV DNN.
- QR Code → OpenCV DNN: inference · QR code models run through OpenCV DNN.
- Optical Flow → OpenCV DNN: inference · Optical flow models run through OpenCV DNN.
- Benchmark Runner → Data Loaders: loads inputs · Benchmark runner uses task-specific dataloaders.
- Benchmark Runner → Metrics: scores runs · Benchmark runner uses task-specific metrics.
- Benchmark Runner → OpenCV DNN: measures · Benchmark runner times inference on the same backend used by model demos.
- Table Gen → Benchmark Runner: consumes results · Table generation aggregates benchmark outputs.
- Eval Tools → Detection: evaluates · Evaluation adapters cover detection-related datasets and models.
- Eval Tools → Face Detect: evaluates · Evaluation adapters include face datasets.
- Eval Tools → Text Detect: evaluates · Evaluation adapters include text datasets.
- Eval Tools → Text Rec: evaluates · Evaluation adapters include text recognition datasets.
- Eval Tools → Classification: evaluates · Evaluation adapters include image classification datasets.
- Quantize Tools → Face Detect: produces int8 · Quantization pipeline generates deployment variants for model packages.
- Quantize Tools → Face Rec: produces int8 · Quantization pipeline generates deployment variants for model packages.
- Quantize Tools → Classification: produces int8 · Quantization pipeline generates deployment variants for model packages.
- Quantize Tools → Text Rec: produces int8 · Quantization pipeline generates deployment variants for model packages.
- Table Gen → Reports: publishes · Benchmark tables are stored as generated report artifacts.