How regob/vehicle_mtmc works
Main Components
• Input and Data Sources:
– Video streams and datasets (see “datasets” folder and config paths).
– Pretrained models (resnet50 re-ID, SVM classifiers for vehicle attributes) with instructions for download.
• Detection Module:
– Uses Yolov5 (note the “detection/yolov5” submodule) for vehicle detection.
• Single-Camera Tracking (MOT):
– Implements trackers inside the “mot” directory with separate implementations for ByteTrack and DeepSORT.
• Re-Identification (ReID):
– Found in the “reid” directory, used to extract embeddings to help match vehicle identities.
• Multi-Target Multi-Camera Tracking (MTMC):
– The “mtmc” directory clusters and merges tracks from different cameras using hierarchical clustering.
– Also involves camera layout configuration (config/cityflow, config/zala) to enforce temporal and spatial constraints.
• Evaluation:
– The “evaluate” directory provides scripts and methods to compute metrics such as MOTA and IDF1.
• Auxiliary Tools & Scripts:
– The “tools” folder contains various helper utilities for data conversion, metric calculations, and video processing.
– Test suites for unit testing are maintained under “test”.
Architecture Relationships and Data Flow
• Input Flow: Video inputs from one or multiple cameras are fed into the detection module.
• Detection: Each frame is processed by the YOLOv5-based detector to locate vehicles.
• Feature Extraction: Detected objects are passed through a re-ID model to extract features. In parallel, the attribute extraction components (using either torch or other ML models) produce additional labels.
• MOT Tracking: With detection and features, the MOT tracker (either ByteTrack or DeepSORT) creates single-camera tracks.
• MTMC Integration: For multi-camera use cases, the “mtmc” module takes the single-camera tracks, applies hierarchical clustering (using configurations like LINKAGE type and MIN_SIM thresholds), and enforces spatial–temporal constraints based on a provided camera layout.
• Evaluation: The “evaluate” module then computes the tracking metrics on the results.
• Output: Results, annotated videos (e.g., stored in “output”), and logs are generated and stored accordingly.
Architectural Patterns & Design Principles
• Modularity: Each major function (detection, tracking, re-ID, MTMC, evaluation) is organized in its own directory.
• Configuration-Driven: The “config” folder (with subfolders for cityflow and zala) contains YAML and calibration files that separate settings from code.
• Extensibility: The system is built to easily support adding new attributes, switching trackers, or integrating new models through configurable parameters.
• Reusability: Use of external submodules (e.g., yolov5, vehicle_reid, and adaptations from DeepSORT and ByteTrack) indicates that the design emphasizes component reuse.
Technologies, Frameworks, and Libraries
• Python (likely 3.7–3.10) as the primary development language.
• Deep learning frameworks like PyTorch, as noticed by pretrained models and references to torch.
• Computer vision and tracking dependencies via YOLOv5, ByteTrack, and DeepSORT.
• Utilities and libraries for evaluation, clustering, and video processing.
• External tools for camera calibration are referenced (e.g., Cal_PNP).