How jay-huy/diffmot works
Project Type and Purpose
• DiffMOT is a research‐grade, full‐pipeline multiple object tracking system.
• It combines:
• A detector (YOLOX)
• A ReID feature extractor (deep‐person‐reid / fast_reid)
• A diffusion‐based motion prediction model (D²MP)
• A tracker core (association, Kalman‐based & non‐linear update)
• It is designed for real‐time, high‐accuracy tracking on benchmarks (DanceTrack, SportsMOT, MOT17/20).
High‐Level Architectural Components
A) Data Preparation & Configuration
• Config files (configs/*.yaml) define dataset paths, thresholds, weights (association, embedding)
• Data processors (dancetrack_data_process.py, sports_data_process.py, mot_data_process.py) convert raw video frames + ground truth into standardized inputs
B) Detection Module (external/YOLOX)
• Pluggable: supports YOLOX‐S/M/L/X
C) ReID Embedding Module (external/deep-person-reid, external/fast_reid)
• Takes cropped detections → outputs deep feature embeddings
D) Motion Prediction Module (models/ & main.py)
• Diffusion‐based motion model (“D²MP”) implemented in models/denoising_diffusion_pytorch.py & related
• Trains to predict non‐linear future positions
E) Tracker Core (tracker/ & tracking_utils/)
• Association logic (matching.py, gmc.py, cmc.py)
• Kalman‐filter variants (tracker/ocsort_tracker/kalmanfilter.py, tracking_utils/kalman_filter.py)
• Track management (DiffMOTtracker.py, basetrack.py)
• Non‐maximum suppression, metric evaluation, logging, visualization
F) Experiment & Evaluation
• experiments/ holds logs, TensorBoard events for training curves
• external/TrackEval for standardized metric computation
Component Relationships & Data Flow
• Raw frames → Data Preparation → Detector → Detected BBoxes → ReID Module → Embeddings → Motion Model → Predicted trajectories → Tracker Core (association & update) → Output tracks
• Config Manager feeds hyperparameters into each module
• Logging & Visualization subscribes to all stages for metrics and debugging
• External libraries (YOLOX, deep‐person‐reid, fast_reid, TrackEval) are integrated as pluggable components
Architectural Patterns & Principles
• Modular Pipeline: Each stage is a well‐defined module with clear I/O
• Pluggable Components: Swap detectors or ReID extractors via config
• Separation of Concerns: Data I/O, model training, inference, tracking logic, evaluation are distinct
• Reuse of External Tools: Leverage off‐the‐shelf detection, ReID, and evaluation libraries
Detailed Elements
• Config Manager (configs/*.yaml) reading
• Data Processors feeding Detector/ReID
• Detector module (external/YOLOX)
• ReID module (external/deep‐person‐reid & fast_reid)
• Diffusion Motion Model (models/*.py, main.py)
• Tracker Core (tracker/, tracking_utils/)
• Evaluation & Logging (experiments/, external/TrackEval)
• Input frames → Detector
• Detector → ReID
• ReID + previous tracks → Motion Model
• Motion Model + embeddings → Association logic
• Association logic → Updated track states
• Final tracks → Output / Visualizer → TrackEval