How mohit-robo/reid works
1. What this repo is
- A multi-camera person tracking / re-identification system, centered on assigning stable global IDs across disjoint camera views.
- It is split between a production-style orchestration layer in scripts/ and a reusable package in trackers/ that contains tracking, IO, evaluation, motion, and tuning subsystems.
2. Top-level runtime flow
- The main live pipeline is scripts/run_pipeline.py.
- It takes one or more video streams, reads frames concurrently, runs per-frame detection and tracking, extracts appearance embeddings, associates local tracks to global identities, and writes annotated output video.
- The README’s core flow is: video frames → YOLOv8 detections → BoTSORT local tracks → OSNet embeddings → global identity assignment via AnchorPool → stabilized labels / rendering.
3. Per-camera local tracking layer
- Each camera stream is processed independently at first.
- The repository includes multiple tracker implementations under trackers/core/: BoT-SORT, ByteTrack, SORT, OCSORT, and CBIoU.
- These implementations share common utilities and tracklet structures, which makes the tracking algorithms interchangeable at the local-camera boundary.
4. Cross-camera identity subsystem
- The MTMC logic lives mostly in scripts/anchor_pool.py, scripts/embedding_associator.py, scripts/embedding_bank.py, scripts/tracklet.py, and scripts/majority_vote.py.
- EmbeddingAssociator post-processes tracker outputs using appearance similarity.
- AnchorPool is the shared registry of global identities; it stores anchor embeddings and performs batched similarity matching to mint or reuse IDs.
- EmbeddingBank maintains per-camera appearance history, and tracklet.py adds expired-track handling for the ghost gallery / re-entry logic.
- MajorityVoteBuffer smooths unstable identity labels before they are shown or emitted.
5. Appearance feature extraction
- The system uses OSNet (osnet_x1_0, pretrained on MSMT17) as the appearance encoder.
- This is the key cross-camera feature source; it replaces relying only on detector/tracker state for identity continuity.
- The architecture therefore has a distinct “visual identity” path separate from the motion-based local tracker path.
6. Camera re-entry and persistence
- The architecture explicitly handles temporary disappearance across cameras.
- Expired local tracklets are moved into a Ghost Gallery, where they remain matchable for a TTL window.
- When a person reappears in another camera, the system can recover the same global ID by matching the new embedding against cached ghost anchors before falling back to a new identity.
7. Motion and state modeling
- The trackers/motion/ and trackers/utils/kalman_filter.py modules provide motion estimation and transformation support.
- These are the state-estimation pieces used by the local trackers and related utilities.
- This is a separate concern from appearance matching: motion stabilizes per-camera tracks, while embeddings solve cross-camera identity continuity.
8. Input/output and data interfaces
- trackers/io/ contains the boundary layer for data ingestion and emission:
- video.py and frames.py for stream/frame handling,
- mot.py for MOT-style interchange,
- paths.py for path normalization and asset resolution.
- The scripts/gst_video_reader.py and scripts/gst_video_writer.py files show that GStreamer is used for robust video IO and to avoid Python data-loading conflicts in the live pipeline.
- Output is annotated video, not just numeric results.
9. Evaluation and tuning
- trackers/eval/ contains standard multi-object tracking metrics and result handling: CLEAR, HOTA, ID metrics, box/result utilities.
- trackers/scripts/eval.py and trackers/scripts/tune.py indicate the project supports offline evaluation and parameter search, not only live inference.
- trackers/tune/tuner.py and trackers/scripts/progress.py suggest an experiment loop around tracker/re-ID settings.
10. Dataset and download support
- trackers/datasets/ and trackers/utils/downloader.py manage dataset manifests and retrieval.
- The root download_videos.sh and filter_and_download.py are focused on fetching or filtering sample videos for pipeline runs.
- This means the repo includes both runtime inference and data acquisition tooling, which is important when diagramming the full system boundary.
11. Packaging and deployment
- The repo is designed to run both locally and in Docker.
- Dockerfile and run_docker.sh package the runtime environment; the README also calls out extra dependencies like torchreid, gdown, torchvision, and system GStreamer libraries.
- In practice, the deployment boundary is: containerized Python runtime + GPU-enabled PyTorch + video IO stack + pretrained weights.
12. Validation boundaries
- tests/ contains install and phase tests (test_install.py, test_phase0.py, test_phase1.py, test_phase2.py).
- These imply the repository is organized around incremental pipeline phases, with tests verifying that the environment and major architecture stages remain functional.
13. How to draw the architecture
- The central diagram should show:
- multiple video inputs,
- per-stream decode/read,
- per-frame detection and local tracking,
- OSNet embedding extraction,
- shared global ID registry / AnchorPool,
- ghost-gallery re-entry path,
- majority-vote stabilization,
- annotated video output,
- and an offline evaluation/tuning branch fed from saved results.
- The trackers/ package is the reusable algorithmic core; scripts/ is the orchestration and productization layer.
Main entry points: run_pipeline.py Run pipeline · base.py Track base · embedding_associator.py Embedding assoc · tracker.py BoTSORT
Components 24
Orchestration
Live MTMC pipeline, global identity coordination, and video I/O wrappers.
- Run pipeline · Entry point that drives multi-camera detection, tracking, re-identification, and rendering.
scripts/run_pipeline.py - Video reader · Robust stream ingestion for live or file-based video sources.
scripts/gst_video_reader.py - Video writer · Annotated video emission for the live pipeline.
scripts/gst_video_writer.py - AnchorPool · Shared store that mints, reuses, and matches global identities by embedding similarity.
scripts/anchor_pool.py - Embedding assoc · Post-processes tracker outputs using cross-camera appearance similarity.
scripts/embedding_associator.py - Embedding bank · Maintains per-camera appearance history and candidate embeddings.
scripts/embedding_bank.py - Tracklet state · Handles expired tracks, re-entry windows, and ghost-gallery persistence.
scripts/tracklet.py - Majority vote · Stabilizes noisy identity assignments before output.
scripts/majority_vote.py
Tracking Core
Reusable local multi-object tracking algorithms and shared tracklet utilities.
- BoTSORT · Primary per-camera tracker implementation used by the pipeline.
trackers/core/botsort/tracker.py - ByteTrack · Alternative per-camera tracking implementation.
trackers/core/bytetrack/tracker.py - SORT · Baseline tracking implementation for local association.
trackers/core/sort/tracker.py - OCSORT · Motion-aware tracker variant for local association.
trackers/core/ocsort/tracker.py - CBIoU · IoU-based tracking variant for local association.
trackers/core/cbiou/tracker.py - Track base · Common tracker abstractions and tracklet state used across algorithms.
trackers/core/base.py
Platform
Input/output, motion/state support, evaluation, tuning, datasets, and helper utilities.
- Video IO · Frame and video handling boundary for ingesting and emitting media.
trackers/io/video.py - MOT IO · MOT-style interchange and record formatting for tracker outputs.
trackers/io/mot.py - Motion models · Motion estimation and geometric transforms supporting local tracking.
trackers/motion/estimator.py - Evaluation · CLEAR, HOTA, identity metrics, and result evaluation utilities.
trackers/eval/evaluate.py - Tuning · Search and progress tooling for tracker and re-ID parameter tuning.
trackers/tune/tuner.py - Datasets · Dataset manifests and download helpers for sample or benchmark data.
trackers/datasets/manifest.py - Utilities · Common device, geometry, detection, and Kalman-filter helpers.
trackers/utils/general.py
Ops
Containerization, downloads, documentation, and tests around the runtime.
- Docker · Packaged runtime environment for GPU-enabled video processing.
Dockerfile - Download tools · Scripts for fetching and filtering source videos.
download_videos.sh - Tests · Install and phased tests covering environment and pipeline stages.
tests/test_phase0.py
Connections 23
- Run pipeline → Video reader: reads · Consumes one or more video streams.
- Run pipeline → Video IO: uses · Coordinates frame and video boundaries.
- Run pipeline → BoTSORT: tracks · Runs per-camera local tracking.
- Run pipeline → Embedding assoc: re-id · Links local tracks to cross-camera identities.
- Run pipeline → AnchorPool: global IDs · Looks up and updates stable identities.
- Run pipeline → Majority vote: stabilizes · Smooths unstable labels before rendering.
- Run pipeline → Video writer: writes · Emits annotated output video.
- BoTSORT → Track base: extends · Builds on shared tracker abstractions.
- ByteTrack → Track base: extends · Builds on shared tracker abstractions.
- SORT → Track base: extends · Builds on shared tracker abstractions.
- OCSORT → Track base: extends · Builds on shared tracker abstractions.
- CBIoU → Track base: extends · Builds on shared tracker abstractions.
- BoTSORT → Motion models: uses · Combines motion cues with detection updates.
- OCSORT → Motion models: uses · Combines motion cues with detection updates.
- Track base → Utilities: depends on · Shares common geometry, state, and filtering helpers.
- Embedding assoc → Embedding bank: updates · Stores per-camera appearance history.
- Embedding assoc → AnchorPool: matches · Compares embeddings against global anchors.
- Tracklet state → AnchorPool: ghost match · Supports re-entry from expired-track memory.
- Evaluation → MOT IO: reads · Evaluates saved tracking outputs.
- Tuning → Evaluation: optimizes · Searches tracker and re-ID settings against metrics.
- Datasets → Download tools: feeds · Supplies source data acquisition and filtering.
- Docker → Run pipeline: packages · Encapsulates the live runtime environment.
- Tests → Run pipeline: verifies · Checks staged pipeline functionality and installation.