How soccernet/sn-gamestate works
Project Type and Purpose
• This is a full-stack computer vision research framework that performs end-to-end game state reconstruction from soccer videos. It integrates multiple computer vision subtasks (e.g., camera calibration, person detection, jersey number recognition, tracking, and team affiliation) to generate a 2D minimap of the soccer game.
• The project is built on top of the TrackLab multi-object tracking framework. It augments TrackLab’s modular pipeline with SoccerNet-specific modules.
High-Level Components
• Input Data and Preprocessing:
• Video Input: Raw soccer game videos.
• Dataset management and downloading (via SoccerNetDownloader and TrackLab’s automatic dataset/model download).
• Calibration and Field Localization Modules:
• Several calibration plugins exist (e.g., “nbjw_calib”, “pnlcalib”, “tvcalib”), each with its own configuration files, models, and utilities. These handle camera calibration and pitch localisation.
• Detection and Tracking Modules:
• Under the sn_gamestate directory, you have code for multi-object tracking, person detection, jersey recognition (under “jersey”), and re-identification (reid).
• Configuration Infrastructure:
• Hydra configuration is used via YAML files present in “sn_gamestate/configs” (such as soccernet.yaml and submodule configs for calibration, pitch, jersey, reid, team, etc.).
• Evaluation and Metrics:
• GS-HOTA evaluation metric is defined in the README—this module receives tracked output and computes both localization and identification similarities.
• There is integration with a SoccerNet fork of TrackEval for evaluator functions.
• Visualization and Output:
• The system generates output videos (the minimap overlays), and components in “sn_gamestate/visualization” provide pitch visualization.
Relationships and Data Flow
• Input videos are first processed by the calibration modules where one can choose among the available pipelines (nbjw_calib, pnlcalib, tvcalib) for camera parameters estimation.
• Calibration feeds into the detection/tracking modules: once the pitch is localized, the pipeline extracts player proposals, identifies jersey numbers (via modules in “jersey”), and performs re-identification (in “reid”).
• Processed detections flow through tracking logic (TrackLab manages the multi-object tracking pipeline) and into a post-processing stage where they are formatted into a 2D minimap.
• The evaluation module then processes the tracker states, comparing predictions to ground truth with GS-HOTA’s combined localization and identity similarity.
• The final output is visualized in video format using the visualization modules.
Architectural Patterns and Design Principles
• Modularity: The architecture is highly modular with interchangeable calibration plugins and configurable pipeline modules (e.g., the calibration, pitch, jersey, team, and reid modules). This eases debugging and improvements on individual subtasks.
• Configuration Driven: Hydra-based YAML configuration (found in “sn_gamestate/configs”) decouples parameters from code, which is essential for experimentation.
• Pipeline-based Data Flow: The design follows a pipeline – each module receives input from its upstream component and passes results downstream.
Relevant Technologies and Frameworks
• TrackLab Framework: The main backbone for multi-object tracking.
• Hydra and YAML: For configuration management.
• Deep Learning Libraries: Modules (e.g., using mmcv, MMOCR) for detection, re-identification, and jersey recognition.
• Evaluation Libraries: A SoccerNet fork of TrackEval is used for computing GS-HOTA.
• Other Common Tools: The project also employs utilities for dataset management, automatic model downloading, etc.