dingfengshi/tridet

Purpose and entry points

TriDet is a PyTorch/CUDA implementation of temporal action detection from pre-extracted video features. It trains a relative-boundary detection model and evaluates temporal segments with mAP. Primary entry points are train.py and eval.py; reproducible dataset/model invocations are wrapped by scripts such as tools/thumos_i3d_script.sh and tools/hacs_slowfast_script.sh.

Configuration-driven execution

YAML configs define dataset, feature source, model, optimization, and evaluation settings. Configuration loading/merging is centralized in libs/core/config.py. Dataset-specific experiment profiles include configs/thumos_i3d.yaml, configs/hacs_slowfast.yaml, and configs/anet_tsp.yaml; train.py consumes one to construct the full training pipeline, while eval.py additionally receives a checkpoint.

Feature dataset and annotation boundary

The system consumes stored feature tensors rather than decoding raw video. Dataset adapters load video-level features and temporal annotations, normalize/sample them for training or inference, and present a common dataset interface:

libs/datasets/datasets.py, libs/datasets/thumos14.py, libs/datasets/anet.py.

Dataset-specific variants also support EPIC-Kitchens and HACS in libs/datasets/epic_kitchens.py and libs/datasets/hacs.py. HACS annotation inputs are versioned JSON assets under data/hacs/annotations/HACS_segments_v1.1.1.json.

Detection model assembly

Model construction is registry/factory-driven through libs/modeling/models.py and the top-level detection architecture in libs/modeling/meta_archs.py. The architecture transforms temporal feature sequences through a backbone and multi-scale neck, then produces class and temporal-boundary predictions. Core reusable layers, including the relative boundary modeling primitives, reside in libs/modeling/blocks.py; temporal feature extraction is defined in libs/modeling/backbones.py, with pyramid/feature adaptation in libs/modeling/necks.py.

Temporal localization, training loss, and optimization

Temporal locations/priors aligned to multi-scale feature maps are generated in libs/modeling/loc_generators.py. During training, the meta-architecture matches predictions to annotated action segments and computes classification/localization losses via libs/modeling/losses.py. The training runtime coordinates model state, dataloading, distributed/single-GPU execution, checkpointing, and validation through train.py and libs/utils/train_utils.py; schedule policies are in libs/utils/lr_schedulers.py.

Inference and evaluation boundary

Evaluation loads a trained checkpoint, runs batched inference, decodes temporal proposals, suppresses duplicates, and computes benchmark metrics. Postprocessing and result formatting are handled by libs/utils/postprocessing.py; temporal NMS dispatch is in libs/utils/nms.py; mAP-style evaluation metrics are in libs/utils/metrics.py. eval.py is the command boundary for producing benchmark results.

Native/runtime dependencies

The repository requires Python, PyTorch (>=1.11 in documented experiments), CUDA, and package dependencies in requirements.txt. Performance-critical CPU NMS is a compiled extension from libs/utils/csrc/nms_cpu.cpp, installed through libs/utils/setup.py. External inputs are downloaded feature archives/checkpoints: ActionFormer-provided THUMOS14/ActivityNet/EPIC features, official HACS I3D or TCANet SlowFast features, and pretrained checkpoints hosted on Google Drive.