# dingfengshi/tridet architecture

> Architecture diagram of the GitHub repository dingfengshi/tridet, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-08-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dingfengshi/tridet
- Repository: https://github.com/dingfengshi/tridet

## Overview

#### Purpose and entry points
TriDet is a PyTorch/CUDA implementation of temporal action detection from pre-extracted video features. It trains a relative-boundary detection model and evaluates temporal segments with mAP. Primary entry points are `train.py` and `eval.py`; reproducible dataset/model invocations are wrapped by scripts such as `tools/thumos_i3d_script.sh` and `tools/hacs_slowfast_script.sh`.

#### Configuration-driven execution
YAML configs define dataset, feature source, model, optimization, and evaluation settings. Configuration loading/merging is centralized in `libs/core/config.py`. Dataset-specific experiment profiles include `configs/thumos_i3d.yaml`, `configs/hacs_slowfast.yaml`, and `configs/anet_tsp.yaml`; `train.py` consumes one to construct the full training pipeline, while `eval.py` additionally receives a checkpoint.

#### Feature dataset and annotation boundary
The system consumes stored feature tensors rather than decoding raw video. Dataset adapters load video-level features and temporal annotations, normalize/sample them for training or inference, and present a common dataset interface:
`libs/datasets/datasets.py`, `libs/datasets/thumos14.py`, `libs/datasets/anet.py`.
Dataset-specific variants also support EPIC-Kitchens and HACS in `libs/datasets/epic_kitchens.py` and `libs/datasets/hacs.py`. HACS annotation inputs are versioned JSON assets under `data/hacs/annotations/HACS_segments_v1.1.1.json`.

#### Detection model assembly
Model construction is registry/factory-driven through `libs/modeling/models.py` and the top-level detection architecture in `libs/modeling/meta_archs.py`. The architecture transforms temporal feature sequences through a backbone and multi-scale neck, then produces class and temporal-boundary predictions. Core reusable layers, including the relative boundary modeling primitives, reside in `libs/modeling/blocks.py`; temporal feature extraction is defined in `libs/modeling/backbones.py`, with pyramid/feature adaptation in `libs/modeling/necks.py`.

#### Temporal localization, training loss, and optimization
Temporal locations/priors aligned to multi-scale feature maps are generated in `libs/modeling/loc_generators.py`. During training, the meta-architecture matches predictions to annotated action segments and computes classification/localization losses via `libs/modeling/losses.py`. The training runtime coordinates model state, dataloading, distributed/single-GPU execution, checkpointing, and validation through `train.py` and `libs/utils/train_utils.py`; schedule policies are in `libs/utils/lr_schedulers.py`.

#### Inference and evaluation boundary
Evaluation loads a trained checkpoint, runs batched inference, decodes temporal proposals, suppresses duplicates, and computes benchmark metrics. Postprocessing and result formatting are handled by `libs/utils/postprocessing.py`; temporal NMS dispatch is in `libs/utils/nms.py`; mAP-style evaluation metrics are in `libs/utils/metrics.py`. `eval.py` is the command boundary for producing benchmark results.

#### Native/runtime dependencies
The repository requires Python, PyTorch (>=1.11 in documented experiments), CUDA, and package dependencies in `requirements.txt`. Performance-critical CPU NMS is a compiled extension from `libs/utils/csrc/nms_cpu.cpp`, installed through `libs/utils/setup.py`. External inputs are downloaded feature archives/checkpoints: ActionFormer-provided THUMOS14/ActivityNet/EPIC features, official HACS I3D or TCANet SlowFast features, and pretrained checkpoints hosted on Google Drive.

Main entry points: [`libs/datasets/thumos14.py`](https://github.com/dingfengshi/tridet/blob/master/libs/datasets/thumos14.py) (Dataset adapters), [`eval.py`](https://github.com/dingfengshi/tridet/blob/master/eval.py) (Evaluation command), [`libs/utils/train_utils.py`](https://github.com/dingfengshi/tridet/blob/master/libs/utils/train_utils.py) (Training runtime), [`libs/core/config.py`](https://github.com/dingfengshi/tridet/blob/master/libs/core/config.py) (Config loader)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_config["Configuration"]
  node_scripts["Experiment scripts<br/>shell wrappers"]
  node_configs["Experiment YAML<br/>configs<br/>[thumos_i3d.yaml]"]
  node_config_loader["Config loader<br/>configuration<br/>[config.py]"]
end

subgraph group_data["Feature data"]
  node_feature_archives[("Feature archives<br/>external input")]
  node_hacs_annotations["HACS annotations<br/>JSON annotations"]
  node_dataset_registry["Dataset interface<br/>dataset registry<br/>[datasets.py]"]
  node_dataset_adapters["Dataset adapters<br/>[thumos14.py]"]
end

subgraph group_model["Detection model"]
  node_model_factory["Model factory<br/>model registry<br/>[models.py]"]
  node_detector["Relative-boundary detector<br/>meta architecture<br/>[meta_archs.py]"]
  node_backbone["Temporal backbone<br/>feature extractor<br/>[backbones.py]"]
  node_neck["Multi-scale neck<br/>feature pyramid<br/>[necks.py]"]
  node_prediction_blocks["Prediction blocks<br/>relative-boundary layers<br/>[blocks.py]"]
  node_location_priors["Temporal priors<br/>location generator<br/>[loc_generators.py]"]
  node_losses["Detection losses<br/>loss functions<br/>[losses.py]"]
end

subgraph group_runtime["Training &amp; evaluation"]
  node_train_entry{{"Training command<br/>CLI entry point<br/>[train.py]"}}
  node_train_runtime["Training runtime<br/>training orchestration<br/>[train_utils.py]"]
  node_schedulers["Learning-rate schedules<br/>scheduler policies<br/>[lr_schedulers.py]"]
  node_eval_entry{{"Evaluation command<br/>CLI entry point<br/>[eval.py]"}}
  node_postprocessing["Proposal postprocessing<br/>result formatting<br/>[postprocessing.py]"]
  node_nms["Temporal NMS<br/>NMS dispatch<br/>[nms.py]"]
  node_metrics["mAP metrics<br/>benchmark metrics<br/>[metrics.py]"]
end

subgraph group_native["Native acceleration"]
  node_cpu_nms_extension["CPU NMS extension<br/>compiled C++ extension<br/>[nms_cpu.cpp]"]
  node_extension_build["Extension build<br/>package build<br/>[setup.py]"]
end

node_scripts -->|"launches"| node_train_entry
node_scripts -->|"launches"| node_eval_entry
node_configs -->|"loaded by"| node_config_loader
node_config_loader -->|"configures"| node_train_entry
node_config_loader -->|"configures"| node_eval_entry
node_feature_archives -->|"video features"| node_dataset_adapters
node_hacs_annotations -->|"segments"| node_dataset_adapters
node_dataset_registry -->|"selects"| node_dataset_adapters
node_dataset_adapters -->|"training batches"| node_train_runtime
node_dataset_adapters -->|"inference batches"| node_eval_entry
node_model_factory -->|"builds"| node_detector
node_detector -->|"extracts temporal features"| node_backbone
node_backbone -->|"feature sequences"| node_neck
node_neck -->|"multi-scale maps"| node_prediction_blocks
node_location_priors -->|"aligned locations"| node_detector
node_prediction_blocks -->|"training predictions"| node_losses
node_losses -->|"optimization loss"| node_train_runtime
node_train_entry -->|"runs"| node_train_runtime
node_train_runtime -->|"trains"| node_model_factory
node_schedulers -->|"learning-rate policy"| node_train_runtime
node_eval_entry -->|"loads checkpoint into"| node_model_factory
node_eval_entry -->|"decoded detections"| node_postprocessing
node_postprocessing -->|"candidate proposals"| node_nms
node_nms -->|"suppressed detections"| node_metrics
node_extension_build -->|"compiles"| node_cpu_nms_extension
node_cpu_nms_extension -.->|"accelerates"| node_nms

click node_scripts "https://github.com/dingfengshi/tridet/blob/master/tools/thumos_i3d_script.sh"
click node_configs "https://github.com/dingfengshi/tridet/blob/master/configs/thumos_i3d.yaml"
click node_config_loader "https://github.com/dingfengshi/tridet/blob/master/libs/core/config.py"
click node_hacs_annotations "https://github.com/dingfengshi/tridet/blob/master/data/hacs/annotations/HACS_segments_v1.1.1.json"
click node_dataset_registry "https://github.com/dingfengshi/tridet/blob/master/libs/datasets/datasets.py"
click node_dataset_adapters "https://github.com/dingfengshi/tridet/blob/master/libs/datasets/thumos14.py"
click node_model_factory "https://github.com/dingfengshi/tridet/blob/master/libs/modeling/models.py"
click node_detector "https://github.com/dingfengshi/tridet/blob/master/libs/modeling/meta_archs.py"
click node_backbone "https://github.com/dingfengshi/tridet/blob/master/libs/modeling/backbones.py"
click node_neck "https://github.com/dingfengshi/tridet/blob/master/libs/modeling/necks.py"
click node_prediction_blocks "https://github.com/dingfengshi/tridet/blob/master/libs/modeling/blocks.py"
click node_location_priors "https://github.com/dingfengshi/tridet/blob/master/libs/modeling/loc_generators.py"
click node_losses "https://github.com/dingfengshi/tridet/blob/master/libs/modeling/losses.py"
click node_train_entry "https://github.com/dingfengshi/tridet/blob/master/train.py"
click node_train_runtime "https://github.com/dingfengshi/tridet/blob/master/libs/utils/train_utils.py"
click node_schedulers "https://github.com/dingfengshi/tridet/blob/master/libs/utils/lr_schedulers.py"
click node_eval_entry "https://github.com/dingfengshi/tridet/blob/master/eval.py"
click node_postprocessing "https://github.com/dingfengshi/tridet/blob/master/libs/utils/postprocessing.py"
click node_nms "https://github.com/dingfengshi/tridet/blob/master/libs/utils/nms.py"
click node_metrics "https://github.com/dingfengshi/tridet/blob/master/libs/utils/metrics.py"
click node_cpu_nms_extension "https://github.com/dingfengshi/tridet/blob/master/libs/utils/csrc/nms_cpu.cpp"
click node_extension_build "https://github.com/dingfengshi/tridet/blob/master/libs/utils/setup.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_scripts,node_configs,node_config_loader toneBlue
class node_feature_archives,node_hacs_annotations,node_dataset_registry,node_dataset_adapters toneAmber
class node_model_factory,node_detector,node_backbone,node_neck,node_prediction_blocks,node_location_priors,node_losses toneMint
class node_train_entry,node_train_runtime,node_schedulers,node_eval_entry,node_postprocessing,node_nms,node_metrics toneRose
class node_cpu_nms_extension,node_extension_build toneIndigo
```

## Components

### Configuration

- [Experiment scripts](https://github.com/dingfengshi/tridet/blob/master/tools/thumos_i3d_script.sh): shell wrappers · Launch reproducible dataset/model runs. · `tools/thumos_i3d_script.sh`
- [Experiment YAML](https://github.com/dingfengshi/tridet/blob/master/configs/thumos_i3d.yaml): configs · Dataset, feature, model, optimization, and evaluation profiles. · `configs/thumos_i3d.yaml`
- [Config loader](https://github.com/dingfengshi/tridet/blob/master/libs/core/config.py): configuration · Loads and merges YAML settings. · `libs/core/config.py`

### Feature data

- **Feature archives**: external input · Downloaded pre-extracted video feature tensors.
- [HACS annotations](https://github.com/dingfengshi/tridet/blob/master/data/hacs/annotations/HACS_segments_v1.1.1.json): JSON annotations · `data/hacs/annotations/HACS_segments_v1.1.1.json`
- [Dataset interface](https://github.com/dingfengshi/tridet/blob/master/libs/datasets/datasets.py): dataset registry · Builds common feature and annotation datasets. · `libs/datasets/datasets.py`
- [Dataset adapters](https://github.com/dingfengshi/tridet/blob/master/libs/datasets/thumos14.py): dataset adapters · Load, normalize, and sample features with temporal labels. · `libs/datasets/thumos14.py`

### Detection model

- [Model factory](https://github.com/dingfengshi/tridet/blob/master/libs/modeling/models.py): model registry · Constructs the configured detection model. · `libs/modeling/models.py`
- [Relative-boundary detector](https://github.com/dingfengshi/tridet/blob/master/libs/modeling/meta_archs.py): meta architecture · Coordinates temporal prediction, matching, and inference. · `libs/modeling/meta_archs.py`
- [Temporal backbone](https://github.com/dingfengshi/tridet/blob/master/libs/modeling/backbones.py): feature extractor · `libs/modeling/backbones.py`
- [Multi-scale neck](https://github.com/dingfengshi/tridet/blob/master/libs/modeling/necks.py): feature pyramid · `libs/modeling/necks.py`
- [Prediction blocks](https://github.com/dingfengshi/tridet/blob/master/libs/modeling/blocks.py): relative-boundary layers · Produces class and temporal-boundary predictions. · `libs/modeling/blocks.py`
- [Temporal priors](https://github.com/dingfengshi/tridet/blob/master/libs/modeling/loc_generators.py): location generator · Generates locations aligned with multi-scale maps. · `libs/modeling/loc_generators.py`
- [Detection losses](https://github.com/dingfengshi/tridet/blob/master/libs/modeling/losses.py): loss functions · Matches predictions to segments and computes classification/localization loss. · `libs/modeling/losses.py`

### Training & evaluation

- [Training command](https://github.com/dingfengshi/tridet/blob/master/train.py): CLI entry point · `train.py`
- [Training runtime](https://github.com/dingfengshi/tridet/blob/master/libs/utils/train_utils.py): training orchestration · Handles dataloading, distributed execution, checkpoints, and validation. · `libs/utils/train_utils.py`
- [Learning-rate schedules](https://github.com/dingfengshi/tridet/blob/master/libs/utils/lr_schedulers.py): scheduler policies · `libs/utils/lr_schedulers.py`
- [Evaluation command](https://github.com/dingfengshi/tridet/blob/master/eval.py): CLI entry point · Loads a checkpoint and produces benchmark results. · `eval.py`
- [Proposal postprocessing](https://github.com/dingfengshi/tridet/blob/master/libs/utils/postprocessing.py): result formatting · Decodes proposals and formats detection results. · `libs/utils/postprocessing.py`
- [Temporal NMS](https://github.com/dingfengshi/tridet/blob/master/libs/utils/nms.py): NMS dispatch · Suppresses duplicate temporal detections. · `libs/utils/nms.py`
- [mAP metrics](https://github.com/dingfengshi/tridet/blob/master/libs/utils/metrics.py): benchmark metrics · `libs/utils/metrics.py`

### Native acceleration

- [CPU NMS extension](https://github.com/dingfengshi/tridet/blob/master/libs/utils/csrc/nms_cpu.cpp): compiled C++ extension · Performance-critical CPU NMS implementation. · `libs/utils/csrc/nms_cpu.cpp`
- [Extension build](https://github.com/dingfengshi/tridet/blob/master/libs/utils/setup.py): package build · `libs/utils/setup.py`

## Connections

- Experiment scripts → Training command: launches
- Experiment scripts → Evaluation command: launches
- Experiment YAML → Config loader: loaded by
- Config loader → Training command: configures
- Config loader → Evaluation command: configures
- Feature archives → Dataset adapters: video features
- HACS annotations → Dataset adapters: segments
- Dataset interface → Dataset adapters: selects
- Dataset adapters → Training runtime: training batches
- Dataset adapters → Evaluation command: inference batches
- Model factory → Relative-boundary detector: builds
- Relative-boundary detector → Temporal backbone: extracts temporal features
- Temporal backbone → Multi-scale neck: feature sequences
- Multi-scale neck → Prediction blocks: multi-scale maps
- Temporal priors → Relative-boundary detector: aligned locations
- Prediction blocks → Detection losses: training predictions
- Detection losses → Training runtime: optimization loss
- Training command → Training runtime: runs
- Training runtime → Model factory: trains
- Learning-rate schedules → Training runtime: learning-rate policy
- Evaluation command → Model factory: loads checkpoint into
- Evaluation command → Proposal postprocessing: decoded detections
- Proposal postprocessing → Temporal NMS: candidate proposals
- Temporal NMS → mAP metrics: suppressed detections
- Extension build → CPU NMS extension: compiles
- CPU NMS extension → Temporal NMS: accelerates