# microsoft/videox architecture

> Architecture diagram of the GitHub repository microsoft/videox, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-07-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/microsoft/videox
- Repository: https://github.com/microsoft/videox

## Overview

1. This repository is a monorepo of four mostly independent PyTorch research projects under the “VideoX” umbrella: 2D-TAN, MS-2D-TAN, SeqTrack, and X-CLIP. The top-level README is the only real cross-project entry point; each subdirectory has its own training, evaluation, configuration, and data layout.

2. The shared architectural pattern across all four projects is:
   - dataset-specific preprocessing and split files
   - YAML experiment/config files
   - a model library with modular subcomponents
   - training and testing entrypoints
   - benchmark-specific evaluation code
   This makes each paper implementation self-contained rather than sharing a single runtime.

3. 2D-TAN is the earliest moment-localization implementation. Its core pipeline is: text query + precomputed video features → temporal proposal map construction → 2D temporal adjacency modeling → scoring candidate moments. The code is organized around `lib/core` for config/engine/eval utilities, `lib/datasets` for ActivityNet/Charades-STA/TACoS loaders, and `lib/models` for frame, fusion, proposal, map, and loss modules. Training and test are launched from `moment_localization/train.py` and `moment_localization/test.py`.

4. 2D-TAN’s model boundary is modular: frame-level pooling lives in `frame_modules`, language/video fusion in `fusion_modules`, temporal proposal generation in `prop_modules`, and 2D convolution over the moment map in `map_modules`. The final network is assembled in `lib/models/tan.py`, which is the main model composition point.

5. MS-2D-TAN extends the 2D-TAN architecture to multiple temporal scales. Compared with 2D-TAN, it adds scale-aware proposal reasoning and extra prediction/fusion structure. The main additions are `clip_modules`, `pred_modules`, `multiscale` proposal logic, and `video_text_fusion`, while keeping the same overall data flow: query/video features in, multi-scale temporal map reasoning, localized moment outputs.

6. MS-2D-TAN keeps the same dataset family and evaluation problem as 2D-TAN, but its experiments are more varied through YAMLs for ActivityNet, Charades, and TACoS. It also includes utility scripts for visualization and dataset splitting, indicating a stronger emphasis on inspecting temporal heatmaps and benchmark-specific preprocessing.

7. SeqTrack is architecturally different: it is a visual tracking system, not a moment-localization system. Its core idea is sequence generation of bounding boxes using an encoder-decoder transformer. The primary model lives in `lib/models/seqtrack`, split into `encoder.py`, `decoder.py`, `seqtrack.py`, and `vit.py`, with utility code in `lib/utils` for boxes, positional embeddings, EMA/lr schedules, and tensor helpers.

8. SeqTrack has two parallel application surfaces:
   - `tracking/` for running and analyzing the SeqTrack tracker
   - `lib/train` and `lib/test` for the research training/evaluation framework
   This separation matters: training uses its own actor/trainer/data abstractions, while testing has benchmark-specific dataset wrappers and tracker entrypoints. The repo also includes `external/AR`, a large embedded dependency/fork of the pytracking/LTR ecosystem for evaluation and legacy tracking baselines.

9. SeqTrack’s data architecture is broader than the moment-localization projects. It supports image and video tracking datasets such as GOT-10k, LaSOT, TrackingNet, COCO, ImageNetVID, and VOT variants, with both native and LMDB-backed dataset variants. The presence of `data_specs`, LMDB writers, and transform utilities shows that data storage and sampling are first-class concerns, not just model code.

10. X-CLIP is a standalone video classification system built around CLIP-style vision-language pretraining. Its main flow is: video frames → image/video encoder and temporal cross-frame attention → text prompt generation for class names → fused video-text prediction. The core model code is in `models/xclip.py`, supported by `models/cct.py`, `models/mit.py`, and `models/prompt.py`, while the pretrained CLIP code and tokenizer are vendored under `clip/`.

11. X-CLIP’s data layer is classification-oriented rather than retrieval-oriented. It uses `datasets/build.py`, `pipeline.py`, `blending.py`, and `rand_augment.py` for video augmentation and dataset composition, with label CSVs for HMDB51, Kinetics-400/600, and UCF101. This is a clear boundary from the other projects: inputs are labeled video clips, not paired text queries or tracking sequences.

12. Configuration and experiment management are file-driven in all projects. The `experiments/` folders contain one YAML per paper setting or backbone variant, so architecture, dataset choice, and scale parameters are encoded outside the model code. This makes the repo diagram friendly: config files feed into training scripts, which instantiate dataset loaders and model modules.

13. External data and assets are important architectural dependencies. 2D-TAN/MS-2D-TAN rely on precomputed visual features and dataset JSON/CSV annotations; X-CLIP relies on CLIP weights and video-class labels; SeqTrack depends heavily on benchmark datasets and the external tracking framework. The repo is therefore more of an experimental system with dataset preparation steps than a single deployable application.

14. If you draw this repository as an architecture diagram, the main boundary should be between the four project subtrees. Inside each subtree, the dominant flow is: dataset/annotation preparation → configurable training/evaluation entrypoint → modular model components → benchmark-specific outputs and analysis. The largest shared theme is not code reuse, but a common research workflow for video understanding.

Main entry points: [`2D-TAN/lib/models/tan.py`](https://github.com/microsoft/videox/blob/master/2D-TAN/lib/models/tan.py) (TAN model), [`README.md`](https://github.com/microsoft/videox/blob/master/README.md) (Root README), [`2D-TAN/moment_localization/train.py`](https://github.com/microsoft/videox/blob/master/2D-TAN/moment_localization/train.py) (Train), [`MS-2D-TAN/moment_localization/run.py`](https://github.com/microsoft/videox/blob/master/MS-2D-TAN/moment_localization/run.py) (Run)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_root["VideoX monorepo"]
  node_readme["Root README<br/>entry point<br/>[README.md]"]
end

subgraph group_tan["2D-TAN"]
  node_tan_train["Train<br/>entrypoint<br/>[train.py]"]
  node_tan_test["Test<br/>entrypoint<br/>[test.py]"]
  node_tan_core["Core utils<br/>config/engine/eval"]
  node_tan_data["Datasets<br/>moment loaders"]
  node_tan_model["TAN model<br/>model assembly<br/>[tan.py]"]
  node_tan_frame["Frame pool<br/>frame encoder<br/>[frame_pool.py]"]
  node_tan_fusion["Fusion<br/>[base_fusion.py]"]
  node_tan_prop["Proposal gen<br/>temporal proposals"]
  node_tan_map["Map conv<br/>2D temporal map<br/>[map_conv.py]"]
  node_tan_loss["Loss<br/>[loss.py]"]
end

subgraph group_ms2d["MS-2D-TAN"]
  node_ms_train["Run<br/>entrypoint<br/>[run.py]"]
  node_ms_eval["Eval<br/>entrypoint<br/>[eval.py]"]
  node_ms_core["Core utils<br/>config/engine/eval"]
  node_ms_data["Datasets<br/>moment loaders"]
  node_ms_model["MS-TAN<br/>model assembly<br/>[TAN.py]"]
  node_ms_multiscale["Multi-scale<br/>proposal reasoning<br/>[multiscale.py]"]
  node_ms_fusion["Video-text fusion<br/>fusion module"]
  node_ms_pred["Predictors<br/>prediction heads<br/>[modules.py]"]
  node_ms_tools["Tools<br/>analysis scripts"]
end

subgraph group_seqtrack["SeqTrack"]
  node_seq_train["Train<br/>entrypoint<br/>[train_script.py]"]
  node_seq_test["Test<br/>entrypoint"]
  node_seq_track["Tracking<br/>app surface"]
  node_seq_model["SeqTrack model<br/>encoder-decoder<br/>[seqtrack.py]"]
  node_seq_encdec["Encoder/decoder<br/>transformer core"]
  node_seq_data["Tracking data<br/>datasets"]
  node_seq_specs["Data specs<br/>split files"]
  node_seq_external["AR fork<br/>external tracking stack"]
end

subgraph group_xclip["X-CLIP"]
  node_xclip_main["Main<br/>entrypoint<br/>[main.py]"]
  node_xclip_model["X-CLIP model<br/>video-text classifier<br/>[xclip.py]"]
  node_xclip_backbones["Vision backbones<br/>encoder modules"]
  node_xclip_clip["CLIP fork<br/>vendored CLIP"]
  node_xclip_data["Data pipeline<br/>classification datasets"]
  node_xclip_labels["Label CSVs<br/>class labels"]
end

node_readme -->|"documents"| node_tan_train
node_readme -->|"documents"| node_ms_train
node_readme -->|"documents"| node_seq_train
node_readme -->|"documents"| node_xclip_main
node_tan_train -->|"uses"| node_tan_core
node_tan_test -->|"uses"| node_tan_core
node_tan_core -->|"loads"| node_tan_data
node_tan_train -->|"builds"| node_tan_model
node_tan_model -->|"contains"| node_tan_frame
node_tan_model -->|"contains"| node_tan_fusion
node_tan_model -->|"contains"| node_tan_prop
node_tan_model -->|"contains"| node_tan_map
node_tan_train -->|"optimizes"| node_tan_loss
node_ms_train -->|"uses"| node_ms_core
node_ms_eval -->|"uses"| node_ms_core
node_ms_core -->|"loads"| node_ms_data
node_ms_train -->|"builds"| node_ms_model
node_ms_model -->|"contains"| node_ms_multiscale
node_ms_model -->|"contains"| node_ms_fusion
node_ms_model -->|"contains"| node_ms_pred
node_ms_train -->|"analyzes"| node_ms_tools
node_seq_train -->|"samples"| node_seq_data
node_seq_train -->|"optimizes"| node_seq_model
node_seq_model -->|"contains"| node_seq_encdec
node_seq_data -->|"driven by"| node_seq_specs
node_seq_test -->|"integrates"| node_seq_external
node_seq_track -->|"runs"| node_seq_model
node_seq_track -->|"extends"| node_seq_external
node_xclip_main -->|"builds"| node_xclip_data
node_xclip_main -->|"trains"| node_xclip_model
node_xclip_model -->|"uses"| node_xclip_backbones
node_xclip_model -->|"relies on"| node_xclip_clip
node_xclip_data -->|"aligns with"| node_xclip_labels

click node_readme "https://github.com/microsoft/videox/blob/master/README.md"
click node_tan_train "https://github.com/microsoft/videox/blob/master/2D-TAN/moment_localization/train.py"
click node_tan_test "https://github.com/microsoft/videox/blob/master/2D-TAN/moment_localization/test.py"
click node_tan_core "https://github.com/microsoft/videox/tree/master/2D-TAN/lib/core"
click node_tan_data "https://github.com/microsoft/videox/tree/master/2D-TAN/lib/datasets"
click node_tan_model "https://github.com/microsoft/videox/blob/master/2D-TAN/lib/models/tan.py"
click node_tan_frame "https://github.com/microsoft/videox/blob/master/2D-TAN/lib/models/frame_modules/frame_pool.py"
click node_tan_fusion "https://github.com/microsoft/videox/blob/master/2D-TAN/lib/models/fusion_modules/base_fusion.py"
click node_tan_prop "https://github.com/microsoft/videox/tree/master/2D-TAN/lib/models/prop_modules"
click node_tan_map "https://github.com/microsoft/videox/blob/master/2D-TAN/lib/models/map_modules/map_conv.py"
click node_tan_loss "https://github.com/microsoft/videox/blob/master/2D-TAN/lib/models/loss.py"
click node_ms_train "https://github.com/microsoft/videox/blob/master/MS-2D-TAN/moment_localization/run.py"
click node_ms_eval "https://github.com/microsoft/videox/blob/master/MS-2D-TAN/moment_localization/eval.py"
click node_ms_core "https://github.com/microsoft/videox/tree/master/MS-2D-TAN/lib/core"
click node_ms_data "https://github.com/microsoft/videox/tree/master/MS-2D-TAN/lib/datasets"
click node_ms_model "https://github.com/microsoft/videox/blob/master/MS-2D-TAN/lib/models/TAN.py"
click node_ms_multiscale "https://github.com/microsoft/videox/blob/master/MS-2D-TAN/lib/models/prop_modules/multiscale.py"
click node_ms_fusion "https://github.com/microsoft/videox/blob/master/MS-2D-TAN/lib/models/fusion_modules/video_text_fusion.py"
click node_ms_pred "https://github.com/microsoft/videox/blob/master/MS-2D-TAN/lib/models/pred_modules/modules.py"
click node_ms_tools "https://github.com/microsoft/videox/tree/master/MS-2D-TAN/tools"
click node_seq_train "https://github.com/microsoft/videox/blob/master/SeqTrack/lib/train/train_script.py"
click node_seq_test "https://github.com/microsoft/videox/tree/master/SeqTrack/lib/test/evaluation"
click node_seq_track "https://github.com/microsoft/videox/tree/master/SeqTrack/tracking"
click node_seq_model "https://github.com/microsoft/videox/blob/master/SeqTrack/lib/models/seqtrack/seqtrack.py"
click node_seq_encdec "https://github.com/microsoft/videox/tree/master/SeqTrack/lib/models/seqtrack"
click node_seq_data "https://github.com/microsoft/videox/tree/master/SeqTrack/lib/train/dataset"
click node_seq_specs "https://github.com/microsoft/videox/tree/master/SeqTrack/lib/train/data_specs"
click node_seq_external "https://github.com/microsoft/videox/tree/master/SeqTrack/external/AR"
click node_xclip_main "https://github.com/microsoft/videox/blob/master/X-CLIP/main.py"
click node_xclip_model "https://github.com/microsoft/videox/blob/master/X-CLIP/models/xclip.py"
click node_xclip_backbones "https://github.com/microsoft/videox/tree/master/X-CLIP/models"
click node_xclip_clip "https://github.com/microsoft/videox/tree/master/X-CLIP/clip"
click node_xclip_data "https://github.com/microsoft/videox/tree/master/X-CLIP/datasets"
click node_xclip_labels "https://github.com/microsoft/videox/tree/master/X-CLIP/labels"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_readme toneBlue
class node_tan_train,node_tan_test,node_tan_core,node_tan_data,node_tan_model,node_tan_frame,node_tan_fusion,node_tan_prop,node_tan_map,node_tan_loss toneAmber
class node_ms_train,node_ms_eval,node_ms_core,node_ms_data,node_ms_model,node_ms_multiscale,node_ms_fusion,node_ms_pred,node_ms_tools toneMint
class node_seq_train,node_seq_test,node_seq_track,node_seq_model,node_seq_encdec,node_seq_data,node_seq_specs,node_seq_external toneRose
class node_xclip_main,node_xclip_model,node_xclip_backbones,node_xclip_clip,node_xclip_data,node_xclip_labels toneIndigo
```

## Components

### VideoX monorepo

Top-level umbrella with four mostly independent research projects.

- [Root README](https://github.com/microsoft/videox/blob/master/README.md): entry point · Umbrella documentation and cross-project starting point. · `README.md`

### 2D-TAN

Moment localization project for text-conditioned temporal proposal reasoning.

- [Train](https://github.com/microsoft/videox/blob/master/2D-TAN/moment_localization/train.py): entrypoint · Launches 2D-TAN training. · `2D-TAN/moment_localization/train.py`
- [Test](https://github.com/microsoft/videox/blob/master/2D-TAN/moment_localization/test.py): entrypoint · Runs 2D-TAN evaluation. · `2D-TAN/moment_localization/test.py`
- [Core utils](https://github.com/microsoft/videox/tree/master/2D-TAN/lib/core): config/engine/eval · Shared config, engine, evaluation, and helper utilities. · `2D-TAN/lib/core`
- [Datasets](https://github.com/microsoft/videox/tree/master/2D-TAN/lib/datasets): moment loaders · ActivityNet, Charades-STA, and TACoS dataset adapters. · `2D-TAN/lib/datasets`
- [TAN model](https://github.com/microsoft/videox/blob/master/2D-TAN/lib/models/tan.py): model assembly · Main network composition for moment localization. · `2D-TAN/lib/models/tan.py`
- [Frame pool](https://github.com/microsoft/videox/blob/master/2D-TAN/lib/models/frame_modules/frame_pool.py): frame encoder · Frame-level pooling and feature preparation. · `2D-TAN/lib/models/frame_modules/frame_pool.py`
- [Fusion](https://github.com/microsoft/videox/blob/master/2D-TAN/lib/models/fusion_modules/base_fusion.py): query-video fusion · Combines language and visual features. · `2D-TAN/lib/models/fusion_modules/base_fusion.py`
- [Proposal gen](https://github.com/microsoft/videox/tree/master/2D-TAN/lib/models/prop_modules): temporal proposals · Dense and sparse temporal proposal generation. · `2D-TAN/lib/models/prop_modules`
- [Map conv](https://github.com/microsoft/videox/blob/master/2D-TAN/lib/models/map_modules/map_conv.py): 2D temporal map · 2D convolution over the moment map for candidate scoring. · `2D-TAN/lib/models/map_modules/map_conv.py`
- [Loss](https://github.com/microsoft/videox/blob/master/2D-TAN/lib/models/loss.py): training loss · Objective used for moment localization training. · `2D-TAN/lib/models/loss.py`

### MS-2D-TAN

Multi-scale extension of 2D-TAN for temporal localization.

- [Run](https://github.com/microsoft/videox/blob/master/MS-2D-TAN/moment_localization/run.py): entrypoint · Launches MS-2D-TAN experiments and evaluation. · `MS-2D-TAN/moment_localization/run.py`
- [Eval](https://github.com/microsoft/videox/blob/master/MS-2D-TAN/moment_localization/eval.py): entrypoint · Standalone evaluation workflow for multi-scale experiments. · `MS-2D-TAN/moment_localization/eval.py`
- [Core utils](https://github.com/microsoft/videox/tree/master/MS-2D-TAN/lib/core): config/engine/eval · Config, training engine, evaluation, and utilities. · `MS-2D-TAN/lib/core`
- [Datasets](https://github.com/microsoft/videox/tree/master/MS-2D-TAN/lib/datasets): moment loaders · Dataset adapters plus transforms for ActivityNet, Charades, and TACoS. · `MS-2D-TAN/lib/datasets`
- [MS-TAN](https://github.com/microsoft/videox/blob/master/MS-2D-TAN/lib/models/TAN.py): model assembly · Multi-scale TAN composition point. · `MS-2D-TAN/lib/models/TAN.py`
- [Multi-scale](https://github.com/microsoft/videox/blob/master/MS-2D-TAN/lib/models/prop_modules/multiscale.py): proposal reasoning · Scale-aware proposal construction and dense/sparse reasoning. · `MS-2D-TAN/lib/models/prop_modules/multiscale.py`
- [Video-text fusion](https://github.com/microsoft/videox/blob/master/MS-2D-TAN/lib/models/fusion_modules/video_text_fusion.py): fusion module · Enhanced query/video fusion for scale-aware localization. · `MS-2D-TAN/lib/models/fusion_modules/video_text_fusion.py`
- [Predictors](https://github.com/microsoft/videox/blob/master/MS-2D-TAN/lib/models/pred_modules/modules.py): prediction heads · Additional prediction and aggregation modules for multi-scale outputs. · `MS-2D-TAN/lib/models/pred_modules/modules.py`
- [Tools](https://github.com/microsoft/videox/tree/master/MS-2D-TAN/tools): analysis scripts · Heatmap visualization and testset splitting helpers. · `MS-2D-TAN/tools`

### SeqTrack

Transformer-based visual tracking system with training and benchmark evaluation.

- [Train](https://github.com/microsoft/videox/blob/master/SeqTrack/lib/train/train_script.py): entrypoint · Research training framework for SeqTrack. · `SeqTrack/lib/train/train_script.py`
- [Test](https://github.com/microsoft/videox/tree/master/SeqTrack/lib/test/evaluation): entrypoint · Benchmark evaluation and tracker testing framework. · `SeqTrack/lib/test/evaluation`
- [Tracking](https://github.com/microsoft/videox/tree/master/SeqTrack/tracking): app surface · Tracker execution, profiling, and analysis surface. · `SeqTrack/tracking`
- [SeqTrack model](https://github.com/microsoft/videox/blob/master/SeqTrack/lib/models/seqtrack/seqtrack.py): encoder-decoder · Main sequence-to-box model implementation. · `SeqTrack/lib/models/seqtrack/seqtrack.py`
- [Encoder/decoder](https://github.com/microsoft/videox/tree/master/SeqTrack/lib/models/seqtrack): transformer core · Encoder, decoder, and ViT backbone components. · `SeqTrack/lib/models/seqtrack`
- [Tracking data](https://github.com/microsoft/videox/tree/master/SeqTrack/lib/train/dataset): datasets · Training datasets, LMDB variants, sampling, and transforms. · `SeqTrack/lib/train/dataset`
- [Data specs](https://github.com/microsoft/videox/tree/master/SeqTrack/lib/train/data_specs): split files · Dataset split and class-map specifications. · `SeqTrack/lib/train/data_specs`
- [AR fork](https://github.com/microsoft/videox/tree/master/SeqTrack/external/AR): external tracking stack · Vendored pytracking/LTR ecosystem for legacy training and evaluation. · `SeqTrack/external/AR`

### X-CLIP

CLIP-style video classification system for action recognition.

- [Main](https://github.com/microsoft/videox/blob/master/X-CLIP/main.py): entrypoint · Top-level training/evaluation script. · `X-CLIP/main.py`
- [X-CLIP model](https://github.com/microsoft/videox/blob/master/X-CLIP/models/xclip.py): video-text classifier · CLIP-style video classification and prompt-fused prediction. · `X-CLIP/models/xclip.py`
- [Vision backbones](https://github.com/microsoft/videox/tree/master/X-CLIP/models): encoder modules · Supporting video/image encoder components and prompts. · `X-CLIP/models`
- [CLIP fork](https://github.com/microsoft/videox/tree/master/X-CLIP/clip): vendored CLIP · Pretrained CLIP implementation and tokenizer. · `X-CLIP/clip`
- [Data pipeline](https://github.com/microsoft/videox/tree/master/X-CLIP/datasets): classification datasets · Dataset building, blending, and video augmentation pipeline. · `X-CLIP/datasets`
- [Label CSVs](https://github.com/microsoft/videox/tree/master/X-CLIP/labels): class labels · Dataset label mappings for HMDB51, Kinetics, and UCF101. · `X-CLIP/labels`

## Connections

- Root README → Train: documents · Root docs point to the project entrypoints.
- Root README → Run: documents · Root docs point to the project entrypoints.
- Root README → Train: documents · Root docs point to the project entrypoints.
- Root README → Main: documents · Root docs point to the project entrypoints.
- Train → Core utils: uses · Training relies on shared config, engine, and evaluation helpers.
- Test → Core utils: uses · Testing relies on shared config, engine, and evaluation helpers.
- Core utils → Datasets: loads · Core runtime instantiates dataset adapters.
- Train → TAN model: builds · Training constructs the TAN network.
- TAN model → Frame pool: contains · Frame-level pooling is part of the model composition.
- TAN model → Fusion: contains · Language-video fusion is part of the model composition.
- TAN model → Proposal gen: contains · Temporal proposal generation is part of the model composition.
- TAN model → Map conv: contains · 2D moment map reasoning is part of the model composition.
- Train → Loss: optimizes · Training uses the localization loss.
- Run → Core utils: uses · Run script relies on shared config, engine, and evaluation helpers.
- Eval → Core utils: uses · Evaluation script relies on shared config, engine, and evaluation helpers.
- Core utils → Datasets: loads · Core runtime instantiates dataset adapters and transforms.
- Run → MS-TAN: builds · Experiments construct the multi-scale TAN network.
- MS-TAN → Multi-scale: contains · Multi-scale proposal reasoning is a central architectural addition.
- MS-TAN → Video-text fusion: contains · Video-text fusion is a central architectural addition.
- MS-TAN → Predictors: contains · Prediction heads refine localized outputs.
- Run → Tools: analyzes · Visualization and split scripts support experiment inspection.
- Train → Tracking data: samples · Training uses tracking dataset loaders and augmentation pipelines.
- Train → SeqTrack model: optimizes · Training uses the SeqTrack model.
- SeqTrack model → Encoder/decoder: contains · Encoder-decoder transformer and ViT backbone make up the model core.
- Tracking data → Data specs: driven by · Training data loading is guided by dataset split specs.
- Test → AR fork: integrates · Benchmark evaluation depends on the vendored AR/pytracking ecosystem.
- Tracking → SeqTrack model: runs · Tracker surface uses the SeqTrack model for analysis and inference.
- Tracking → AR fork: extends · Tracker tooling interoperates with the external AR stack.
- Main → Data pipeline: builds · Main script constructs classification datasets and pipelines.
- Main → X-CLIP model: trains · Main script trains or evaluates the X-CLIP model.
- X-CLIP model → Vision backbones: uses · Model combines video encoders and prompt modules.
- X-CLIP model → CLIP fork: relies on · Model depends on vendored CLIP components.
- Data pipeline → Label CSVs: aligns with · Pipeline is driven by dataset label CSVs.
